Subject: Re: draft-wbeebee-ipv6-cpe-router-04 comments

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 13 Jul 2009 13:23:22 +0200 (CEST)

On Sun, 5 Apr 2009, Mag Pat wrote:

Hi,

I know I am a bit late, but I read thru the 
http://tools.ietf.org/html/draft-ietf-v6ops-ipv6-cpe-router-00
and I have a question:

In section:

8.5. 6to4 Automated Tunneling (MEDIUM)/Dual-Stack Lite (DEV)/ISATAP (MEDIUM)

Would it make sense to put in a recommendation about doing MSS-adjust for 
packets traversing v6-in-v4 tunnels? I'm personally doing this today to 
avoid MTU related problems, and I think it's a better way than doing 
tunnel packet fragmentation (which is problematic as for instance our 6to4 
gateway has 4470 MTU on its core facing interfaces so any 1500 byte IPv6 
packet will be over 1500 when encapsulated in ipv6-in-ip, and then there 
might be a v4 packet fragmentation problem on the way to the tunnel 
endpoint and the packet might be dropped).

Setting MSS to 1350 (or other value substantially lower than what is 
required to pass unfragmented thru ip-in-ip tunnels) avoids this 
fragmentation problem. It doesn't fix UDP of course, but it would avoid 
performance and functionality problems in TCP (which in my experience make 
up the majority of Internet traffic).

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: "Templin, Fred L" <[email-masked]>
Date: Mon, 13 Jul 2009 05:53:58 -0700

Mikael,

As you say, MSS clamping can be applied to TCP the same
as for any link and in fact is a common operational
practice. However, TCP headers are not always available
in-the-clear, and as you say not all traffic is TCP.
IMHO, operators can already do MSS clamping w/o the
need for additional text in these documents.

The idea with setting the tunnel MTU is to set a size
that is highly unlikely to fragment, as sustained
fragmentation is dangerous in any case. An additional
alternative under development you may not be aware of
is SEAL, which fixes fragmentation to the point that
larger MTUs can be realized - even up to jumbogram
size if that is desired. See:
'draft-templin-intarea-seal'.

Fred
[email-masked]

> -----Original Message-----
> From: Mikael Abrahamsson [mailto:[email-masked]]
> Sent: Monday, July 13, 2009 4:23 AM
> To: [email-masked]
> Subject: Re: draft-wbeebee-ipv6-cpe-router-04 comments
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 13 Jul 2009 15:02:14 +0200 (CEST)

On Mon, 13 Jul 2009, Templin, Fred L wrote:

> As you say, MSS clamping can be applied to TCP the same as for any link 
> and in fact is a common operational practice. However, TCP headers are 
> not always available in-the-clear, and as you say not all traffic is 
> TCP. IMHO, operators can already do MSS clamping w/o the need for 
> additional text in these documents.

In the case of 6to4, a common platform to do this on is Cisco 7600 (it 
does it pretty much wirespeed), in that platform you cannot do MSS adjust. 
Therefore it would make more sense to do this in the CPE.

> The idea with setting the tunnel MTU is to set a size that is highly 
> unlikely to fragment, as sustained fragmentation is dangerous in any 
> case. An additional alternative under development you may not be aware 
> of is SEAL, which fixes fragmentation to the point that larger MTUs can 
> be realized - even up to jumbogram size if that is desired. See: 

Yes, I was a bit unclear, myself (since I control both ends of the tunnel 
(I don't use 6to4 but a statically configured ipv6-in-ip) I have actually 
set the MTU of this tunnel to 1350 which means this is the lowest common 
denominator for PMTUD (is my guess anyway) so this solves my "problem".

I still think it would make most sense to do the MSS clamping in the 6to4 
tunnel CPE because it knows for sure whether traffic is going to a tunnel 
or not.

---

From: "Wes Beebee (wbeebee)" <[email-masked]>
Date: Mon, 13 Jul 2009 10:56:03 -0400

What do you think of making MSS clamping a "MAY" in the CPE Router
document, while leaving it up to the vendor whether to make it default
or not?  Along with a note that this is one way to deal with PMTUD
problems with tunnels.

- Wes 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Mikael Abrahamsson
Sent: Monday, July 13, 2009 9:02 AM
To: [email-masked]
Subject: RE: draft-wbeebee-ipv6-cpe-router-04 comments

On Mon, 13 Jul 2009, Templin, Fred L wrote:

> As you say, MSS clamping can be applied to TCP the same as for any 
> link and in fact is a common operational practice. However, TCP 
> headers are not always available in-the-clear, and as you say not all 
> traffic is TCP. IMHO, operators can already do MSS clamping w/o the 

In the case of 6to4, a common platform to do this on is Cisco 7600 (it
does it pretty much wirespeed), in that platform you cannot do MSS
adjust. 
Therefore it would make more sense to do this in the CPE.

> of is SEAL, which fixes fragmentation to the point that larger MTUs 
> can be realized - even up to jumbogram size if that is desired. See:

Yes, I was a bit unclear, myself (since I control both ends of the
tunnel (I don't use 6to4 but a statically configured ipv6-in-ip) I have
actually set the MTU of this tunnel to 1350 which means this is the
lowest common denominator for PMTUD (is my guess anyway) so this solves
my "problem".

I still think it would make most sense to do the MSS clamping in the
6to4 tunnel CPE because it knows for sure whether traffic is going to a
tunnel or not.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 13 Jul 2009 17:23:10 +0200 (CEST)

> or not?  Along with a note that this is one way to deal with PMTUD 

That is perfectly fine for me.

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 14 Jul 2009 04:59:12 -0700

> From: Wes Beebee (wbeebee) [mailto:[email-masked]]
> Sent: Monday, July 13, 2009 7:56 AM
> To: Mikael Abrahamsson; [email-masked]

MSS clamping has been known for a long time, and in
fact is already implemented in some systems. However,
it is generally regarded as a band-aid fix. Indeed,
the IPv6/IPv4 tunneling mechanisms (e.g., RFC3056,
RFC4213, RFC5214, etc.) do not make a recommendation
for MSS clamping even after many years of deliberations
on ngtrans, v6ops and other discussion forums.

In addition to not working for non-TCP sessions and when
TCP headers are not available in the clear (e.g., IPsec),
MSS clamping also doesn't work when the initial packets
of a TCP session go through a different path and then
the session gets re-routed over the tunnel. For these
and other reasons, it therefore cannot be considered
as a generalized method for dealing with tunnel PMTUD
problems.

In short, I don't think this document should make a
recommendation for a solution that may or may not mask
one set of operational issues in the short term but will
almost certainly lead to a different set of operational
issues down the road. 

Fred

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 20 Jul 2009 18:07:01 +0200

On 13 jul 2009, at 15:02, Mikael Abrahamsson wrote:

>> The idea with setting the tunnel MTU is to set a size that is  
>> highly unlikely to fragment

That's not good enough. You have to get it right.

>> as sustained fragmentation is dangerous in any case.

It makes no sense to mess up IPv6 because of an IPv4 limitation. Let  
them eat fragments as an incentive to upgrade.

> I have actually set the MTU of this tunnel to 1350 which means this  
> is the lowest common denominator for PMTUD (is my guess anyway)

Why do you guess this?

There are people who set their MTU to 576 in IPv4 (not too many  
anymore, I hope) while the IPv6 minimum MTU is 1280. You can't win  
this by guessing.

Also, TCP MSS clamping is not the right solution to limit IPv6 packet  
sizes. We have the RA MTU option for that, you can use this to  
advertise the MTU for a subnet, which applies to TCP as well as other  
protocols.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 20 Jul 2009 18:23:30 +0200 (CEST)

On Mon, 20 Jul 2009, Iljitsch van Beijnum wrote:

>> I have actually set the MTU of this tunnel to 1350 which means this is the 
>> lowest common denominator for PMTUD (is my guess anyway)
>
> Why do you guess this?

Because the people who mess up usually have larger packet sizes than 1350, 
because the brokenness comes from tunnels. If I have a tunnel with 1350, 
the only likely even less MTU subnet I might come across is much more 
likely to get it right than the guys who try to run 1500 and get it wrong.

I do realise this isn't the right thing to do, but it works much better 
than doing nothing. So the question is, do we try to work around the 
brokenness or do we just do the right thing ourselves and call everybody 
else "broken" and sit there with a less working IPv6 than we could have.

> There are people who set their MTU to 576 in IPv4 (not too many anymore, I 
> hope) while the IPv6 minimum MTU is 1280. You can't win this by guessing.

I "win" more than if I stick to 1500 and experience all the brokenness.

So I think the question is, is this document about doing it right or 
getting it to work to the largest extent as possible?

---

From: Gert Doering <[email-masked]>
Date: Mon, 20 Jul 2009 18:53:28 +0200

Hi,

> I "win" more than if I stick to 1500 and experience all the brokenness.

Which might actually be considered the right way forward: experience the
brokenness, and *repair it while still possible*

I run my home network with MTU 1500 via an ADSL link with MTU 1492, and
this has caused me to find a few brokennesses in PMTU discovery already
- and most of them have been fixed now :-)

Gert Doering
-- 
Total number of prefixes smaller than registry allocations:  128645

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (89) 32356-444            USt-IdNr.: DE813185279

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 20 Jul 2009 20:38:50 +0200

> Because the people who mess up usually have larger packet sizes than  
> 1350

But why 1350 and not 1349 or 1351?

> because the brokenness comes from tunnels.

Brokenness comes from people filtering ICMP but still having PMTUD  
enabled.

> much more likely to get it right than the guys who try to run 1500  
> and get it wrong.

You confuse "works" with "right". That's exactly how the problem you  
try to solve started. While I can imagine operators displaying this  
level of pragmatism, we have to do better in the IETF.

> So the question is, do we try to work around the brokenness or do we  
> just do the right thing ourselves and call everybody else "broken"  
> and sit there with a less working IPv6 than we could have.

Going along with people who break the specifications is a road that  
leads nowehere fast.

The problem with setting an arbitrary packet size is not just that  
it's a race to the bottom (of course now someone runs a tunnel over a  
link with a 1350 MTU so you need to use a 1330 MTU etc etc) but also  
that there is no exit strategy. Even if you upgrade everything to use  
a link layer that supports bigger packets, you're still sending 1350- 
byte packets.

Now at least that last problem is solvable by not hardcoding "1350"  
but "link layer MTU - IPv4 (+IPsec+...) header".

> getting it to work to the largest extent as possible?

See the E in IETF.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 20 Jul 2009 21:06:28 +0200 (CEST)

> But why 1350 and not 1349 or 1351?

Why do you need a precise number? 1350 is low enough.

> Brokenness comes from people filtering ICMP but still having PMTUD enabled.

That is still not true, at least not at the ISP level. There are NAT/load 
balancers which do not handle PMTUD properly even if all ICMP is allowed, 
and ICMP messages won't be generated when there are MTU mismatches.

Just because people keep blaming stupid admins for blocking ICMP doesn't 
mean it's the whole truth, there are plenty of manufacturers who get it 
wrong, and ISPs that misconfigure their devices (or the devices don't 
handle PMTUD/fragmentation properly at all).

> You confuse "works" with "right". That's exactly how the problem you try 
> to solve started. While I can imagine operators displaying this level of 
> pragmatism, we have to do better in the IETF.

Doing the "right" thing might be "right", but that doesn't mean it'll work 
in real life.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 20 Jul 2009 21:45:33 +0200

> That is still not true, at least not at the ISP level. There are NAT/ 
> load balancers which do not handle PMTUD properly even if all ICMP  
> is allowed, and ICMP messages won't be generated when there are MTU  
> mismatches.

That's no excuse. You can easily turn off PMTUD or have too bigs  
generated elsewhere in the network, PMTUD black holes are 100%  
avoidable.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Mon, 20 Jul 2009 22:21:41 +0200 (CEST)

> generated elsewhere in the network, PMTUD black holes are 100% 

In the E in IETF it's avoidable, yes.

In the real world, sorry, not always. Also, we're not very good at 
providing tools for testing this so that people actually know where and 
why it's a problem, and we're not good at inventing mechanisms to handle 
the imperfect world.