Subject: [Int-area] draft-bonica-intarea-frag-fragile-01

From: Ron Bonica <[email-masked]>
Date: Mon, 5 Mar 2018 14:16:59 +0000

Folks,

Please review draft-bonica-intarea-frag-fragile-01 and provide comments. The URL is https://tools.ietf.org/html/draft-bonica-intarea-frag-fragile-01.

                                                                                       Ron

---

From: Joe Touch <[email-masked]>
Date: Mon, 5 Mar 2018 07:01:24 -0800

This doc completely overlooks the role of fragmentation in IP over IP tunneling and the reason fragmentation is critical (IP has a maximum packet size, not just a minimum).

See draft-ietf-intarea-tunnels.

As a result, IMO the recommendations and conclusions are incorrect.

Joe

> On Mar 5, 2018, at 6:16 AM, Ron Bonica <[email-masked]> wrote:
> 
> Folks,
> 
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area

---

From: Ron Bonica <[email-masked]>
Date: Mon, 5 Mar 2018 15:42:51 +0000

Hi Joe,

The draft does talk about a case where outer fragmentation is required. This is a problem that remains to be solved.

Please remember that this is a very new draft. It doesn't claim to have all of the answers, yet. Let's chat in London.

                                                                                         Ron

> -----Original Message-----
> From: Joe Touch [mailto:[email-masked]]
> Sent: Monday, March 5, 2018 10:01 AM
> To: Ron Bonica <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [Int-area] draft-bonica-intarea-frag-fragile-01
> 
> This doc completely overlooks the role of fragmentation in IP over IP
> tunneling and the reason fragmentation is critical (IP has a maximum packet
> size, not just a minimum).
> 
> > Please review draft-bonica-intarea-frag-fragile-01 and provide comments.
> The URL is https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__tools.ietf.org_html_draft-2Dbonica-2Dintarea-2Dfrag-2Dfragile-
> 2D01&d=DwIFAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-
> ndb3voDTXcWzoCI&r=Fch9FQ82sir-BoLx84hKuKwl-
> AWF2EfpHcAwrDThKP8&m=gSNpTcaRgeLalrZ3qiQd75blm0UVJ8xCMRqRFbX
> UqgQ&s=wk0DX-Xnmbk2xXTQPuWpTkC76wT_PQRvDMvbe2UjlEE&e=.
> >
> >                                                                                       Ron
> >
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_int-
> 2Darea&d=DwIFAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-
> ndb3voDTXcWzoCI&r=Fch9FQ82sir-BoLx84hKuKwl-
> AWF2EfpHcAwrDThKP8&m=gSNpTcaRgeLalrZ3qiQd75blm0UVJ8xCMRqRFbX
> UqgQ&s=IgKKm8ZzwQrQquREE1eEzDHxQyIpdXLba8-kYLj-Dzc&e=

---

From: Joe Touch <[email-masked]>
Date: Mon, 5 Mar 2018 07:57:30 -0800

Hi, Ron,

I will not be in London.

Joe

> On Mar 5, 2018, at 7:42 AM, Ron Bonica <[email-masked]> wrote:
> 
> Hi Joe,
>

---

From: Bob Hinden <[email-masked]>
Date: Mon, 5 Mar 2018 08:46:34 -0800

Joe,

> On Mar 5, 2018, at 7:01 AM, Joe Touch <[email-masked]> wrote:
> 

Good point.  We will take a look at intarea-tunnels and look at some changes to the fragmentation draft.

Bob

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 6 Mar 2018 10:57:22 +0100 (CET)

On Mon, 5 Mar 2018, Ron Bonica wrote:

> The URL is 
> https://tools.ietf.org/html/draft-bonica-intarea-frag-fragile-01.

I like it.

4.6. There are cases where this "misconfiguration" is due to vendor 
default not being changed. I do not equate "misconfiguration" with "didn't 
change default configuration". Some others might. It might also be due to 
"hardware limitation". Generally, I do not like the "filtering" (I have 
opposed to this in other drafts), as for me "filtering" conveys intent. If 
there is no intent, there is no filtering, but instead there is "dropping" 
or some other word.

4.7 Can we please have 4.7 that describes cases where ICMP PTB are never 
emitted because of misconfiguration? For instance intermediate L2 switch 
that has lower MTU than the L3 nodes connected to it, or mismatched 
MTU/MRU on two nodes connected to each other.

5.1. Can we have some kind of strong recommendation that hosts enable 
PLMTUD for TCP?

6. "IP encapsulations". Shouldn't this be "some packet-in-packet 
encapsulations"? Or does "IP encapsulations" mean "anything encapsulated 
in IP"? 6.3 talks about this as well, I think it's worthwile to put in a 
sentence that whatever is said in this document, probably applies to all 
kinds of encapsulations.

6.1. Err, last paragraph, aren't we getting ahead of ourselves here? I 
guess this is because of Geoff Hustons claims? That last paragraph is in 
dispute (I'd say, from talking to other people involved in DNS).

7.2. I strongly believe we need more text here. It should be something 
along the lines of:

"As per RFC4890, network operators MUST assure proper operation of PMTUD 
by making sure that PTB packets are emitted by all equipment when it can't 
fit a packet into a smaller MTU link, and that large MTU packets are not 
silently discarded due to misconfiguration. Network operators MUST NOT 
filter ICMP PTB packets."

...

As a last comment, do we know documents that tell application developers 
how to do what this document recommends in 5.2? If someone developers 
applications that use UDP for instance, how do they know what the 
operating system PMTUD is at any given time, to avoid the host stack 
fragmenting the packet? I've been interacting with people who had this 
specific problem, and it wasn't easy to figure out exactly how to do what 
is being said in this text (which I agree should be done).

Generally, I think the IETF should strongly recommend application/protocol 
developers to not rely on IP fragmentation, generally. So the ones listed 
in 6 (and I imagine there are more of them), should change the way the 
protocol is done. This includes DNS. So all working groups should be put 
on notice to start working on this problem if they don't already have a 
solution for it.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Joe Touch <[email-masked]>
Date: Tue, 6 Mar 2018 06:38:40 -0800

You might also want to look at tsvwg-udp-options

In one case, they’re used to support PLPMTUD (which you already note).

Alternatively, they provide transport-layer frag/reassembly that might be useful for DNSSEC as well as enabling NAT-traversal by retaining ports across fragments.

Joe

> On Mar 5, 2018, at 8:46 AM, Bob Hinden <[email-masked]> wrote:
> 
> Joe,
>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 6 Mar 2018 08:02:44 -0800

> Please review draft-bonica-intarea-frag-fragile-01 and provide comments. The URL is https://tools.ietf.org/html/draft-bonica-intarea-frag-fragile-01.
Ron,

This draft should reference RFC4459 which gives a very good
description of MTU and fragmentation problems and handling for network
tunnels.

GUE could be mentioned in the "IP encapsulations" sections. As
proposed by RFC4459, GUE extensions include a fragmentation option
within the encapsulation headers in lieu of doing IP fragmentation.
This is addresses several of the problems described for IP
fragmentation.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 6 Mar 2018 09:38:40 -0800

Agreed but note that draft tunnels will update that RFC in some important ways. 

Joe

> On Mar 6, 2018, at 8:02 AM, Tom Herbert <[email-masked]> wrote:
>

---

From: Ole Troan <[email-masked]>
Date: Tue, 6 Mar 2018 20:16:38 +0100

Joe,

> Agreed but note that draft tunnels will update that RFC in some important ways.

With other concerns than those raised in e.g. 4459 and 7597?
Unfortunately there are cases where there are no other choice than to do fragmentation/reassembly on tunnel endpoints, but still the recommendation holds.
It is so problematic, that it is strongly recommended to engineer the network to avoid that happening.

Cheers,
Ole

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 6 Mar 2018 19:40:55 +0000

I think it is important to remember that this draft is about *IP* layer
fragmentation. Tunnels can employ tunnel-layer fragmentation at a
layer above IP:

https://datatracker.ietf.org/doc/draft-ietf-intarea-gue-extensions/
https://datatracker.ietf.org/doc/draft-templin-intarea-grefrag/

Or, if the draft intends to also cover tunnel-layer fragmentation it
should probably be updated to say so. 

Thanks - Fred

> From: Int-area [mailto:[email-masked]] On Behalf Of Ole Troan
> Sent: Tuesday, March 06, 2018 11:17 AM
> To: Joe Touch <[email-masked]>
> Cc: [email-masked]; Ron Bonica <[email-masked]>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 6 Mar 2018 11:41:34 -0800

> It is so problematic, that it is strongly recommended to engineer the network to avoid that happening.
Hi Ole,

That may be recommendation, but that is not was has been done all
networks. I worked in two very large datacenter networks that use a
lot encapsulation, but they didn't do anything special to engineer the
network for encapsulation. There was reluctance to set an MTU below
1500 and jumbo frames were always discussed, but in the end they
deploying them seem to be more complicated than it was worth. So
fragmentation did occur pretty regularly at tunnel ingress.

Fragmentation never popped up as a problem because of some mitigating
factors: 1) packets from the Internet be encapsulated usually were
already less than 1500 bytes(when encapsulating for VIPs) 2) This
always generated precisely two fragments 3) encapsulation is being
done on a closed network so there is no chance of DOS attack on the
reassembly cache (fragments from the Internet are dropped) 4)
encapsulation is otherwise rare relative non-encapsulation, so
lowering the MTU would only benefits the special case and hurts the
common case.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 6 Mar 2018 18:57:23 -0800

> On Mar 6, 2018, at 11:16 AM, Ole Troan <[email-masked]> wrote:
> 
> Joe,
> 

draft-tunnels corrects an error in 4459 that deals with the details, not the overall recommendation (AFAIR, at least).

> It is so problematic, that it is strongly recommended to engineer the network to avoid that happening.

IMO, there are two truths:

1) use of IP fragmentation SHOULD be avoided where possible, largely because it has reliability issues (ICMP blocking, NATs won’t tunnel frags and fail to [as required if they act on transport info] reassemble, etc.)

2) support for IP fragmentation MUST remain required, as MUST (IMO) NAT reassembly before transport rewriting

Yeah, I know a lot of devices fail the MUSTs in #2, but the requirements ought to set the goal, not describe the (sorry) current state.

#2 has to persist until we deprecate IP-in-IP tunneling (including tunnel-mode IPsec), as well as any IP-in-X*-in-IP for zero or more intermediate layers X where no layer supports fragmentation and reassembly

I’ve been working to fix the need for IP frag by developing support for that in UDP, but it doesn’t mean we should be ready to outlaw it.

I’m not sure what this doc does to add to this scene, though - it might be useful if the authors could explain how it affects 1 and 2 above and what else it adds in a *brief* post.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 7 Mar 2018 15:30:18 +0000

Hi Tom,

> From: Int-area [mailto:[email-masked]] On Behalf Of Tom Herbert
> Sent: Tuesday, March 06, 2018 8:03 AM
> proposed by RFC4459,

RFC4459 does not propose tunnel fragmentation (i.e., fragmentation
at a layer above IP but below the transport protocol).

> GUE extensions include a fragmentation option

Tunnel fragmentation was first discussed in RFC2764, then later by
me in RFC5320 and in all of my subsequent writings, which is how
it found its way into GUE extensions.

Thanks - Fred
[email-masked]

---

From: Ron Bonica <[email-masked]>
Date: Wed, 7 Mar 2018 15:39:47 +0000

Joe,

Your "Two Truths" are in line with the recommendations in Section 7 of draft-bonica-intarea-frag-fragile-01. The draft recommends that upper-layer protocols avoid doing things that cause fragmentation. It does not recommend the deprecation of fragmentation.

                                                                                   Ron

.
> Sent: Tuesday, March 6, 2018 9:57 PM
> To: Ole Troan <[email-masked]>
> Cc: Tom Herbert <[email-masked]>; Ron Bonica
> <[email-masked]>; [email-masked]
> draft-tunnels corrects an error in 4459 that deals with the details, not the
> overall recommendation (AFAIR, at least).
> 
> > Unfortunately there are cases where there are no other choice than to do
> fragmentation/reassembly on tunnel endpoints, but still the
> recommendation holds.
> 1) use of IP fragmentation SHOULD be avoided where possible, largely
> because it has reliability issues (ICMP blocking, NATs won’t tunnel frags and
> fail to [as required if they act on transport info] reassemble, etc.)
> 
> Yeah, I know a lot of devices fail the MUSTs in #2, but the requirements
> ought to set the goal, not describe the (sorry) current state.
> 
> #2 has to persist until we deprecate IP-in-IP tunneling (including tunnel-
> mode IPsec), as well as any IP-in-X*-in-IP for zero or more intermediate
> layers X where no layer supports fragmentation and reassembly
> 
> I’ve been working to fix the need for IP frag by developing support for that in
> UDP, but it doesn’t mean we should be ready to outlaw it.
> 
> I’m not sure what this doc does to add to this scene, though - it might be
> useful if the authors could explain how it affects 1 and 2 above and what else
> it adds in a *brief* post.
> 
> Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 7 Mar 2018 07:46:38 -0800

On Mar 7, 2018 7:30 AM, "Templin, Fred L" <[email-masked]> wrote:

Hi Tom,

> > Please review draft-bonica-intarea-frag-fragile-01 and provide
comments. The URL is https://tools.ietf.org/html/draft-bonica-
> intarea-frag-fragile-01.

Yes, RFC4459 suggests the possibility of fragmenting the inner IP packet,
but that is not possible if it's IPv6. Fragmentation with the encapsulation
doesn't have that limitation.

Thanks for the clarification.

Tom

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Wed, 7 Mar 2018 08:52:12 -0800

> On Mar 7, 2018, at 7:39 AM, Ron Bonica <[email-masked]> wrote:
> 
> Joe,
> 

Understood, but without #2 being included and explicitly co-reinforced there is an implication to router designers that I would hope can be avoided.  

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 7 Mar 2018 08:56:52 -0800

Also IMO the lack of frag support should be called out as a bug, not merely acknowledged or (worse) commended. Reassembly by middle boxes should be called out as well - or by any DPI (or the equivalent, I e by reassembling a copy used for frag processing context.)

Joe

> On Mar 7, 2018, at 8:52 AM, Joe Touch <[email-masked]> wrote:
> 
> 
>

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 7 Mar 2018 17:13:03 +0000

Joe,

About middle-box reassembly,  it should probably also mention Virtual Fragment
Reassembly where the middlebox gathers fragments but does not reassemble
them. Then, when all fragments have been received the middlebox performs
any transformations then releases the fragments. Several cisco webpages talk
about this.

Thanks - Fred

> From: Int-area [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Wednesday, March 07, 2018 8:52 AM
> > Your "Two Truths" are in line with the recommendations in Section 7 of draft-bonica-intarea-frag-fragile-01. The draft recommends
> that upper-layer protocols avoid doing things that cause fragmentation. It does not recommend the deprecation of fragmentation.
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 7 Mar 2018 09:33:39 -0800

> On Mar 7, 2018, at 9:13 AM, Templin, Fred L <[email-masked]> wrote:
> 
> Joe,
> 
> them.

That’s part of what I was referring to in my other post. 

Joe

> Then, when all fragments have been received the middlebox performs