Subject: Is Fragmentation at IP layer even needed ?

From: Alexey Eromenko <[email-masked]>
Date: Sun, 7 Feb 2016 14:47:21 +0200

Hi All,

I'm re-evaluating TCP/IP stack again with my ongoing IP-FF research.

My question: Is packet fragmentation at IP layer even needed ?

Basically here are few possibilities:

1. Fragmentation-and-reassembly at every hop. (I don't know if anybody
implements it)
2. IPv4 style-fragmentation -- fragmentation per every hop, reassembly at
destination end.
3. IPv6-style-fragmentation -- fragmentation only at source end, reassembly
at destination end.
4. No fragmentation at all (the advantage here: faster Router processing vs
#1 or #2 and less implementation bugs); Assuming standard packet size is
defined at 1280 bytes, like in IPv6
5. MTU path discovery via ICMP -- RFC-1981
6. MTU path discovery via TCP (or other Transport) -- RFC-4821 (or other
way)

I'm leaning towards 4 + 6 solution in my own protocol, IP-FF.
What do you think ?
Should IP layer provide fragmentation ?

-- 
-Alexey Eromenko "Technologov"

---

From: Yoav Nir <[email-masked]>
Date: Sun, 7 Feb 2016 17:07:09 +0200

> On 7 Feb 2016, at 2:47 PM, Alexey Eromenko <[email-masked]> wrote:
> 
> Hi All,
> 

Short answer: yes. 

> 1. Fragmentation-and-reassembly at every hop. (I don't know if anybody implements it)

This is the Internet. Nobody controls every hop. There can be devices who do this, but it does not force other devices on the path to do this. This would be slower than passing the fragments as they are, so routers typically don’t do this unless they’re modifying the packet. Our VPN gateway may or may not re-assemble and re-fragment when doing IPsec for IPv4, depending on configuration.

> 2. IPv4 style-fragmentation -- fragmentation per every hop, reassembly at destination end.

Definitely implemented widely.

> 3. IPv6-style-fragmentation -- fragmentation only at source end, reassembly at destination end.

Definitely implemented by everyone who wants to be compliant with IPv6.

> 4. No fragmentation at all (the advantage here: faster Router processing vs #1 or #2 and less implementation bugs); Assuming standard packet size is defined at 1280 bytes, like in IPv6
> 6. MTU path discovery via TCP (or other Transport) -- RFC-4821 (or other way)

And what if you’re using anything other than TCP/SCTP?  If your application is UDP and it is sending a 3000-byte payload you have to do the fragmentation at some layer. 

> Should IP layer provide fragmentation ?

Some layer must, and assuming backwards compatibility with existing transport and higher layers is required, I don’t see how an IP layer without fragmentation would work.

Yoav

---

From: Alexey Eromenko <[email-masked]>
Date: Mon, 8 Feb 2016 13:11:46 +0200

1. IP v6 fragmentation is limited from 1280 bytes up to 1500 bytes. Any
higher than that is optional to implement,  and may not work.
2. What kind of UDP applications use such big packets, over 1280 bytes ?
On Feb 7, 2016 5:07 PM, "Yoav Nir" <[email-masked]> wrote:

> This is the Internet. Nobody controls every hop. There can be devices who
> do this, but it does not force other devices on the path to do this. This
> would be slower than passing the fragments as they are, so routers
> typically don’t do this unless they’re modifying the packet. Our VPN
> gateway may or may not re-assemble and re-fragment when doing IPsec for
> IPv4, depending on configuration.
>
> > 4. No fragmentation at all (the advantage here: faster Router processing
> vs #1 or #2 and less implementation bugs); Assuming standard packet size is
> And what if you’re using anything other than TCP/SCTP?  If your
> application is UDP and it is sending a 3000-byte payload you have to do the
> fragmentation at some layer.
>
> >
> Some layer must, and assuming backwards compatibility with existing
> transport and higher layers is required, I don’t see how an IP layer
> without fragmentation would work.
>
> Yoav
>
>

---

From: Yoav Nir <[email-masked]>
Date: Mon, 8 Feb 2016 13:58:14 +0200

IKE

Just as an example

> On 8 Feb 2016, at 1:11 PM, Alexey Eromenko <[email-masked]> wrote:
> 
> 1. IP v6 fragmentation is limited from 1280 bytes up to 1500 bytes. Any higher than that is optional to implement,  and may not work. 
> On Feb 7, 2016 5:07 PM, "Yoav Nir" <[email-masked] <mailto:[email-masked]>> wrote:
> 
> > On 7 Feb 2016, at 2:47 PM, Alexey Eromenko <[email-masked] <mailto:[email-masked]>> wrote:
> >
> > Hi All,
> >

---

From: "John Levine" <[email-masked]>
Date: 8 Feb 2016 15:52:14 -0000

>2. What kind of UDP applications use such big packets, over 1280 bytes ?

Um, the DNS with EDNS0, and particularly with DNSSEC?  Is this a trick question?

R's,
John

---

From: Alexey Eromenko <[email-masked]>
Date: Mon, 8 Feb 2016 18:08:16 +0200

> >Um, the DNS with EDNS0, and particularly with DNSSEC?  Is this a trick
Yes.
The trick lies right there in IPv6 specification:

"A node must be able to accept a fragmented packet that, after
   reassembly, is as large as 1500 octets."
... which is not much greater than 1280 bytes defined as minimum MTU for
IPv6.

Basically, Fragmentation, as specified in RFC-2460 (IPv6), solves the
problem ONLY for packet-sized between 1280 and 1500 bytes. Which is why I
doubt if it is useful at all.

(I also understand, that IPv4 fragmentation is much more flexible, yes. It
can go as small as 68 byte packets.)

---

From: David Borman <[email-masked]>
Date: Mon, 8 Feb 2016 11:05:02 -0600

> On Feb 8, 2016, at 10:08 AM, Alexey Eromenko <[email-masked]> wrote:
> 
> ... which is not much greater than 1280 bytes defined as minimum MTU for IPv6.
> 
> Basically, Fragmentation, as specified in RFC-2460 (IPv6), solves the problem ONLY for packet-sized between 1280 and 1500 bytes. Which is why I doubt if it is useful at all.

That’s the minimum required implementation, which guarantees that you can send at least a basic ethernet sized packet to any host.  What is omitted in the above qoute is next 2 sentences:
   "A node is permitted to
   accept fragmented packets that reassemble to more than 1500 octets.
   An upper-layer protocol or application that depends on IPv6
   fragmentation to send packets larger than the MTU of a path should
   not send packets larger than 1500 octets unless it has assurance that
   the destination is capable of reassembling packets of that larger
   size.”
So if you are writing an application that needs >1500 octets, use an IPv6 implementation that supports >1500 octet fragmentation and reassembly.

		-David Borman

> 
> (I also understand, that IPv4 fragmentation is much more flexible, yes. It can go as small as 68 byte packets.)

---

From: Ronald Bonica <[email-masked]>
Date: Mon, 8 Feb 2016 17:21:30 +0000

Hi Alexey,

This question comes up every few years. The short answer is:

-          The vast majority of Internet traffic rides over TCP or UDP

-          Generally speaking, traffic that rides over TCP does not rely on IP fragmentation

-          However, traffic the rides over UDP absolutely relies on IP fragmentation

So, as things stand, IP fragmentation is required to support UDP. However, the conversation doesn’t end at that.

Operational experience has taught us that IPv6 fragmentation does not work so well. Unlike IPv4, IPv6 encodes fragmentation information in an IPv6 extension header. Sadly, many operators discard packets containing that extension header. So, as specified, IPv6 provides fragmentation services, but as deployed, it does not.

Many end-stations work around this problem by sending packets no longer that the IPv6 minimum MTU (1280 bytes). This ensures that IPv6 fragmentation services will never be required. However, it also prevents applications and networks from realizing the benefits of larger packets.

So, the internet community has the following options:

1)      Live with the status quo / Send only packets < 1280 bytes

a.       Say nothing in the standards about the issue, beyond what has already been said

b.      Write an RFC informing developers of UDP applications of the problem and advising them not to rely on protocol MTU > 1280

c.       Deprecate IPv6 fragmentation

2)      Fix the problem / Allow end-stations to send larger packets

a.       Convince operators not to drop fragmented packets

b.      Design a UDP replacement that provides fragmentation service and migrate UPD applications to the replacement protocol

Options 2a and 2b may not achievable, because they require action on the part of many, many parties. So, we seem to be stuck with Options 1a, 1b and 1c.

In light of this, your original question is not only appropriate, it is telling.

                                                                                                        Ron

From: ietf [mailto:[email-masked]] On Behalf Of Alexey Eromenko
Sent: Sunday, February 07, 2016 7:47 AM
To: ietf <[email-masked]>
Subject: Is Fragmentation at IP layer even needed ?

Hi All,

---

From: Warren Kumari <[email-masked]>
Date: Mon, 08 Feb 2016 18:23:20 +0000

On Mon, Feb 8, 2016 at 9:05 AM David Borman <[email-masked]> wrote:

> That’s the minimum required implementation, which guarantees that you can
> send at least a basic ethernet sized packet to any host.  What is omitted
> in the above qoute is next 2 sentences:
>    "A node is permitted to
> So if you are writing an application that needs >1500 octets, use an IPv6
> implementation that supports >1500 octet fragmentation and reassembly.
>

... but as an application writer (or, basically anyone else), I have no
control over the "IPv6 implementation". Even if I'm in an environment where
I do control the OS / model of all devices, and I know they support >1500
octet, it seems like a bad idea to *rely* on that. Sometime I'm going to
want to change OS / add some other device, be able to interact with some
other system. This sounds like vendor lock at its worst...

W

---

From: David Borman <[email-masked]>
Date: Mon, 8 Feb 2016 12:56:57 -0600

> On Feb 8, 2016, at 12:23 PM, Warren Kumari <[email-masked]> wrote:
...
> On Mon, Feb 8, 2016 at 9:05 AM David Borman <[email-masked]> wrote:
...
> ... but as an application writer (or, basically anyone else), I have no control over the "IPv6 implementation". Even if I'm in an environment where I do control the OS / model of all devices, and I know they support >1500 octet, it seems like a bad idea to *rely* on that. Sometime I'm going to want to change OS / add some other device, be able to interact with some other system. This sounds like vendor lock at its worst…

If you wind up in a scenario where you get locked to a particular OS vendor because it’s the only one that supports IPv6 fragmentation >1500 octets, then that is probably the least of your worries.  I’d be much more worried about IPv6 fragmentation in light of Ron Bonica’s comment that intermediary nodes drop packets with extension headers, which is bad news even for fragmented packets in the 1280-1500 range.

			-David

---

From: Mark Andrews <[email-masked]>
Date: Tue, 09 Feb 2016 06:54:41 +1100

In message <[email-masked]>, David Borm
an writes:
> > So if you are writing an application that needs >1500 octets, use an
> IPv6 implementation that supports >1500 octet fragmentation and
> reassembly.
> >
> control over the "IPv6 implementation". Even if I'm in an environment
> where I do control the OS / model of all devices, and I know they support
> >1500 octet, it seems like a bad idea to *rely* on that. Sometime I'm
> going to want to change OS / add some other device, be able to interact
> with some other system. This sounds like vendor lock at its worst…
>
> If you wind up in a scenario where you get locked to a particular OS
> vendor because it’s the only one that supports IPv6 fragmentation >1500
> octets, then that is probably the least of your worries.  I’d be much
> more worried about IPv6 fragmentation in light of Ron Bonica’s comment
> that intermediary nodes drop packets with extension headers, which is bad
> news even for fragmented packets in the 1280-1500 range.
>
> 			-David

Just about every OS supports > 1500 byte fragmentation reassembly.  4K, 8K and
greater reassembly buffers commonly exist.

Mark
-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]

---

From: Mark Andrews <[email-masked]>
Date: Tue, 09 Feb 2016 07:09:43 +1100

In message <[email-masked].
outlook.com>, Ronald Bonica writes:
> So, as things stand, IP fragmentation is required to support UDP.
> However, the conversation doesnt end at that.
>
> Operational experience has taught us that IPv6 fragmentation does not
> work so well. Unlike IPv4, IPv6 encodes fragmentation information in an
> IPv6 extension header. Sadly, many operators discard packets containing
> that extension header. So, as specified, IPv6 provides fragmentation
> services, but as deployed, it does not.

Actually fragmentation works well unless you have a firewall that
drops fragments.  When they are not being deliberately blocked the
packets get through and are reassembled.  It is also not many
operators.  It is some operators.

Additionally there is zero reasons why firewalls can't open <src,
dst, frag offset != 0> when they open <src, dst, proto, src port,
dst port> for reply traffic for those that are paranoid about just
letting all non-zero fragment offset through.  I just let the
non-zero offset fragments through.

You might get a few extra packet through.

> Many end-stations work around this problem by sending packets no longer
> that the IPv6 minimum MTU (1280 bytes). This ensures that IPv6
> fragmentation services will never be required. However, it also prevents
> applications and networks from realizing the benefits of larger packets.
>
> b.      Write an RFC informing developers of UDP applications of the
> problem and advising them not to rely on protocol MTU > 1280
>
> b.      Design a UDP replacement that provides fragmentation service and
> migrate UPD applications to the replacement protocol
>
> Options 2a and 2b may not achievable, because they require action on the
> part of many, many parties. So, we seem to be stuck with Options 1a, 1b
> and 1c.
>
> From: ietf mailto:[email-masked] On Behalf Of Alexey Eromenko
> vs #1 or #2 and less implementation bugs); Assuming standard packet size
> is defined at 1280 bytes, like in IPv6

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Feb 2016 15:44:04 -0500

Seems to me that we might be misreading the original proposal. There
are two ways to read it:

1) In future all Internet routing gear MUST NOT fragment IP packets.
2) Take fragmentation out of the endpoint stack.

It seems to me that these are very different proposals. I fully
endorse the first if we add the caveat 'except to support legacy, non
jumbo frame capable links'.

When those limits were set, I was programming a machine with 32K of memory.

The second is obviously premature. But we could move the description
of fragmentation from being a 'current feature' to a 'legacy support'
issue.

Didn't we have a presentation on 'sane ways to create TCP like things
atop UDP' in an IETF plenary a couple of meetings back?

We seem to be something of a prisoner of the 1980s era Ethernet spec.
I am pretty sure that most of the equipment I buy is capable of doing
jumbo packets. But quite a few of them turn out to require jumbo
packets to be turned on by hand.

The ability to consistently support 64KB packets and thus high
throughput is potentially one of the main selling points for IPv6. I
don't believe in trying to persuade people to move to IPv6 through
differences in function. It will be a decade minimum before I consider
making use of an IPv6 feature not supported in IPv4 in an application
protocol. Performance is something else, I will encourage people to
upgrade to get faster performance.

Given all the sprockets, widgets and doo-dahs in the IPv6 spec, I am
pretty sure that it should be possible to get it to correctly
negotiate use of jumbo frames in an environment where fragmentation
might occur on the path and react accordingly.

So this is really a problem of UDP. And there I see three categories
of application:

1) DNS - is in a class of its own due to its function as the Internet
discovery protocol.

2) 'Can't help' - there are many many UDP applications today that have
all dealt with fragmentation in their own idiosyncratic ways.

3) 'Right way to use UDP' - a new specification describing a 'right'
way to use UDP to get TCP like features in a NAT compatible, etc
fashion could provide value.

I have been coding the past 6 months and doing little else. What
happened to that UDP proposal?

---

From: Joe Touch <[email-masked]>
Date: Mon, 8 Feb 2016 13:19:36 -0800

On 2/8/2016 8:08 AM, Alexey Eromenko wrote:
> problem ONLY for packet-sized between 1280 and 1500 bytes. Which is why
> I doubt if it is useful at all.

It's exactly what lets encapsulation tunneling occur and reoccur at will.

I.e., once you split a 1500 byte packet into 1280 and 220 and
encapsulate each of the results, you get to do it again.

Without fragmentation, arbitrary recursive direct (IP-in-IP) tunneling
is impossible.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 8 Feb 2016 13:22:59 -0800

> 1) In future all Internet routing gear MUST NOT fragment IP packets.

IPv6 is that future, FWIW.

> jumbo frame capable links'.

Where legacy == IPv4, yes. But even then, IPv4 fragmentation except at
the endpoints is already somewhat deprecated based on IP ID values (see
RFC 6864).

Joe

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Feb 2016 16:27:39 -0500

> IPv6 is that future, FWIW.

Hopefully yes.

But is it written down anywhere that IPv6 routers MUST accept packets
up to the full IP payload? The conversation in this thread suggests
not.

One of the tricks the WiFi folk use to keep people upgrading is to add
a little suffix to the protocol. So folk started to look for
802.11a/c/n/an.

As a marketing matter, an IPv6f profile (f for FAST) might help grease
the skids a bit.

---

From: Joe Touch <[email-masked]>
Date: Mon, 8 Feb 2016 13:38:05 -0800

> up to the full IP payload? The conversation in this thread suggests

RFC2460 specifies both relaying and source/sink requirements.

> As a marketing matter, an IPv6f profile (f for FAST) might help grease

If "f" = flawed, sure.

Joe

---

From: Ronald Bonica <[email-masked]>
Date: Mon, 8 Feb 2016 21:54:28 +0000

Hi Mark,

> 
> Actually fragmentation works well unless you have a firewall that drops
> fragments.  When they are not being deliberately blocked the packets get
> through and are reassembled.  It is also not many operators.  It is some
> operators.
> 

The words "many" and "some" don't do justice to the conversation.  https://tools.ietf.org/html/draft-ietf-v6ops-ipv6-ehs-in-real-world-02 provides more concrete numbers from real-world observation.

Beyond that, I agree that IPv6 fragmentation works perfectly unless firewalls are configured to make it stop working. Sadly, the number of network in which firewalls are so configured is too large to ignore. See the draft mentioned above.

> Additionally there is zero reasons why firewalls can't open <src, dst, frag
> offset != 0> when they open <src, dst, proto, src port, dst port> for reply
> traffic for those that are paranoid about just letting all non-zero fragment
> offset through.  I just let the non-zero offset fragments through.
> 

So, you are voicing support for Option 2a (i.e., Convince operators not to drop fragmented packets). This will clearly take time. Do you think that we should do anything else in the interim? Maybe 1b) Write an RFC informing developers of UDP applications of the problem and advising them not to rely on protocol MTU > 1280.

                                                                                                                                                                     Ron

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Feb 2016 16:59:01 -0500

> RFC2460 specifies both relaying and source/sink requirements.

So the shorter answer to the original question would be that
fragmentation is already deprecated.

Since IPv4 infrastructure is still with us, we are nowhere close to
removing legacy support for fragmentation.

There is however still no simple way to know if the level of routing
gear I buy for the house (cheaper office type) is well behaved or not.
Problem is that we tend to leave that to IEEE. Which made sense in the
days when non-IP ethernet was still an issue.

---

From: Joe Touch <[email-masked]>
Date: Mon, 8 Feb 2016 14:10:17 -0800

> fragmentation is already deprecated.

In-network, yes.

Edge (source), no. Not for IPv4 or IPv6.

Joe

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 08 Feb 2016 23:39:01 +0100

> The words "many" and "some" don't do justice to the conversation.  https://tools.ietf.org/html/draft-ietf-v6ops-ipv6-ehs-in-real-world-02 provides more concrete numbers from real-world observation.

Ah, but the result is much simpler.

Some other real world data (Google QUIC experiments) already tell us
that a sizable part of the Internet (was it 7 %?) is not reachable via
UDP at all.  This just ups that number slightly for IPv6 and UDP
protocols that don't have their own segmentation.

UDP, it was nice to have known you.

Grüße, Carsten

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 8 Feb 2016 17:44:27 -0500

I would note that tunnel mechanisms either need a very good path "size" 
reporting mechanism or a way to fragment.  As the tunnel entry and exit 
are end-points, and as path "size" discovery has many problems, many 
tunnel mechanisms do us IP (v4 or v6) fragmentation.
Admittedly, none of them like it, and most recommend that if you can do 
path MTU discovery you should, but we want things to work.

Yours,
Joel

On 2/8/16 1:23 PM, Warren Kumari wrote:
>
>
> On Mon, Feb 8, 2016 at 9:05 AM David Borman <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>
>      > On Feb 8, 2016, at 10:08 AM, Alexey Eromenko <[email-masked]
>     <mailto:[email-masked]>> wrote:
>      >
>      > On Mon, Feb 8, 2016 at 5:52 PM, John Levine <[email-masked]
>     <mailto:[email-masked]>> wrote:
>      > Basically, Fragmentation, as specified in RFC-2460 (IPv6), solves
>     the problem ONLY for packet-sized between 1280 and 1500 bytes. Which
>     is why I doubt if it is useful at all.
>
>     That’s the minimum required implementation, which guarantees that
>     you can send at least a basic ethernet sized packet to any host.
>     What is omitted in the above qoute is next 2 sentences:
>         "A node is permitted to
> where I do control the OS / model of all devices, and I know they
> support >1500 octet, it seems like a bad idea to *rely* on that.
> Sometime I'm going to want to change OS / add some other device, be able
> to interact with some other system. This sounds like vendor lock at its
> worst...
>
> W
>
>
>
>                      -David Borman
>
>      >
>      > (I also understand, that IPv4 fragmentation is much more
>     flexible, yes. It can go as small as 68 byte packets.)
>

---

From: Theodore V Faber <[email-masked]>
Date: Mon, 8 Feb 2016 22:49:14 +0000

>UDP, it was nice to have known you.

There are certainly some UDP applications that don’t cross major
providers.  It’s not an encouraging sign for UDP fans, though.

-- 
Ted Faber <[email-masked]>
Engineering Specialist
Computer Systems Research Department
310-336-7373

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Feb 2016 18:20:15 -0500

> UDP, it was nice to have known you.

Maybe what we needed all along was a better TCP that allowed data to
be sent on the first packet.

That is what people keep seeming to re-invent.

Another of those cases where people keep telling me that there are
good reasons not to do that but don't ever get round to explaining
what they are. Not a form of argument that I find convincing. Having
conceded the point that data can be sent via UDP, the idea of
combining a UDP packet with a TCP open request seems fairly obvious.

---

From: Ted Hardie <[email-masked]>
Date: Mon, 8 Feb 2016 15:56:31 -0800

On Mon, Feb 8, 2016 at 2:39 PM, Carsten Bormann <[email-masked]> wrote:

> Ronald Bonica wrote:
> > The words "many" and "some" don't do justice to the conversation.
> https://tools.ietf.org/html/draft-ietf-v6ops-ipv6-ehs-in-real-world-02
> provides more concrete numbers from real-world observation.
>
> UDP at all.

I think you may have misunderstood Jana's numbers slightly.  The graph he
showed in the Bar BoF did show that QUIC was not used in 7% of the cases,
but that's not the same as saying it was unreachable.

Chrome races TCP and QUIC and if TCP wins, it won't try QUIC again for the
domain for a set interval; it then exponentially backs off with subsequent
attempts.  Note as well that QUIC's handshake packets are a fixed size
(1392 bytes), and always the largest packet in the connection.  So you may
be seeing some race loss, some path MTU issues and some blockage of UDP
that is port specific.  This last can occur when firewalls do not expect
UDP on that port and block it on that port, but permit it on other ports
(e.g. in the range used by RTP).

I've added Jana, in case he should have updated information or corrections.

This just ups that number slightly for IPv6 and UDP
> UDP, it was nice to have known you.
It seems a bit premature to say good bye,

regards,

Ted

---

From: Fernando Gont <[email-masked]>
Date: Mon, 8 Feb 2016 20:48:50 -0300

> non-zero offset fragments through.

If and only if the packets do not employ other EHs and all the nodes
behind the fw implement RFC5722...

-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Fernando Gont <[email-masked]>
Date: Mon, 8 Feb 2016 20:46:00 -0300

Hi, David,

On 02/08/2016 03:56 PM, David Borman wrote:
>> So if you are writing an application that needs >1500 octets, use
>> an IPv6 implementation that supports >1500 octet fragmentation and
>> reassembly.
>> 
>> ... but as an application writer (or, basically anyone else), I
>> have no control over the "IPv6 implementation". Even if I'm in an
>> environment where I do control the OS / model of all devices, and I
>> know they support >1500 octet, it seems like a bad idea to *rely*
>> on that. Sometime I'm going to want to change OS / add some other
>> device, be able to interact with some other system. This sounds
>> like vendor lock at its worst…
> 
> vendor because it’s the only one that supports IPv6 fragmentation
> >1500 octets, then that is probably the least of your worries.  I’d
> be much more worried about IPv6 fragmentation in light of Ron
> Bonica’s comment that intermediary nodes drop packets with extension
> headers, which is bad news even for fragmented packets in the
> 1280-1500 range.

FWIW, when communicating with public {web, mail, DNS} servers, you get
packet drop rates of over 25% if you employ IPv6 fragmentation.

(My pleasure to hear from you ;-) )

Thanks,

---

From: Fernando Gont <[email-masked]>
Date: Mon, 8 Feb 2016 20:53:58 -0300

On 02/08/2016 05:44 PM, Phillip Hallam-Baker wrote:
[...]
> upgrade to get faster performance.

You mean >64KB? -- 'cause IPv4 can do 64KB packets...

---

From: Fernando Gont <[email-masked]>
Date: Mon, 8 Feb 2016 21:06:12 -0300

> be sent on the first packet.

FWIW, TCP does allow you to send data in the SYN. However, it's not
friendly with DoS mitigatons such as SYN-cookies.

Thanks!

Cheers,

---

From: Warren Kumari <[email-masked]>
Date: Tue, 09 Feb 2016 00:08:37 +0000

On Mon, Feb 8, 2016 at 4:06 PM Fernando Gont <[email-masked]> wrote:

> FWIW, TCP does allow you to send data in the SYN.

https://tools.ietf.org/html/rfc7413 for those who are wondering.
W

> However, it's not

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Feb 2016 19:47:31 -0500

> You mean >64KB? -- 'cause IPv4 can do 64KB packets...

By 'doing' I mean, it actually works in practice with stuff bought off
the shelf without any special configuration or expertise or additional
software beyond that provided by the platform vendors.

I am aware that IPv4 was always theoretically capable of jumbo frames.
But practice has been rather different. And one of the biggest
problems we still have is that while we talk about IP everywhere, the
reality is 'IP atop IEEE link layer' and the IEEE people still insist
on supporting non IP protocols.

Problem is that most of us have ethernet hubs rather than true IP
switches. If we had real IP everywhere we could deprecate MAC
addresses.

---

From: Fernando Gont <[email-masked]>
Date: Mon, 8 Feb 2016 22:02:03 -0300

On 02/08/2016 09:08 PM, Warren Kumari wrote:
> 
> 
> On Mon, Feb 8, 2016 at 4:06 PM Fernando Gont <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     > On Mon, Feb 8, 2016 at 5:39 PM, Carsten Bormann <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >> Ronald Bonica wrote:

Even pre-fast open TCP already allowed this -- however, those data can
only be *consumed* once the 3WHS is complete...

Cheers,

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 8 Feb 2016 20:18:30 -0500

> only be *consumed* once the 3WHS is complete...

Yep, as an apps guy, I don't really care about theoretical
capabilities of the protocols that aren't exposed by the available
network stack APIs.

The job isn't done until you have persuaded the platform people to
give access to the capabilities. Hence we end up locked in UDP-ville.

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 9 Feb 2016 13:51:07 +0900

> encapsulate each of the results, you get to do it again.

Considering a possibility of recursive tunneling, that is a
poor way of fragmentation. 1500 should be divided into 750 and
750 as is documented in rfc791:

      For example, one could implement
      a fragmentation procedure that repeatly divided large datagrams in
      half until the resulting fragments were less than the maximum
      transmission unit size.

> Without fragmentation, arbitrary recursive direct (IP-in-IP) tunneling

It does not mean fragmentation must be supported by outer or inner
IP layer as IP over fragmented_IP or fragmented_IP over IP.

Instead, fragmentation information may be placed between outer and
inner IP headers as IP-tunnelling_header_with_fragmentation_info-IP
or IP over fragmentation_layer over IP.

					Masataka Ohta

---

From: Yoav Nir <[email-masked]>
Date: Tue, 9 Feb 2016 07:01:16 +0200

> On 9 Feb 2016, at 1:20 AM, Phillip Hallam-Baker <[email-masked]> wrote:
> 

And could be stateless on the server

And you could get the same size data as the other side sent you without adding your own message layer.

Yoav

---

From: Masataka Ohta <[email-masked]>
Date: Tue, 9 Feb 2016 14:04:37 +0900

> So, as things stand, IP fragmentation is required to support UDP.

With theoretical IPv4 minimum MTU of 68B, maybe, but, with
practical IPv4 minimum MTU > 1400B, it is not.

> work so well. Unlike IPv4, IPv6 encodes fragmentation information in
> an IPv6 extension header. Sadly, many operators discard packets
> containing that extension header.

Interesting. What a proper operation!

Considering that operational experience of IPv4 has taught us that
IP option is rather harmful than useful and totally unnecessary,
what was sad is that the infamous IPv6 design committee ignored
the operational experience, which is corrected by real world
operators.

						Masataka Ohta

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 9 Feb 2016 00:36:52 -0500

> And you could get the same size data as the other side sent you without adding your own message layer.

The stateless server is the real win for UDP and the reason I am very
skeptical of the idea of using TCP in DPRIV.

What rather puzzles me is that people don't seem to accept that a
server can still be stateless if it receives one UDP request packet
and returns ten (say).

Obviously unreliable transport means that there is a limit to the
number of packets that can be returned without some sort of
flow/retransmit scheme. But it is a lot more than one.

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 9 Feb 2016 11:00:54 +0100

> And you could get the same size data as the other side sent you without adding your own message layer.

Never forget that five packets are required.

http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=1093283&url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F26%2F23863%2F01093283.pdf%3Farnumber%3D1093283

---

From: Tony Finch <[email-masked]>
Date: Tue, 9 Feb 2016 12:06:39 +0000

Phillip Hallam-Baker <[email-masked]> wrote:
> what they are.

http://roland.grc.nasa.gov/tcp-impl/list/archive/1292.html

But I thought TCP fast open https://tools.ietf.org/html/rfc7413
fixed the design errors in T/TCP, so is it still considered a bad idea?

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Fair Isle: Cyclonic 5 or 6 at first in south, otherwise northerly 7 to severe
gale 9, decreasing 6 later. Rough in southeast, otherwise very rough or high.
Rain or showers. Good, occasionally poor.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 9 Feb 2016 08:14:19 -0500

> fixed the design errors in T/TCP, so is it still considered a bad idea?

Perhaps it does. But as I said, it only counts as solved when it is in
the stacks I can use. And no, Linux doesn't count.

---

From: Warren Kumari <[email-masked]>
Date: Tue, 09 Feb 2016 15:59:48 +0000

> the stacks I can use. And no, Linux doesn't count.
Yup.

<rant>
There seems to be a fair amount of discussion requiring knowledge of the
host stack, or understanding of the capabilities of a specific network
(e.g: all the hosts support [reassembly of "large" fragments | TCP fast
open], all the routers in my network support looking deep into EH, all my
devices set flow labels, etc.).

This feels deeply flawed to me - applications shouldn't need to have deep
knowledge of the network or end system stack behavior, and relying on
specific behavior of a system / network makes the application brittle and
non-portable[0].

Until a behavior is supported by the lowest common denominator / (almost)
everything, it probably makes sense to avoid it[1].

As an example (which I'll use because it's well known / simple, not because
it is still applicable), reassembly of >1500 byte packets:
"An upper-layer protocol or application that depends on IPv6
fragmentation to send packets larger than the MTU of a path should
not send packets larger than 1500 octets unless it has assurance that
the destination is capable of reassembling packets of that larger
size."

If I'm writing a general purpose application (e.g a gaming app), how on
earth do I know if the destination is capable of reassembling >1500 octets?
I could:
A: blindly assume it does
B: limit myself to platform that do
C: I can probe

A is a poor option until I be very sure that everything I (conceivably)
want to talk to supports it, and will continue to for for the foreseeable
future.

B is a poor option for obvious reasons - even on something like a phone I'd
like to be able to port this to one of the other ecosystems with minimal
work.

C is a poor option because I need to add significant complexity - either I
have a negotiation / capabilities exchange at session startup, or I probe
in mid-session. I can probe in parallel, or try it and wait for failure
(which is also tricky - was the packet dropped *because* it is >1500 bytes,
or was it random congestion?)

I'm just trying to write a game - I'm not a network weenie. With limited
resources, it makes sense to just use the well defined, well known,
universally supported services, not rely on something that only works on
most of the devices, most of the time.
Unless the behavior provides *significant* benefit, I'm likely to avoid it.

Yes, this sucks. It means that innovation slows down - apps avoid
non-ubiquitous features, which means that vendors / stack have less
incentive to build / deploy them. Waving the protocol bible and saying "but
the spec says..." doesn't really change the incentives of reasonable people
optimizing for their own (selfish) reasons.

</rant>

W
[0]: This is changing with "apps" - e.g mobile apps.E.g:  If you write an
iOS app, you know it will run on iOS.
[1]: Unless it provides compelling benefits, you have spare development
cycles, or you have a pet project / protocol.

---

From: Harald Alvestrand <[email-masked]>
Date: Tue, 9 Feb 2016 17:51:44 +0100

> give access to the capabilities. Hence we end up locked in UDP-ville.

UDP is the application layer's IP.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 9 Feb 2016 14:10:00 -0500

On Tue, Feb 9, 2016 at 10:59 AM, Warren Kumari <[email-masked]> wrote:

> host stack, or understanding of the capabilities of a specific network (e.g:
> all the hosts support [reassembly of "large" fragments | TCP fast open], all
> the routers in my network support looking deep into EH, all my devices set
> flow labels, etc.).
>

Absolutely. And I see similar issues when people try to get me to use
HTTP features in Web Services.

I know all about said HTTP features. I wrote some of them. But when I
am running a Web Service over HTTP, HTTP is a lower level protocol
layer and I don't want my application to depend on any feature at that
level unless there is a clean separation of concerns.

> everything, it probably makes sense to avoid it[1].

Probably. But there is another option: Put all the wood behind one arrow.

The IETF is architected as a research lab rather than a standards
body. That has good points and bad points. One of the bad points is
that we don't end up with one consistent and coherent way to achieve
an outcome, we end up with multiple options and a hope that 'the
market' will come to a decision. And then people are upset when the
market decides that it is quite happy where it is.

The other problem is that as Warren points out, the process doesn't
really produce proposals that are fully interchangeable. For years
people were trying to persuade people to move to DNSSEC and IPv6 with
what I call the 'boat anchor' strategy of forcing other specs to build
on them as a platform requirement. I once sat through a BOF where a
group of people who claimed to be doing 'home automation' (what we
called IoT before) who started off by mandating IPv6.

If you are writing an application protocol and it mandates IP let
alone a specific version then you are doing it wrong.

> optimizing for their own (selfish) reasons.

TCP fast start does at least have the advantage that the endpoints can
always gracefully degrade to regular TCP. So it does have something of
a transition strategy. But probably not enough right now to convince
me that I should move away from a UDP hack (if I was doing one).

---

From: Joe Touch <[email-masked]>
Date: Tue, 9 Feb 2016 12:24:08 -0800

On 2/8/2016 2:44 PM, Joel M. Halpern wrote:
> reporting mechanism or a way to fragment.

If you don't have a way to fragment, you end up with a hard limit on the
amount of tunneling and tunnel overhead. Otherwise, at some point, you
end up with a "size" of zero.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 9 Feb 2016 12:26:31 -0800

On 2/8/2016 9:21 AM, Ronald Bonica wrote:
> fragmentation services will never be required.

IPv6 1280 may itself not be fragmented, but it may end up in multiple IP
fragments within an IP tunnel if the tunnel cannot assume an MTU that is
larger.

So if you have tunnels, fragmentation services are always ultimately
required.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 9 Feb 2016 12:27:51 -0800

On 2/8/2016 4:47 PM, Phillip Hallam-Baker wrote:
...
> switches. If we had real IP everywhere we could deprecate MAC

Except that we derive self-assigned IPv6 addresses from MAC addresses.

Joe

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 9 Feb 2016 15:47:53 -0500

> Except that we derive self-assigned IPv6 addresses from MAC addresses.

If we didn't need them to be MAC addresses we could go to EUI-64 and
have 16 shiny new bits to play with.

> end up with a "size" of zero.

By definition a tunnel has two ends. There is no reason why
fragmentation in a tunnel should make use of IP fragmentation as
opposed to an in-tunnel fragmentation scheme.

---

From: Doug Royer <[email-masked]>
Date: Tue, 9 Feb 2016 14:46:19 -0700

> So if you have tunnels, fragmentation services are always ultimately

And when broken (like with my ISP), I had to reduce my MTU to 1442.
Because it looks like fragmentation is failing over any IPv6 connection
I can use. Or perhaps its some kind of IPv6 restriction I can' find.

When Firefox switched to defaulting to IPv6 when it detects you have it
available, all my web results got VERY slow, until I reduced by MTU.

   not an IPv6 expert, just liking to feel I participate :-)

-- 

Doug Royer - (http://DougRoyer.US)
[email-masked]
714-989-6135

---

From: Joe Touch <[email-masked]>
Date: Tue, 9 Feb 2016 14:31:42 -0800

> have 16 shiny new bits to play with.

*You* wouldn't get to play with them; MAC vendors would. How would that
help, given they're already intended to be unique?

> opposed to an in-tunnel fragmentation scheme.

Reason #1: IP reassembly is already deployed. Yes, we could use other
protocols as a shim to support IP-in-IP (and we do), but that doesn't
mean that they necessarily won't end up with the same problem - assuming
*their* IP should be 1280.

So let me understand:

	- first, you claim IP fragmentation is a network problem
	because it obscures info you need at forwarding devices
	(because you're peeking at L4)

	- now you want that info even further obscured by another
	layer of encapsulation

There's simply no pleasing you ;-)

Joe

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 9 Feb 2016 18:09:24 -0500

> help, given they're already intended to be unique?

I don't want a unique identifier associated with my machine going on the wire.

I was one of the first people arguing that WiFi devices should declare
a random MAC address. The idea of putting permanent linkable
information on the wire is an abomination.

> *their* IP should be 1280.

Lots of things are already deployed. You presented a corner case. I
pointed out that there is no need to handle that corner case in the IP
layer.

I don't think you can use IP fragment reassembly to solve the problem
you are suggesting using it for. 1280 isn't actually a lot of bytes.
You are going to use that up fairly quickly if you are not careful.

>         (because you're peeking at L4)

No, that wasn't my argument. My argument was that I want all my
network gear to be capable of 64K networking because 1280 bytes is a
stupid maximum frame size in 2016.

Having insisted that all my gear be 64K capable, I am quite happy with
a modest restriction to allow tunneling overhead. Lets say the max
payload an endpoint SHOULD generate is 64256 bytes, that would leave
1280 bytes for tunneling overhead.

>         - now you want that info even further obscured by another

No, you raised the tunnel in tunnel corner case. I didn't suggest
requiring anything of the sort.

If your tunnel can't fit the input packet on a 64K clean pipe, then it
should have the responsibility to figure out how to fragment.

---

From: Joe Touch <[email-masked]>
Date: Tue, 9 Feb 2016 15:17:49 -0800

On 2/9/2016 3:09 PM, Phillip Hallam-Baker wrote:
...
> I don't want a unique identifier associated with my machine going on the wire.

Your MAC address can also be self-assigned as long as you do DAD within
the same subnet covered by the router advertisement.

...
> pointed out that there is no need to handle that corner case in the IP

That's very NIMBY of you.

> You are going to use that up fairly quickly if you are not careful.

It is already used for that purpose whenever there's IP in IP
encapsulation and where there are intermediate layers that don't support
fragmentation separately (e.g., UDP).

> stupid maximum frame size in 2016.

It doesn't matter what size you pick. If it's 64KB, then the first time
you end up encountering 64KB in 64KB you'll need to fragment.

> 1280 bytes for tunneling overhead.

Whatever number you think is reasonable will end up being consumed by
someone else.

> should have the responsibility to figure out how to fragment.

As long as it doesn't use "your" protocol to do it (IP)? Why?

Someone has to support tunneling somewhere. IP is intended to be the
universal interoperability layer - which means that it ought to be the
layer to support it.

Pushing it off to somewhere else will just end up with you complaining
that I've hidden some part of the L3, L4, ... header behind many layers
of cruft that your routers can't parse efficiently (again).

This is shuffling the deck chairs. Frag/reassembly is needed from
whatever layer relays messages.

Joe

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 10 Feb 2016 13:20:56 +1300

>>>> Except that we derive self-assigned IPv6 addresses from MAC addresses.

No we didn't; they were derived from EUI-64 all along (IEEE MAC being a
proper subset of EUI-64 may have confused people). But that's history,
since the recommendation is being changed to 64 pseudo-random bits
by various recent 6man documents.

> information on the wire is an abomination.

Maybe, although it does lead to interesting results like Wikipedia blocking
the IPv4 addresses of various Swiss civil servants recently. However,
IPv6 provided a layer 3 fix for this years ago (at the time of the Intel CPU
serial number controversy, iirc). Fixing layer 2 is not our department.

   Brian

---

From: Fernando Gont <[email-masked]>
Date: Tue, 9 Feb 2016 21:15:45 -0300

> I don't want a unique identifier associated with my machine going on the wire.

Use RFC7217. Linux is in the process of defaulting to it.

Thanks,

> information on the wire is an abomination.

+1 -- nice to hear that!  (and that's even worse when you do that in a
layer-3 address).

Thanks!

Cheers,

---

From: Fernando Gont <[email-masked]>
Date: Tue, 9 Feb 2016 21:20:32 -0300

> I don't want a unique identifier associated with my machine going on the wire.

FWIW, for an analysis of the possible root of this sort of issue, please
see:
<https://www.ietf.org/internet-drafts/draft-gont-predictable-numeric-ids-00.txt>

Thanks!

Cheers,

---

From: Fernando Gont <[email-masked]>
Date: Tue, 9 Feb 2016 21:24:46 -0300

> alone a specific version then you are doing it wrong.

FWIW, part of that has to do with using low-level APIs in apps.

If you think about it, an app tipically has to be DNS-aware, know about
ports, etc., rather than "get_me_a_connection()".

Thanks,

---

From: Fernando Gont <[email-masked]>
Date: Tue, 9 Feb 2016 21:28:00 -0300

> I can use. Or perhaps its some kind of IPv6 restriction I can' find.

Dropping of IPv6 fragments is, unfortunately, widespread. Please see:
<https://www.ietf.org/id/draft-ietf-v6ops-ipv6-ehs-in-real-world-02.txt>.

Thanks!

Cheers,

---

From: joel jaeggli <[email-masked]>
Date: Tue, 9 Feb 2016 17:03:13 -0800

> headers, which is bad news even for fragmented packets in the

For those of us with ecmp load balancing the challenge of associating a
fragement with the rest of the flow are also a problem. In my own case I
can engineer circumstances where I should never receive such a fragment,
so I can safely drop them anyway but I doubt everyone has that luxury.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 9 Feb 2016 20:14:25 -0500

> That's very NIMBY of you.

Not really. A tunneling layer is going to have to deal with
fragmentation under your model.

[Since Joe doesn't seem to be responding to the argument I made, I'll
not flog that horse further]

> As long as it doesn't use "your" protocol to do it (IP)? Why?

Because the Internet architecture says to keep the core as simple as
possible. IP fragmentation introduces complexity that should be kept
out of the core. That is why.

We needed that feature in the Inter-network in 1980. We do not need
that feature int the core today. So it SHOULD NOT be an IP feature. It
can be a higher layer feature, it can be a feature that can still be
used at the network level.

Tunneling, encapsulation, VPNs, IP-in-IP are all network activities.
They shouldn't ever occur on the Inter-Network by definition.

> universal interoperability layer - which means that it ought to be the

No, IP is the Inter-network Protocol.

The universal interop layers are UDP and TCP.

---

From: Joe Touch <[email-masked]>
Date: Tue, 9 Feb 2016 17:43:35 -0800

> Tunneling, encapsulation, VPNs, IP-in-IP are all network activities.

Tunneling is an end system activity.

Nodes that encap or decap are acting as sources or sinks, not relays.

This is true regardless of the protocol you use for tunneling. If you
use a protocol layer other than IP, you have just moved the deck chairs.
And if you don't support this at IP, you have to build a network that
can transit messages far larger than you will typically use just so you
can have the headroom for encapsulation.

So either way you're over-engineering - either by building to an MTU you
won't typically use or by supporting fragmentation. However, once you
support fragmentation you're done. There's no way to enforce overbuilt MTUs.

Joe

---

From: Fernando Gont <[email-masked]>
Date: Wed, 10 Feb 2016 00:19:34 -0300

> so I can safely drop them anyway but I doubt everyone has that luxury.

Google does, too, don't they?

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 10 Feb 2016 15:05:58 +0900

Joe Touch wrote:

> Reason #1: IP reassembly is already deployed.

The reality is that wise operators denied deployment of
stupid idea of extension headers including that for IP
reassembly.

> 	- now you want that info even further obscured by another

Wrong. The worst kind of obscurity is a transport header at
the end of a chain of 1000 or more IPv6 extension headers.

Note that the transport header may not be placed in the
first fragment.

As following a long chain means vulnerability to DOS, there
should be some upper bound on the chain length and the most
reasonable value for the upper bound is 0, because all the
extension headers are useless.

						Masataka Ohta

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 10 Feb 2016 15:21:00 +0900

> Nodes that encap or decap are acting as sources or sinks, not relays.

A problem is that relays (firewalls) are involved in decap.

						Masataka Ohta

---

From: Fernando Gont <[email-masked]>
Date: Wed, 10 Feb 2016 04:05:10 -0300

Hi, Masataka,

> extension headers are useless.

RFC7112 imposes some basic constraints: the entire EH chain must be
present in the first fragment.

Thanks,

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 10 Feb 2016 16:50:30 +0900

> Hi, Masataka,

Hi,

> present in the first fragment.

Thank you for the information. But, I'm afraid the fix is too late
and too insufficient to change the reality above. That is, my point
on DOS is still valid and, worse, some combination of extension
headers may results in yet unnoticed vulnerability, which is
partially why allowing extension headers is a stupid thing to do.

Moreover, the rfc should also limit header chain length below
256B or so. Though DNS message over UDP over IPv4, with 576B
reassembly buffer, can be 508B (in practice, 548B) long, DNS
message over UDP over IPv6, with 1280B reassembly buffer,
can be less than 100B, if header chain is lengthy.

Worse, though the rfc require an entire upper layer header
included in the first fragment, the requirement is too much.

According to rfc792, the first 8B should be enough. As for
ICMPv6, ICMPv6 should also be required to contain all the
header chain up to the first 8B of an upper layer header.

						Masataka Ohta

---

From: Fernando Gont <[email-masked]>
Date: Wed, 10 Feb 2016 05:25:25 -0300

Hi,

On 02/10/2016 04:50 AM, Masataka Ohta wrote:
[....]

> 256B or so.

I tried that in 6man, but there was opposition to my proposal at the time.

> Though DNS message over UDP over IPv4, with 576B

Agreed.

> included in the first fragment, the requirement is too much.

Not sure what you mean.

> header chain up to the first 8B of an upper layer header.

... which may not be feasible if a long EH chain is employed in the
packet that generated the error message.

Thanks,

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 10 Feb 2016 19:29:48 +0900

> I tried that in 6man, but there was opposition to my proposal at the time.

OK, collective stupidity of IPv6 committee is same as I tried
it about 20 years ago.

>> included in the first fragment, the requirement is too much.

After the last mail, I have noticed that this is a serious defect
of the rfc.

That is, if an upper layer protocol is not known to some box,
the box has no idea on how long is the header, which
means the box is now authorized to discard the packet, even if
actual header length is small and no extension header exist.

One thing we should learn from the glorious history of IPv4
is meaning of specification in rfc792 that:

  |      Internet Header + 64 bits of Original Data Datagram      |

which makes ICMP implementation independent from upper
layer header structure and requires upper layer specifications
contain port number (or something like that) information in
the first 8B of their headers.

> packet that generated the error message.

That's one of a reason why long EH must be banned.

						Masataka Ohta

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Feb 2016 13:26:15 -0800

> A problem is that relays (firewalls) are involved in decap.

The only problem there is believing that a device is defined by its
product literature, rather than its behavior.

I repeat: nodes that encap or decap are acting as sources or sinks, not
relays.

Nodes such as NATs and firewalls act as end hosts on the public side and
routers on the private side. Which is why they need to obey RFC1122
semantics on the public side. What happens on the private side that
drives the public side behavior is irrelevant.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Feb 2016 13:26:15 -0800

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 10 Feb 2016 23:45:36 +0000

Here are two tunnel proposals that do fragmentation as part of the
encapsulation layer instead of at the IP layer:

https://datatracker.ietf.org/doc/draft-herbert-gue-fragmentation/
https://datatracker.ietf.org/doc/draft-templin-intarea-grefrag/

Thanks  - Fred
[email-masked]

---

From: Alexey Eromenko <[email-masked]>
Date: Thu, 11 Feb 2016 11:10:49 +0200

Yes, thanks, that's informative !

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 11 Feb 2016 23:05:10 +0900

> I repeat: nodes that encap or decap are acting as sources or sinks, not

I'm afraid firewalls are relays.

> semantics on the public side.

So, you think firewalls should reassemble fragments. Wow!

							Masataka Ohta

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Feb 2016 09:58:12 -0800

> I'm afraid firewalls are relays.

A firewall that filters on L3 is a router regardless of which side you
look at.

A firewall that encaps/decapsulates is a host on the public side and a
router on the private side. A firewall that inspects beyond L3 is a host
as well, for similar reasons.

So the term "firewall" isn't the issue; it's the behavior that is.

> So, you think firewalls should reassemble fragments. Wow!

And yet that is exactly the correct conclusion regarding most behaviors
that firewalls perform that act like end hosts. Once you realize that
inspecting L4 or encaps/decaps is acting like a host, the requirements
become very clear - even if you don't like them.

So yes, a firewall that inspects L4 or encap/decaps either needs to
reassemble fragments or act like that's what's happening (e.g., to
retain a copy of the first fragment of a set to direct later fragments
within that set).

The model takes you to exactly the right conclusion.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 11 Feb 2016 18:07:26 +0000

Hi Joe,

> -----Original Message-----
> From: ietf [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: Thursday, February 11, 2016 9:58 AM
> To: Masataka Ohta; [email-masked]
> Subject: Re: Is Fragmentation at IP layer even needed ?
> 
> 
> 

Correct- Cisco calls that "Virtual Fragmentation Reassembly", i.e., gather
up all of the fragments in the set and then release them unassembled
once the firewall has determined that the packet is acceptable.

Thanks - Fred

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 11 Feb 2016 13:19:18 -0500

> A firewall that filters on L3 is a router regardless of which side you

Using 'layers' to describe Internet architecture can be very
misleading because the Internet isn't layered according to the ISO
model and the layers don't necessarily stack up the way people expect
once tunneling is involved.

For example, if I have an SSH channel to a system (or a TLS firewall),
I have a transport layer protocol that is presenting a packet layer
interface.

So if we number the layers, we have 1, 2, 3, 4, 5, 3 [4, 5, 7].

One of the things I learned early on programming Microsoft BASIC was
to not use sequential line numbers. And I was really glad to get rid
of line numbers when I moved to machines with decent amounts of RAM.
Seems to me that the numbered layer model confuses rather than
clarifies and especially so when tunneling is being discussed.

A tunnel should be a tunnel. If you fragment at the tunnel ingress,
you should defragment at the egress. Otherwise you are simply pushing
your state maintenance requirements onto the receiving endpoint in a
way that isn't scaleable.

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Feb 2016 10:35:16 -0800

> once tunneling is involved.

Internet layers correspond to ISO layers up through 4 fairly well.

Layers have always been relative, though - what I think is layer 4 might
be layer 2 to you if you're using my UDP to transit your IP inside.

> So if we number the layers, we have 1, 2, 3, 4, 5, 3 [4, 5, 7].

Strictly, you have two different views:

	SSH [1, 2, 3, 4, 5] acting as [1* 2*] to the transit protocol.

	The complete SSH view is:

		[1, 2, 3, 4, 5, 7#] where 7# is the transit
		(where the transit acts as an application)

	The complete transit view is:

		[1*, 2*, 3, 4, 5, 7], where 2* is a tunnel (which *is*
		just a link) and 1* is virtual

SSH needs to treat the transit as an application.

The transit needs to treat SSH as a link (including support, if needed,
for ARP, MTU discovery, etc.)

> clarifies and especially so when tunneling is being discussed.

Once you move to tunnels you need to think in relative terms, not absolutes.

> A tunnel should be a tunnel.

A tunnel *is* (and always has been) a link.

The only difference between a tunnel and a link is that a link is a
special subset of "all tunnels that run over physical connections".

> If you fragment at the tunnel ingress,

The beauty of considering a tunnel a link is that the same rules apply,
as they always should have. Just as a link that can't transit an IP
packet requires frag/reassembly within the link (e.g., ATM does this),
so should a tunnel.

Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Thu, 11 Feb 2016 18:57:44 +0000

Hi Joe,

> Sent: Thursday, February 11, 2016 10:35 AM
> To: Phillip Hallam-Baker
> Cc: IETF Discussion Mailing List

This is coming close to repeating a discussion we had in intarea back in the
August 2015 timeframe, where I thought we had reached a conclusion.

Thanks - Fred

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Feb 2016 11:04:50 -0800

> August 2015 timeframe, where I thought we had reached a conclusion.

We haven't talked about it yet, but yes - this is closely related to the
INTAREA discussion on this issue and the current WG document on tunnels.

Joe

---

From: Masataka Ohta <[email-masked]>
Date: Fri, 12 Feb 2016 05:54:44 +0900

> become very clear - even if you don't like them.

The reality is that you don't like the reality.

> retain a copy of the first fragment of a set to direct later fragments

Remember, with IPv6, the firewall can't fragment the reassembled
packets. So, no, unless the firewall output reassembled packets,
which may be larger than MTU of an outgoing link, it is not "act
like that's what's happening".

> The model takes you to exactly the right conclusion.

The wrong conclusion above means your model is broken.

Simplistic view is not applicable to complicated things.

							Masataka Ohta

---

From: Mark Andrews <[email-masked]>
Date: Fri, 12 Feb 2016 14:30:31 +1100

> like that's what's happening".

The key words were "act like that's what's happening".  You can
hold fragments until you see the first fragment, check it, then
release all matching fragments.  You can virtually reassemble all
the fragments then release them all if you need to see the entire
packet.  There has never been a need to throw away all fragments.
Only poor purchasing decisions causing everyone else to have to
work around them.

---

From: Masataka Ohta <[email-masked]>
Date: Fri, 12 Feb 2016 13:00:31 +0900

Mark Andrews wrote:

> release all matching fragments.

Thus, a set of packets are investigated and there is no
reassembly happening.

It is merely that some firewalls sometimes change filtering
behavior by investigating a set of packets (like snooping
ftp command stream to open data port, which no one call virtual
TCP streaming), regardless of whether the packets are fragments
of a packet or not.

> You can virtually reassemble all

Ok, ok. Though something you call "virtual reassembly" is not
reassembly at all, its processing cost is equivalent to real
reassembly. That is, you are saying fragmentation and reassembly
are so easy that there is no need to avoid them.

So, let's revise IPv6 and use fragmentation everywhere. There has
never been a need for impossible PMTUD.

> Only poor purchasing decisions causing everyone else to have to

It is caused primarily by stupid design of IPv6.

						Masataka Ohta

---

From: Mark Andrews <[email-masked]>
Date: Fri, 12 Feb 2016 15:32:08 +1100

> It is caused primarily by stupid design of IPv6.

Given you see the same stupidity of devices dropping IPv4 fragments
I don't think you can attribute that to IPv6.

Mark

---

From: Masataka Ohta <[email-masked]>
Date: Fri, 12 Feb 2016 13:49:15 +0900

> Given you see the same stupidity of devices dropping IPv4 fragments

Stupidity is in IPv6 having fragments even with minimum MTU of
1280B and lengthy extension headers to make transport header
obscure, neither of which has anything to do with IPv4. And,
wrong fix of rfc7112 has made the situation worse.

OTOH, IPv4 fragmentation is there because it was necessary for
possible network environment at the time it was designed.

						Masataka Ohta

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Feb 2016 10:26:56 -0800

...
> packets.

Routers shouldn't reassemble, but then routers aren't supposed to look
beyond L3. You cannot have it both ways.

Once you inspect L4, you *are* acting as a host.

As Mark pointed out, you don't need to strictly reassemble (i.e., to
emit a corresponding reassembled packet). You just need to reassemble
the information.

> So, no, unless the firewall output reassembled packets,

As Fred pointed out, existing devices already emulate reassembly without
emmitting the reassembled result.

--

Remember too, that if the firewall is "translating" the headers it ends
up completely acting as a host - because it sources IP packets with its
own IP addresses. In that case, it can apply source fragmentation.

Yes - this also means that a firewall that changes headers needs to
assign new, unique ID values for any fragmented packets too.  And it
needs to act as a terminus for ICMP PTB errors to adjust its
fragmentation size.

Again, the model leads you to the correct conclusions.

Joe

---

From: Warren Kumari <[email-masked]>
Date: Fri, 12 Feb 2016 19:10:54 +0000

> beyond L3. You cannot have it both ways.

You keep saying that.... and then a bunch of operators say "Yeah, but I
have an actual network to run, and I need to look beyond L3 because my
customers want me to mitigate their DoS, I want to filter on L4 before
handing data to internal services, and I use ECMP and need L4 because I
cannot rely on flow labels".

> Once you inspect L4, you *are* acting as a host.
So, this entire thread (which has reminded me why I stopped participating
in v6ops) is just a terminology issue? ;-)
Actually, I've just noticed that this thread is actually on ietf@ --
perhaps it should be moved to v6ops?

> emit a corresponding reassembled packet). You just need to reassemble

Which requires keeping state, yes? This is not realistic in modern large
network devices.
Saying "vendors should jolly well do a better job and redesign their gear
so that it can, and operators should simply pay whatever the cost is... oh,
and redesign their networks for flow consistency, because *that's what the
specifications require*" is likely to continue having people say "Yeah,
sure, whatever. But I've got a real network to run...."

... and yet we see lots of evidence that fragments (and EH) have issues in
real world testing.

W

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Feb 2016 11:20:48 -0800

On 2/12/2016 11:10 AM, Warren Kumari wrote:
> 
> 
> On Fri, Feb 12, 2016 at 1:27 PM Joe Touch <[email-masked]
> <mailto:[email-masked]>> wrote:
...
> cannot rely on flow labels".

Operators who don't want to pay for the devices that will actually do
the work involved to support the filtering model they want to employ.

This is a lot like selling empty boxes of cereal - because I need to
sell boxes of cereal, and the cost of the food is getting in the way.

> So, this entire thread (which has reminded me why I stopped
> participating in v6ops) is just a terminology issue? ;-)

It's about what level of work you should be expected to expend to
produce a desired behavior. Ultimately, it's about the conceptual
Internet architecture.

...
> Which requires keeping state, yes? This is not realistic in modern large

Translation: finding a place to keep all that cereal isn't realistic
when what I really want to do is make money.

Sorry - ante up. If you want to have the benefit of acting like a host,
you need to do the work of a host.

And note that some commercial devices do already support this sort of
state, so it's not theoretical.

> Saying "vendors should jolly well do a better job and redesign their
> gear so that it can, and operators should simply pay whatever the cost
> is... oh, and redesign their networks for flow consistency, because
> *that's what the specifications require*" is likely to continue having
> people say "Yeah, sure, whatever. But I've got a real network to run...."

There are real profits to be made. All that cereal is getting in
the way. It's a lot more profitable to sell empty boxes.

The real issue here is the lack of compliance testing and certification.
If we had that, vendors wouldn't be able to sell boxes that didn't
comply with requirements and operators wouldn't be able to make promises
they can't keep.

...
> ... and yet we see lots of evidence that fragments (and EH) have issues
> in real world testing.

What we see is operator and vendor greed being incompatible with
Internet architecture. We need to push back on *that*, not water down
our definition of the Internet to support their current business model.

Joe

---

From: Masataka Ohta <[email-masked]>
Date: Sun, 14 Feb 2016 10:26:11 +0900

> Once you inspect L4, you *are* acting as a host.

That is an abstract nonsense by you and others who do not
understand layering at all, which constitutes part of the
collective stupidity.

QoS (not CoS but real QoS) capable routers must inspect L4. Though
fragmentation is generally prohibited for QoS, it is merely because
increased link overhead changes bandwidth requirement, which does
not mean QoS capable routers are acting as a host.

> emmitting the reassembled result.

As I pointed out, if it were acceptable, we could widely accept
fragmentation overhead.

> participating in v6ops) is just a terminology issue? ;-)

People in v6 committee are not acting as professionals.

							Masataka Ohta

---

From: Joe Touch <[email-masked]>
Date: Sun, 14 Feb 2016 20:00:16 -0800

On 2/13/2016 5:26 PM, Masataka Ohta wrote:
> QoS (not CoS but real QoS) capable routers must inspect L4.

That must be why there are QoS indicators in the L4 header.

Oh, wait - those are in L3 (RFC2474 and its successors).

Yes, layering is a difficult concept.

Joe

---

From: Masataka Ohta <[email-masked]>
Date: Mon, 15 Feb 2016 16:06:19 +0900

> That must be why there are QoS indicators in the L4 header.

There are not.

> Oh, wait - those are in L3 (RFC2474 and its successors).

Can you read this?

	QoS (not CoS but real QoS)

> Yes, layering is a difficult concept.

Layering is trivially easy. If you think it difficult,
that is your problem.

						Masataka Ohta

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 16 Feb 2016 07:43:23 +1300

> Yes, layering is a difficult concept.

That's also why the IPv6 flow label is recommended to be a hash
of layer 3 and layer 4 information (RFC 6437, yes, it took us some
years to get that ~right).

   Brian

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 17 Feb 2016 17:08:36 +0900

Brian E Carpenter wrote:

> of layer 3 and layer 4 information

> (RFC 6437, yes, it took us some years to get that ~right).

Get that right? Or, is "~" a symbol of logical negation?

The RFC is a complete mess, in various ways. It says flow IDs are
good because it is random, but, at the same time, it says flow
IDs may not be random. It also says flow ID must not be changed
but may be changed.

The most obvious defect can be seen here:

   As a general practice, packet flows should not be reordered,

which is subtly wrong (order of consecutive packets may sometimes
be exchanged without harming TCP performance), but badly
contradicts with:

      *  A forwarding node might use the 5-tuple to define a flow
         whenever possible but use the 2-tuple when the complete 5-tuple
         is not available.  In this case, unfragmented and fragmented
         packets belonging to the same transport session would receive
         different flow label values, altering the effect of subsequent
         load distribution based on the flow label

and, another option of:.

      *  A forwarding node might use the 2-tuple to define a flow in all
         cases.  In this case, subsequent load distribution would be
         based only on IP addresses.

is not a efficient way of load balancing.

So, sane implementers of load distributors will ban packets with
extension headers and/or fragmentation to keep relying on 5-tuples.

Oh, and you are one of a editor of the mess. Thank you very much for
contributing to the collective stupidity to make simple things
unusably complex.

						Masataka Ohta

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 18 Feb 2016 17:27:56 +0900

Masataka Ohta (I) wrote:

> IDs may not be random.

I found the rfc is even worse.

The most important thing the rfc must have stated (it
does not, of course) is:

	(SRC1, DST1, flow_ID1)

of a stateful flow MUST be unique (not used by packets
not belonging to the flow) within the Internet,
which can be guaranteed only by an end (source or
destination), which is a straight forward manifestation
of the end to end argument.

But, the rfc allow routers (firewalls) change flow IDs to
nonzero value.

So, if a router changes flow ID of (SRC1, DST1, flow_ID2),
from flow_ID2 to flow_ID3, then, there is a possibility
that flow_ID1==flow_ID3, which is fatal for the stateful
flow, if the modified packets are merged to the stateful
flow (certain protection against merging possible but
not robust against route changes).

Of course, section 6.1 of the rfc on covert channels is
abstract nonsense, because covert channels may be created
in various ways to carry information, for example, with
extension headers (fragmentation boundaries, for example,
can be arbitrary), which means firewalls should reject
packets with extension headers.

					Masataka Ohta

---

From: Mark Andrews <[email-masked]>
Date: Fri, 19 Feb 2016 07:15:19 +1100

> packets with extension headers.

No, it doesn't.  Firewalls have a purpose.  Most of the time the
purpose isn't to block communication.  It is to block wasting
resources or to try to prevent poorly written applications / ip
stacks being compromised.

Often people forget that firewalls need to let packets though that
are part of a legitimate communications flow through.  You don't
actually need to stop *every* potential packet that isn't part of
a communications flow.  You just need to make it hard enough that
it is not worth the effort to find the open paths if you are not
part of a legitimate flow.