Subject: Re: [Int-area] Recycle IPv4 bits

From: Teco Boot <[email-masked]>
Date: Fri, 1 Apr 2011 10:14:13 +0200

Hi Joe,

You made me more optimistic. 

Op 31 mrt 2011, om 19:45 heeft Joe Touch het volgende geschreven:

> Hi, Deco,
It is Teco.

> 
> This proposal is a moot point; the FO needs to be zero for a packet to be ATOMIC. see below:
> 
> On 3/31/2011 4:41 AM, Teco Boot wrote:
>> Having a discussion on recycling used IPv4 header bits, I made
>> a snapshot on what was in the air at same point in time, somewhere.
>> I found out the ID field of atomic packets was filled for almost
>> all packets.
>> 
>> IPv4 total:      151081
>> Atomic packets:  105357 (70%)
>> On atomic packets:
>>   ID == 0     1557 (  1,5%)
>>   ID != 0   103800 ( 98,5%)
>>   FO == 0   105357 (100,0%)
>> 
>> The Fragment Offset was always set on zero, I read this as unused.
> 
> That is incorrect. It is part of what makes a packet atomic, as per the ipv4-id-update draft:
> 
> 	(DF==1)&&(MF==0)&&(frag_offset==0)

But this is a modification of RFC 791:
    An internet datagram can be marked "don't fragment."  Any internet
    datagram so marked is not to be internet fragmented under any
    circumstances.
It is reported there were packets around with illegal fragmentation.
But since when do we adjust specs because there are bugs or non-compliant 
implementations? Update RFC 4963 also, where such behavior is disapproved?

> 
> Let's look at the cases:
> 
> DF MF FO
> 0  x  x  	CAN FRAGMENT
> 		non-atomic packet that can be fragmented downstream;
> 		that fragmentation would overwrite MF and FO

This leads towards:
  ! (DF=0) == ! ( CAN FRAGMENT) == CANNOT FRAGMENT  == ATOMIC
Or:
  (DF=1) == ATOMIC

> 1  0  0		ATOMIC		
> 		basically this is a lot like saying
> 		"this is the first and last fragment"
> 
> 1  0  nonzero	LAST FRAGMENT
> 		packet has already been fragmented and
> 		this is the last fragment
> 
> 1  1  0		FIRST FRAGMENT
> 		packet has already been fragmented, and this is the
> 		first fragment and NOT the last (other fragments
> 		with higher offsets are expected)
> 
> 1  1  nonzero	MIDDLE FRAGMENT
> 		packet has already been fragmented, and this is
> 		not the first fragment, and other fragments are
> 		expected

This makes me optimistic: packets with DF=1 and FO!=0 are valid and
intermediate nodes MUST NOT drop such packets.

When DF=1, MF is unused also, so there are 14 bits waiting to be recycled.

> Note that thus:
> 
> 	ATOMIC packets are *defined* by having all the FO bits as zero.
> 
> > What comes up in my mind: can we recycle the 13 fragment offset bits?
> > Instead of ID field? At least it may be a well defined starting point.
> 
> When the bits aren't zero, an otherwise ATOMIC packet is really a last fragment.
> 
> So if you try to use those bits, all you will accomplish is flooding the reassembly cache.

No. RFC 791 compliant nodes receiving such packets SHOULD drop these. 
Re-usage of the bits for atomic packets needs updated nodes. A requirement 
would be a test if the bits can be used.

Thanks, Teco

> Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 01 Apr 2011 02:16:05 -0700

On 4/1/2011 1:14 AM, Teco Boot wrote:
...
> implementations? Update RFC 4963 also, where such behavior is disapproved?

I was never talking about a case where the DF bit is ignored.

>    ! (DF=0) == ! ( CAN FRAGMENT) == CANNOT FRAGMENT  == ATOMIC

DF=1 means "don't fragment what I just sent".

The source can fragment, though, and THEN set DF on the fragments. That 
is exactly how an IPv4 host most closely behaves like an IPv6 host - the 
latter permitting only source fragmentation.

This is why ipv4-id-update doesn't use the term "DF=1"; it uses the term 
ATOMIC because atomic means:
	- must not fragment from this point forward
	AND
	- has not already been fragmented yet

DF=1 needs to be preserved to avoid interfering with upstream detection 
of MTUs along a path, but it can be changed from 0 to 1 by a router. 
However, even ignoring that case, DF=1 isn't enough to know the 
difference between packets that have already been fragmented by the source.

> intermediate nodes MUST NOT drop such packets.

Absolutely. They're packets that have already been fragmented, e.g., by 
the source. That also means you can't use the FO field, though.

> When DF=1, MF is unused also, so there are 14 bits waiting to be recycled.

No; see above. These are valid cases of source fragmentation.

> No. RFC 791 compliant nodes receiving such packets SHOULD drop these.

If you do, you're disabling source fragmentation. RFC 791 definitely 
does not prohibit that.

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 01 Apr 2011 02:44:51 -0700

PS - this is discussed indirectly in Sec 3 of the ipv4-id-update doc, 
but it seems clear from this discussion that further explanation of 
source vs. downstream fragmentation would be useful to include.

I'll update the ID shortly with that info, as well as confirming whether 
downstream routers can change DF. They clearly MUST NOT change from 1 to 
0, but it nothing appears to prohibit them from changing it from 0 to 1 
it's not mentioned in RFC 1812 or its updates), and that's dangerous 
because that router might not be on the return path of the ICMP too-big 
messages, so they wouldn't be able to react and thus the change has no 
utility. It may be useful to add a MUST NOT (a SHOULD NOT at least, but 
I can't think of a safe exception) on setting the DF bit as a result.

Joe

PS - RFC 4963 is informational; updates apply to standards-track docs 
only AFAICT. I'll add that to the doc as well, though, since it does 
include discussions of some of the issues discussed.

---

From: Teco Boot <[email-masked]>
Date: Fri, 1 Apr 2011 11:58:44 +0200

Op 1 apr 2011, om 11:16 heeft Joe Touch het volgende geschreven:

> The source can fragment, though, and THEN set DF on the fragments. That is exactly how an IPv4 host most closely behaves like an IPv6 host - the latter permitting only source fragmentation.
> 
> This is why ipv4-id-update doesn't use the term "DF=1"; it uses the term ATOMIC because atomic means:
> DF=1 needs to be preserved to avoid interfering with upstream detection of MTUs along a path, but it can be changed from 0 to 1 by a router. However, even ignoring that case, DF=1 isn't enough to know the difference between packets that have already been fragmented by the source.

The RFC 791 DF example (page 25) describes the use case where DF=1 packets are atomic, and both source and destination are aware. When we update RFC 791 for reuse FO, source and destination must make sure the bits are recycled. As you pointed out, nodes in the middle MUST handle such packets as normal.

> Absolutely. They're packets that have already been fragmented, e.g., by the source. That also means you can't use the FO field, though.

FO can be reused, if both source and destination choose to do so.

> No; see above. These are valid cases of source fragmentation.

If hosts follow RFC 791.

> If you do, you're disabling source fragmentation. RFC 791 definitely does not prohibit that.

Nor does describe it.

This brings us to recycle, or not. I don't see any technical hindrance anymore.

Teco

---

From: Joe Touch <[email-masked]>
Date: Fri, 01 Apr 2011 03:34:12 -0700

On 4/1/2011 2:58 AM, Teco Boot wrote:
...
>> The source can fragment, though, and THEN set DF on the fragments.
>> That is exactly how an IPv4 host most closely behaves like an IPv6
>> host  - the latter permitting only source fragmentation.
>>
>> DF=1 needs to be preserved to avoid interfering with upstream
>> detection of MTUs along a path, but it can be changed from 0 to 1
>> by a router. However, even ignoring that case, DF=1 isn't enough to
>> know the difference between packets that have already been
>> fragmented by the source.
>
> The RFC 791 DF example (page 25) describes the use case where DF=1
> packets are atomic, and both source and destination are aware.

The source knows it's setting DF, but the destination has no way of 
knowing that. As I noted in a subsequent post, there doesn't appear to 
be any prohibition on routers changing DF=0 to DF-1, even though it 
doesn't seem safe.

> When we update RFC 791 for reuse FO, source and destination must make
> sure the bits are recycled. As you pointed out, nodes in the middle
> MUST handle such packets as normal.

The ipv4-id-update doc does *not* update RFC791 to reuse FO, just to be
clear. If you want to do that, there's much more work to do - e.g., to 
check whether existing routers set DF.

> FO can be reused, if both source and destination choose to do so.

There are two possibilities:

- a source NEVER sets DF=0

- a source sets DF=0 and no routers on the path ever set it

ipv4-id-update doesn't prohibit either case per se, and the receiver 
cannot know either case just from the packets it sees

Even if we describe how routers setting DF is dangerous, we can't know 
when it's happening.

>> No; see above. These are valid cases of source fragmentation.

Sure; we're surely not talking about violations of spec. There's no 
point in discussing that case further.

>> If you do, you're disabling source fragmentation. RFC 791 definitely does not prohibit that.

There are two behaviors not explicitly described by RFC791:
A- valid uses of source fragmentation with DF=1
B- dropping source fragmented packets where DF=1, solely for that reason

The example reassembly algorithm in sec 3.2 does not examine the DF bit, 
which permits case A.

However, case B is prohibited by RFC1191, since it would disable all of 
source fragmentation.

> This brings us to recycle, or not. I don't see any technical hindrance anymore.

There are at least three problems:

1) dropping fragments where DF=1 isn't required by any RFC

2) dropping fragments where DF=1 would break current standards, e.g., 
PMTUD RFC1191 (draft standard)

3) dropping fragments where DF=1 for IPv4 would prohibit the defining 
case for fragmentation in IPv6 (which explicitly describes its use of 
fragmentation in terms of "DF=1 always set, source permitted")

Finally, I'd wager that the *bulk* of current fragments have DF=1.

Joe

---

From: Teco Boot <[email-masked]>
Date: Fri, 1 Apr 2011 14:06:18 +0200

Op 1 apr 2011, om 12:34 heeft Joe Touch het volgende geschreven:

> The source knows it's setting DF, but the destination has no way of knowing that.

FO re-usage is based on DF=1. Coordination is required.

> As I noted in a subsequent post, there doesn't appear to be any prohibition on routers changing DF=0 to DF-1, even though it doesn't seem safe.

We discuss DF=1.

> clear. If you want to do that, there's much more work to do - e.g., to check whether existing routers set DF.

FO re-use expects existing routers do not clear DF. This is prohibited, as you posted today.

> - a source sets DF=0 and no routers on the path ever set it

The conditions are not written down yet. The basics are DF=1 and source and destination do cooperate. Usage is for specific flows.

> ipv4-id-update doesn't prohibit either case per se, and the receiver cannot know either case just from the packets it sees

The receiver knows, based on cooperation.

> Even if we describe how routers setting DF is dangerous, we can't know when it's happening.
> 
> Sure; we're surely not talking about violations of spec. There's no point in discussing that case further.
> 
>>>>> Note that thus:
>>>>> 
> The example reassembly algorithm in sec 3.2 does not examine the DF bit, which permits case A.
> 
> However, case B is prohibited by RFC1191, since it would disable all of source fragmentation.

OK. We don't need to discuss fragmentation and PMTU.

> 1) dropping fragments where DF=1 isn't required by any RFC
OK.

> 2) dropping fragments where DF=1 would break current standards, e.g., PMTUD RFC1191 (draft standard)
OK.

> 3) dropping fragments where DF=1 for IPv4 would prohibit the defining case for fragmentation in IPv6 (which explicitly describes its use of fragmentation in terms of "DF=1 always set, source permitted")
OK.

> Finally, I'd wager that the *bulk* of current fragments have DF=1.

I did't see one yet. And if such packets fly around, nothing breaks. It just makes clear that not all bits can be recycled.

Teco

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 1 Apr 2011 05:31:31 -0700

Teco,

I know of at least one RFC that seems to suggest that it
is OK to fragment IPv4 packets regardless of the DF setting.
I have not personally collected any data, but I have seen
suggestions that there is currently-deployed networking
gear that ignores the DF setting.

As Joe knows, I have tried for many years and have come
to the conclusion that trying to code the IP header is a
dubious enterprise. Perhaps it can be done to a certain
extent within a bounded and controlled network, but not
so much out in the wild of the public Internet.

Fred  

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Teco Boot
> Sent: Friday, April 01, 2011 5:06 AM
> To: Joe Touch
> Cc: [email-masked]
> Subject: Re: [Int-area] Recycle IPv4 bits
> 
> > As I noted in a subsequent post, there doesn't appear to be 
> any prohibition on routers changing DF=0 to DF-1, even though 
> it doesn't seem safe.
> 
> We discuss DF=1.
> 
> > clear. If you want to do that, there's much more work to do 
> - e.g., to check whether existing routers set DF.
> 
> >>> Absolutely. They're packets that have already been 
> fragmented, e.g., by the source. That also means you can't 
> use the FO field, though.
> >> 
> The conditions are not written down yet. The basics are DF=1 
> and source and destination do cooperate. Usage is for specific flows.
> 
> > ipv4-id-update doesn't prohibit either case per se, and the 
> receiver cannot know either case just from the packets it sees
> 
> > Sure; we're surely not talking about violations of spec. 
> There's no point in discussing that case further.
> > 
> >>>>> Note that thus:
> >>>>> 
> >>>>> So if you try to use those bits, all you will 
> accomplish is flooding the reassembly cache.
> >>>> 
> > The example reassembly algorithm in sec 3.2 does not 
> examine the DF bit, which permits case A.
> > 
> > 2) dropping fragments where DF=1 would break current 
> standards, e.g., PMTUD RFC1191 (draft standard)
> OK.
> 
> > 3) dropping fragments where DF=1 for IPv4 would prohibit 
> the defining case for fragmentation in IPv6 (which explicitly 
> describes its use of fragmentation in terms of "DF=1 always 
> set, source permitted")
> OK.
> 
> I did't see one yet. And if such packets fly around, nothing 
> breaks. It just makes clear that not all bits can be recycled.
> 
> Teco
> 
> _______________________________________________
> Int-area mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/int-area
>

---

From: Teco Boot <[email-masked]>
Date: Fri, 1 Apr 2011 14:56:00 +0200

Fred,

It is april 1st. Has the RFC to do with such date?
And again, since when do we respect bugs?

Packets in the wild _can_ be manipulated. Especially middleboxes that provide protection for payload sometimes illegally change headers.

Teco

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 1 Apr 2011 06:14:51 -0700

Teco, 

> From: Teco Boot [mailto:[email-masked]] 
> Sent: Friday, April 01, 2011 5:56 AM
> To: Templin, Fred L
> Cc: Joe Touch; [email-masked]

Not an April Fools RFC, and I'm sure implementatons
don't care so much which specs are violated.

> Packets in the wild _can_ be manipulated. Especially 
> middleboxes that provide protection for payload sometimes 
> illegally change headers.

Sure, but wouldn't that invalidate your desire to code
something in the header at the source that you could
count on still being there in the same form when the
packet reaches the destination?

Thanks - Fred

---

From: Teco Boot <[email-masked]>
Date: Fri, 1 Apr 2011 15:40:30 +0200

> don't care so much which specs are violated.

Please provide a pointer.

> packet reaches the destination?

The only guarantee to prevent malformed packets at destination is to keep them at the source.

Teco

---

From: Joe Touch <[email-masked]>
Date: Fri, 01 Apr 2011 07:06:40 -0700

> is OK to fragment IPv4 packets regardless of the DF setting.

Hi, Fred,

Can you forward that so I can cite it specifically?

> gear that ignores the DF setting.

I hope we all know that this is the case ;-)

It's also the case that existing sources reuse the ID, either when 
sending more than 6.4 Mbps on a connection - or, in some cases, for all 
packets. Ignoring the DF bit ends up creating more errors, which either 
behaves like a flakey link (if dropped) or a flakey OS. I wouldn't be 
surprised if some uses that result in phones or PCs acting odd or 
rebooting is caused by this, FWIW.

This is why the ID tries not to *rely* on any proposed changes; it just 
recognizes existing behavior.

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 01 Apr 2011 07:10:14 -0700

On 4/1/2011 6:40 AM, Teco Boot wrote:
...
> The only guarantee to prevent malformed packets at destination is to keep them at the source.

The whole point of a protocol is to provide some level of guarantee. 
Otherwise, we would be able to just send raw data over the wire (e.g., 
as we do inside a host, in some cases).

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 01 Apr 2011 07:11:20 -0700

Hi, Teco,

On 4/1/2011 5:06 AM, Teco Boot wrote:
...
> FO re-usage is based on DF=1. Coordination is required.

What protocol provides this coordination?

> FO re-use expects existing routers do not clear DF. This is prohibited, as you posted today.

FO reuse expects that routers do not clear the DF (which is prohibited), 
but also expects that routers do not SET the DF (which is not currently 
prohibited).

The second case is the problem. I agree that we now should explain the 
dangers of setting the DF anywhere except the source, but we cannot 
simply assume that this is always the case.

...
> OK. We don't need to discuss fragmentation and PMTU.

If we're talking about fragmentation, we really do. That's the common 
use for setting DF=1.

Joe

---

From: Teco Boot <[email-masked]>
Date: Mon, 4 Apr 2011 08:29:01 +0200

Fred,
 
Op 1 apr 2011, om 15:52 heeft Templin, Fred L het volgende geschreven:

> Teco,
> 
> I am looking at RFC3931, Section 4.1.4. All it says is
> that the LCCE may fragment IPv4 packets

I don't read RFC3931 like this. It says ideal solutions work with 
RFC1191. And RFC2661 is normative reference, which point to RFC791.

> - it does not
> say that the LCCE must honor the DF bit.

It doesn't say the LCCE must not do illegal operations. I do agree 
that it were better to mention a DF=0 condition. Make errata?
Old:
   An LCCE MAY fragment a packet before encapsulating it in L2TP.  For
   example, if an IPv4 packet arrives at an LCCE from a Remote System
   that, after encapsulation with its associated framing, L2TP, and IP,
   does not fit in the available path MTU towards its LCCE peer, the
   local LCCE may perform IPv4 fragmentation on the packet before tunnel
   encapsulation. 
New:
   An LCCE MAY fragment a packet before encapsulating it in L2TP.  For
   example, if an IPv4 packet with DF=1 arrives at an LCCE from a Remote System
   that, after encapsulation with its associated framing, L2TP, and IP,
   does not fit in the available path MTU towards its LCCE peer, the
   local LCCE may perform IPv4 fragmentation on the packet before tunnel
   encapsulation. 

Thanks, Teco

>> Sent: Friday, April 01, 2011 6:41 AM
>> To: Templin, Fred L
>> The only guarantee to prevent malformed packets at 
>> destination is to keep them at the source.
>> 
>> Teco
>> 
>> 
>>

---

From: Teco Boot <[email-masked]>
Date: Mon, 4 Apr 2011 08:48:26 +0200

Hi Joe,

> What protocol provides this coordination?

Currently none is specified. One suggested reusing the evil bit as Recycled Flag.

> FO reuse expects that routers do not clear the DF (which is prohibited), but also expects that routers do not SET the DF (which is not currently prohibited).

I don't get this. When FO is reused, DF MUST be set at source.

> The second case is the problem. I agree that we now should explain the dangers of setting the DF anywhere except the source, but we cannot simply assume that this is always the case.
> 
> ...
> If we're talking about fragmentation, we really do. That's the common use for setting DF=1.

If we can't enforce fragmentation to be disabled, we can't reuse FO.

Teco

---

From: Joe Touch <[email-masked]>
Date: Mon, 4 Apr 2011 06:36:18 -0700

Hi Teco, et al., 

>   example, if an IPv4 packet with DF=1 arrives at an LCCE from a Remote System

If DF=0

---

From: Teco Boot <[email-masked]>
Date: Mon, 4 Apr 2011 16:24:12 +0200

...
>> Make errata?

Done.
http://www.rfc-editor.org/errata_search.php?eid=2766

Teco

---

From: Joe Touch <[email-masked]>
Date: Mon, 04 Apr 2011 10:34:17 -0700

Hi, Teco (et al.),

> Currently none is specified. One suggested reusing the evil bit as Recycled Flag.

The evil bit was an Apr 1 RFC. Presumably you're thinking of the 
high-order bit of the offset. However...

> I don't get this. When FO is reused, DF MUST be set at source.

If DF is set at the source, and FO != 0, then all you have described is 
source fragmentation.

That is the desired way to fragment; it most directly corresponds to 
IPv6 fragmentation. In that case, all bits of FO are used; there is no 
available 'high' bit.

So your mechanism works only if there is no source fragmentation. That's 
not realistic.

> If we can't enforce fragmentation to be disabled, we can't reuse FO.

I'm not following you.

You are seeking, I presume, to reuse bits from FO when the source has 
not fragmented the packet AND the DF flag is set so no further 
fragmentation is possible.

However, when DF=1 at the source, and FO != 0, you have a fragment 
already, and FO is meaningful.

Can we put this issue to rest?

Joe

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 5 Apr 2011 12:31:21 +0200

On 1 apr 2011, at 11:44, Joe Touch wrote:

> I'll update the ID shortly with that info, as well as confirming whether downstream routers can change DF. They clearly MUST NOT change from 1 to 0, but it nothing appears to prohibit them from changing it from 0 to 1 it's not mentioned in RFC 1812 or its updates),

I assume this is in the spirit of the date of your message.

We have enough broken PMTUD as it is without people creating even more. One of the IP layer's jobs is to adjust to varying MTUs. That can only happen in two ways: PMTUD and fragmentation in routers. With IPv4, PMTUD is not mandatory, and people sending packets with DF=0 is a good indication that PMTUD can't be performed. Don't forget that with IPv4, unlike with IPv6, fragmentation at the source based on too big messages is not performed, only resegmentation in the case of TCP. And even if it were, a packet gets lost and only the retransmission is fragmented at the source, which can be very bad for the performance of protocols such as DNS.

There is really no upside to messing with all this stuff at this point in time.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 5 Apr 2011 12:33:42 +0200

> gear that ignores the DF setting.

I know I brought up the fact that I configured a dial-up concentrator to do exactly this in the past. The tunnel to the modems would not support MTUs larger than 576 bytes, causing many PMTUD problems. Most people "fix" this with TCP MSS rewriting (or "clamping"), though.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 5 Apr 2011 12:34:26 +0200

> And again, since when do we respect bugs?

Since when do we go out of our way to create new specs that break existing behavior?

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 5 Apr 2011 08:00:58 -0400

> Since when do we go out of our way to create new specs that break existing behavior?

I fully agree.  The IETF isn't the Internet Cops.  We're supposed to
be about interoperability.  This proposal so far seems not to talk
about its potential interoperability effects, and I therefore think it
should not be pursued, at least until there's a pretty good argument
that it is unlikely to break things and that whatever gains there are
to be had are greater than the costs of whatever breakage it causes.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Joe Touch <[email-masked]>
Date: Tue, 05 Apr 2011 11:47:14 -0700

Hi, Iljitsch,

On 4/5/2011 3:31 AM, Iljitsch van Beijnum wrote:
>> I'll update the ID shortly with that info, as well as confirming
> whether downstream routers can change DF. They clearly MUST NOT change
> from 1 to 0, but it nothing appears to prohibit them from changing it
> from 0 to 1 it's not mentioned in RFC 1812 or its updates),
>

I'll remember to avoid email altogether next Apr 1.

> We have enough broken PMTUD as it is without people creating even
> more. One of the IP layer's jobs is to adjust to varying MTUs. That can
> only happen in two ways: PMTUD and fragmentation in routers. With IPv4,
> PMTUD is not mandatory, and people sending packets with DF=0 is a good
> indication that PMTUD can't be performed. Don't forget that with IPv4,
> unlike with IPv6, fragmentation at the source based on too big messages
> is not performed, only resegmentation in the case of TCP.

RFC792 says that fragmentation needed ICMPs are sent back to the 
originating source, and cites RFC791. Note - it does NOT cite RFC793.

So I can only conclude that the whole point of the too-big is to get the 
source to fragment *at the source*. Source fragmentation definitely 
occurs when the outgoing message is bigger than the interface supports, 
but I'd have to do some checking on what actually happens when an ICMP 
too-big is received. I'd be surprised if that isn't handled in some way, 
though. If that's the case, then DF=0 is the desired behavior, though, 
because you can't assume that devices on the path ignore DF=1 (i.e., 
safe source behavior is always better than expecting noncompliant 
behavior downpath).

> And even if it
> were, a packet gets lost and only the retransmission is fragmented at
> the source, which can be very bad for the performance of protocols such
> as DNS.

Your message is confusing; I am not suggesting we prohibit packets with 
DF=0.

I'm saying that once a packet is sent with DF=0, changing it to DF=1 
en-route is dangerous and serves no useful purpose AFAICT.

Joe

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 5 Apr 2011 22:33:28 +0200

Hi Joe,

> I'll remember to avoid email altogether next Apr 1.

:-)

>> Don't forget that with IPv4,

> RFC792 says that fragmentation needed ICMPs are sent back to the originating source, and cites RFC791. Note - it does NOT cite RFC793.

You are reading this with today's notion of PMTUD in mind. For IPv4, "too big" is a subtype of destination unreachable, which typically doesn't carry the connotation that a retransmission is useful. (For IPv6 "too big" is no longer grouped under unreachable.)

Because IPv4 never had ICMP feedback related fragmentation at the source (just "whatever fits in the MTU") de facto IPv4 PMTUD only works with TCP because unlike most UDP protocols, TCP can arbitrarily adjust its packet size without any higher layer implications.

> I'd have to do some checking on what actually happens when an ICMP too-big is received. I'd be surprised if that isn't handled in some way, though.

Prepare to be surprised. Perhaps this is brought into line with IPv6 in newer implementations recently, but I've never seen it happen.

> I'm saying that once a packet is sent with DF=0, changing it to DF=1 en-route is dangerous and serves no useful purpose AFAICT.

Ok, then we agree on that part.

Iljitsch

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 6 Apr 2011 10:47:38 -0700

Hi Iljitsch, 

> [mailto:[email-masked]] On Behalf Of Iljitsch van Beijnum
> Sent: Tuesday, April 05, 2011 1:33 PM
> To: Joe Touch
> Cc: Internet Area
> > RFC792 says that fragmentation needed ICMPs are sent back 
> to the originating source, and cites RFC791. Note - it does 
> NOT cite RFC793.
> 
> You are reading this with today's notion of PMTUD in mind. 
> For IPv4, "too big" is a subtype of destination unreachable, 
> which typically doesn't carry the connotation that a 
> retransmission is useful. (For IPv6 "too big" is no longer 
> grouped under unreachable.)
> 
> Because IPv4 never had ICMP feedback related fragmentation at 
> the source (just "whatever fits in the MTU") de facto IPv4 
> PMTUD only works with TCP because unlike most UDP protocols, 
> TCP can arbitrarily adjust its packet size without any higher 
> layer implications.

While it is true that some UDP applications have no way
to reduce the size of packets they send (and therefore
set DF=0), I don't think it is possible to say that
"PMTUD only works with TCP". Do you know of any surveys
showing that PMTUD is broken for other than TCP?

Thanks - Fred
[email-masked] 

> > I'd have to do some checking on what actually happens when 
> an ICMP too-big is received. I'd be surprised if that isn't 
> handled in some way, though.
> 
> Prepare to be surprised. Perhaps this is brought into line 
> with IPv6 in newer implementations recently, but I've never 
> seen it happen.
> 
> > I'm saying that once a packet is sent with DF=0, changing 
> it to DF=1 en-route is dangerous and serves no useful purpose AFAICT.
>

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 6 Apr 2011 22:53:44 +0200

Hi Fred,

On 6 apr 2011, at 19:47, Templin, Fred L wrote:

>> de facto IPv4 
>> PMTUD only works with TCP

> showing that PMTUD is broken for other than TCP?

Only what I've seen myself.

You're right of course that there is no reason why IPv4 can't handle this at the IP layer like IPv6 does, but it just doesn't happen in practice. Hence the "de facto" above.

It's such a shame that we never got this right. Maybe with IPvA...

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 6 Apr 2011 17:23:43 -0700

Hi Iljitsch, 

> Sent: Wednesday, April 06, 2011 1:54 PM
> To: Templin, Fred L
> Cc: Internet Area
> You're right of course that there is no reason why IPv4 can't 
> handle this at the IP layer like IPv6 does, but it just 
> doesn't happen in practice. Hence the "de facto" above.
> 

Any ideas on how we are going to get to 9KB jumbograms
everywhere then, or are we just going to punt on that?

Thanks - Fred

---

From: Joel Jaeggli <[email-masked]>
Date: Wed, 6 Apr 2011 18:29:29 -0700

Joel's widget number 2

> everywhere then, or are we just going to punt on that?

Still need pmtud when you have jumbo frames support because you don't know how many encapsulations there are on the wire.

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 7 Apr 2011 10:39:11 +0200

Hi Fred,

> everywhere then, or are we just going to punt on that?

Thanks for the setup.

Obviously there will be stuff out there that is limited to 1500 or 1280 bytes for a long, long time. But hopefully there will also be stuff that can do more than 9k. We shouldn't start an effort to upgrade the whole internet to 9k because then that will be just as unpleasant a limitation as 1500 is today, what we need is a way for nodes with different MTUs to live together on the same subnet. Then, the ones that support 9k or more can use that packet size between them without leaving the nodes with lesser capabilities in the cold. My draft has mechanisms to accomplish that:

http://tools.ietf.org/html/draft-van-beijnum-multi-mtu-03

---

From: Carlos Pignataro <[email-masked]>
Date: Fri, 22 Apr 2011 00:08:20 -0400

Joe,

Please find one comment inline, which may have been discussed already in
the thread, apologies if I missed it.

> 		not the first fragment, and other fragments are

In these last 3 cases you include fragments (first, middle, last) with
the DF bit set. This mimics IPv6's source-only fragmentation, but it is
not clear to me how common that is.

But if this is a spec exercise, RFC791 says 'An internet datagram can be
marked "don't fragment."' Agreed that perhaps such datagram can be a
complete datagram or a fragment (although I read the intention as
meaning a complete packet), but the use case is that so that the
receiver does not reassemble (hence meaning that it makes sense only in
a complete datagram). The examples of fragmentation procedures in RFC791
do not set the DF bit. I know of systems that source-fragment without
setting the DF bit, and a fragment can in turn be fragmented in flight.
If the source is setting DF for fragments, is not following the
procedure in RFC 791?

Thanks,

-- Carlos.

---

From: Carlos Pignataro <[email-masked]>
Date: Fri, 22 Apr 2011 00:08:17 -0400

Teco, Fred, Joe,

Apologies for jumping in late. Please see inline.

>> that the LCCE may fragment IPv4 packets

I do not agree with this interpretation and I have a different read of
the second para of S4.1.4 of RFC 3931. This text says:

   For

And the key here is "the local LCCE may perform IPv4 fragmentation".
Performing IPv4 fragmentation does not imply that the result is
fragmenting a datagram (or a fragment). The result of the fragmentation
procedure can just be check the DF and discard.

When RFC 791 describes the fragmentation procedures, the first step is
to compare the TL with the MTU, and if the TL > MTU then check the DF,
and discard the datagram if DF is set. That is, check DF and discard is
part of the "fragmentation procedure". This means that "perform IPv4
fragmentation" can be: if the datagram is too big for the tunnel, then
check DF and if set then drop and send an ICMP3/4 to the source (with a
Next-Hop MTU that accounts for the tunnel encap). I am aware of
implementations that work like this. And this allows for PMTUD to work
as if the tunnel is just another link.

Excerpts of RFC 791 that support this are:

    An Example Fragmentation Procedure

      Procedure:

        IF TL =< MTU THEN Submit this datagram to the next step
             in datagram processing ELSE IF DF = 1 THEN discard the
        datagram ELSE
        To produce the first fragment:

The paragraph of RFC3931 describes pre-fragmentation (i.e.,
fragmentation before encapsulation). This is further covered in the
later S3.4 of RFC 4459:

http://tools.ietf.org/html/rfc4459#section-3.4

3.4.  Fragmentation of the Inner Packet

   A final possibility is fragmenting the inner packet, before
   encapsulation, in such a manner that the encapsulated packet fits in
   the tunnel's path MTU (discovered using PMTUD).  However, one should
   note that only IPv4 supports this "in-flight" fragmentation;
   furthermore, it isn't allowed for packets where the Don't Fragment
   bit has been set.  Even if one could ignore IPv6 completely, so many
   IPv4 host stacks send packets with the DF bit set that this would
   seem unfeasible.

> that it were better to mention a DF=0 condition. Make errata?

I will provide my 2¢ to this errata on the l2tpext list.

Thanks,

-- Carlos.

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 22 Apr 2011 09:54:43 -0700

Section 8 of RFC4821 discusses host fragmentation using
the same technique Joe is describing.

Thanks - Fred

> [mailto:[email-masked]] On Behalf Of Carlos Pignataro
> Sent: Thursday, April 21, 2011 9:08 PM
> To: [email-masked]
> > This proposal is a moot point; the FO needs to be zero for 
> a packet to 
> > be ATOMIC. see below:
> > 
> > That is incorrect. It is part of what makes a packet 
> atomic, as per the 
> > ipv4-id-update draft:
> > 
> > When the bits aren't zero, an otherwise ATOMIC packet is 
> really a last 
> > fragment.
> > 
> > So if you try to use those bits, all you will accomplish is 
> flooding the 
> > reassembly cache.
> > 
> > Joe

---

From: "Templin, Fred L" <[email-masked]>
Date: Fri, 22 Apr 2011 10:06:35 -0700

Carlos,

It has been suggested in the past that "various types of
VPN/tunneling equipment have resorted to ignoring the
DF bit under some circumstances" (see Section 9.3 of
'draft=ietf-pmtud-method-04.txt'). Also, some documents
have taken it upon themselves to be more explicit in their
specifications of inner fragmentation relating to tunneling
(see Section 5.4.1 of 'draft-ietf-lisp-11.txt'). 

Note that I never posted anything regarding RFC3931 to
the list nor agreed that any errata should be written.

Thanks - Fred

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Fri, 22 Apr 2011 20:43:47 -0400

Sure, but this technique is not in RFC 791 which is what I understand the ipv4-id I-D cites and is based on. For the purposes of the ident field, for a fragment (first, middle, or last) the DF value is a don't care.

Thanks,

Thumb typed by Carlos Pignataro.