Subject: Advanced sockets API draft (rfc2292bis) query

From: Robert Elz <[email-masked]>
Date: Wed, 05 Apr 2000 22:03:21 +1000

There has been a little off line discussion about DNS and IPv6,
which led (due to a "I don't know of a way to ..." from me, and then a
"see section ... in draft ..." response) to an examination of
section 11.1 in draft-ietf-ipngwg-rfc2292bis-01.txt (this is
new since 2292).

After a brief explanation of what is being done, section 11.1
says ...

   This specification defines a mechanism to avoid fragmentation by
   sending at the minimum IPv6 MTU (1280 bytes).  This can be enabled
   using the IPV6_USE_MIN_MTU socket option.

Unfortunately, it doesn't say what that means (there's more text on
how to call the option, and what values mean on & off, etc).

The context is sending UDP packets ("A potential example is a DNS server").
The question now is what does "to avoid fragmentation" really mean here?

The best interpretation (for the DNS anyway) would be if the actual
meaning of this option was "to cause fragmentation" rather than "to
avoid fragmentation".   That is, when enabled, this option would cause
packets larger than specified (1280 as it is now) to be fragmented
from the source, so they should never fail with a packet to big error.

If that is the intent, and I suspect it might be, then "to avoid fragmentation"
seems like a very poor description, as what is being done is causing
fragmentation, not avoiding it.   Of course, that is what needs to be done
to allow large UDP packets to traverse the net (without requiring the
application to engage in PMTU discovery, which is a truly evil thing to
require of a DNS server, which will typically be sending exactly one packet
to the particular destination).

If that isn't what the option is intended to achieve, then two things
arise - first, something needs to be added to that section to say exactly
what it is meant to achieve (actually, that probably needs to be done
anyway) and second, an option that will cause fragmentation to occur
needs to be added.

Then, while on the topic, I feel this option would be better if instead
of taking a boolean value, it actually took an int value, so the
application could set the size above which fragments would be created.
This one isn't for DNS servers, for which 1280 seems like a fine value
to have set, but for other UDP applications for which use of PMTU discovery
(and the IPV6_RECVPATHMTU setsockopt defined in section 11.2) is a
reasonable thing to do - but which do need to be able to send single
packet datagrams which might be bigger than the MTU discovered.  There
fragmentation is still required, but 1280 is not necessarily the best
number to use (one possible example here is for DHCP servers, which when
lots of data needs to be conveyed to the client might need to send a packet
bigger than 1500 bytes - but which are sending it only to a few local
ethernets, all of which have an MTU of 1500, which can easily be discovered).

So, if a change is to be made to section 11.1, and I think one is eneded,
can I suggest that it also be changed to use an int value, where 1280 would
be a common case, and 0 would mean "don't fragment at all, send what I send"
equivalent to the current "off" status ... assuming I am guessing correctly
at what IPV6_USE_MIN_MTU actually means).

Matt?  Erik?   Is one of you currently responsible for this draft?

kre

--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: [email-masked]
Date: Wed, 05 Apr 2000 22:37:12 +0900

>   using the IPV6_USE_MIN_MTU socket option.

	I agree the "avoid fragmentation" is a little vague.

>ethernets, all of which have an MTU of 1500, which can easily be discovered).

	I'm not sure if I understand the intent here.  Why userland
	application needs to think about link MTU?

>at what IPV6_USE_MIN_MTU actually means).

	I believe the default behavior is not just "don't fragment at all".

	Normally, without any setsockopt, the kernel behavior would be like:
		if (cached PMTU value is available for destination)
			x = cached PMTU value;
		else
			x = link MTU for outgoing interface;
		fragment packet to fit into x;
	so this is not "don't fragment at all".

	2292bis-01 behavior would be (I'm guessing):
		/* if you would like to honor PMTU or link MTU */
		if (cached PMTU value is available for destination)
			x = cached PMTU value;
		else
			x = link MTU for outgoing interface;
		if (x > 1280 && IPV6_USE_MIN_MTU is specified)
			x = 1280;
		fragment packet to fit into x;
	or:
		/* if you really want to enforce 1280 by setsockopt */
		if (IPV6_USE_MIN_MTU is specified)
			x = 1280;
		else if (cached PMTU value is available for destination)
			x = cached PMTU value;
		else
			x = link MTU for outgoing interface;
		fragment packet to fit into x;

	Could you describe your suggestion in more detail?

itojun

---

From: "Matt Crawford" <[email-masked]>
Date: Wed, 05 Apr 2000 09:05:34 -0500

It looks to me like the option is useful, but the description is, as
kre says, entirely wrong.  But I don't think he captured what I see
as the full effect of setting the option.

On a datagram (UDP) socket, this would force fragmentation on the
initial transmission rather than after a possible ICMPv6 "Too Big"
message comes back.

On a stream (TCP) socket, this would set the packetization quantum of
the stream and indeed avoid fragmentation.

				Matt

---

From: Kazu Yamamoto (山本和彦) <[email-masked]>
Date: Wed, 05 Apr 2000 23:36:15 +0900 (JST)

From: "Matt Crawford" <[email-masked]>
Subject: Re: Advanced sockets API draft (rfc2292bis) query 

> the stream and indeed avoid fragmentation.

Or clarify that this sockopt is for UDP/IPv6 only.

I think this option is not necessary for TCP/IPv6 if RFC 1981 Section
5.4 is implemented correctly.

--Kazu

---

From: Robert Elz <[email-masked]>
Date: Thu, 06 Apr 2000 00:47:04 +1000

Date:        Wed, 05 Apr 2000 22:37:12 +0900
    From:        [email-masked]
    Message-ID:  <[email-masked]>

  | 	I'm not sure if I understand the intent here.  Why userland
  | 	application needs to think about link MTU?

Because with UDP, there is no-ne else (the kernel is just a packet
sender).

  | 	I believe the default behavior is not just "don't fragment at all".

Hmm - perhaps.   Is that stated somewhere?   In IPv4 fragmentation happens
unless explicitly disabled, but in IPv6 it needs to be explicitly enabled.
I was kind of expecting the API to have made that adjustment as well.

  | 	Normally, without any setsockopt, the kernel behavior would be like:
[...]
  | 	so this is not "don't fragment at all".

And if I, as the application, want to get back the "packet too big"
message so I can send something smaller?   This would be rational for
a TFTP server (which is, in a sense, just a userland implementation of
a poor man's TCP).

  | 	2292bis-01 behavior would be (I'm guessing):
[...]
  | 	or:
[...]

perhaps one of those.

An alternative might be ...

	len = lengthof(packet);
	x = 1280 (or value set by this setsoockopt if it gets an int value).
	if (len > x && x > 0)
		fragment into ((len + x - 1) / x) pieces;
	send the packet (or fragments).
	if any fragment (or the entire packet if not fragmented) is
	too big for the link MTU, return E2BIG (or something).

  | 	Could you describe your suggestion in more detail?

Aside from improving the language, I'm not sure I have a specific
suggestion (aside from not setting the 1280 number in stone, and
instead allowing it to be set by the application - either of your
algorithms would work with that modification).

Perhaps all options can be achieved by making the default value for
the section 11.1 setsockopt (whose name might perhaps change) be the
cached path MTU for the destination (which will default to the link MTU
of the first hop).   That way, unless the application does something
special, it gets the behaviour you described.

If it never wants fragmentation, it can set the frag limit huge (2^32 - 1)
though anything larger than 65536 would do (nothing bigger can be fragmented
anyway).   If it wants minimum MTU type fragments (as a DNS server might)
it can set 1280.   Then 0 would return to the default behaviour.

kre

ps: I meant to say what Matt Crawford said...

---

From: [email-masked]
Date: Thu, 06 Apr 2000 00:20:55 +0900

>I was kind of expecting the API to have made that adjustment as well.

	For raw socket, RFC2292 page 13 (5th paragraph) says that it would 
	automatically fragment packet to fit into path MTU.
	I found no explicit descrptiion about UDP sockets, but I think it makes
	sense to have similar behavior with raw socket case.

	Since we have interpreted the text differently, we may need some
	clarification in some of the documents (RFC2553?).

>a TFTP server (which is, in a sense, just a userland implementation of

	RFC2348 (TFTP blocksize option)?

	The TFTP server should, in my guess, behave like this:
	0. (get request from client)
	1. query link MTU for directly attached link.  The TFTP server needs to
	   look at its interface status by itself, since there will be nobody
	   who would send ICMPv6 TOOBIG to the server (NOTE: TOOBIG is sent
	   by router only - see RFC2463 3.2).
	2. send a packet that fits into link MTU.
	3-a. if path MTU >= link MTU, the packet should reach the client okay.
	3-b. if path MTU < link MTU, path MTU discovery would occur (TOOBIG
	     reaches server from intermediate routers).  The TFTP server would
	     use to use IPV6_RECVPATHMTU (2292bis-01 11.2) to get the informed
	     path MTU.  Go back to 2 for resend, with the informed path MTU
	     (smaller than before).

itojun

---

From: Vladislav Yasevich <[email-masked]>
Date: Wed, 05 Apr 2000 13:11:52 -0400

Well, how about the following behavior:

If the user sets the IPV6_USE_MIN_MTU option with a 0 value, use the
cached or default media mtu. 

If the user sets this option with a value of 1, use the minimum 
IPv6 mtu (1280).

If the user sets this option with another value, use that value as
the mtu.

I also think that there should be a different option to completely 
disable fragmentation so that an application (ex: traceroute) can
do path mtu discovery (starting with the first hop) itself.

-vlad 
++++++++++++++++++++++++++++++++++++++++++++++++++++
Vladislav Yasevich		Tel: (603) 884-1079
Compaq Computer Corp.		Fax: (435) 514-6884
110 Spit Brook Rd ZK03-3/T07
Nashua, NH 03062
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 5 Apr 2000 22:15:25 -0700 (PDT)

> Matt?  Erik?   Is one of you currently responsible for this draft?

Yes, I pretend to be responsible.

I agree the document is a bit terse :-)
And incorrect.

First of all I expect most existing implementations automatically fragment
UDP packets that exceed the path MTU (and by path MTU I mean the outgoing
link MTU if no pmtu has been cached yet).
This is necessary to provide minimal porting for UDP applications that
send around large packets - NFS/UDP being one prime example.
To minimize the "suprise" with IPv6 I think we want to keep it that way,
even though no document I know of states this.

I take your point on being able to set a specific value for protocols
on top of UDP that need to be PMTU aware. For such applications the behavior
needs to be to get an error when the packet size exceeds the cached pmtu
value in the kernel, since the pmtu value might have been established by
some other communication hence this application might not have
seen the current pmtu with IPV6_RECVPATHMTU.

If we go this path it makes sense to have an IPV6_USEMTU option
(instead of IPV6_USE_MIN_MTU) which takes a uint32_t as an argument
(for those that want to control jumbogram sizes).
Default value of option is zero, which has to above "fragment to the
path mtu/link mtu bahevior".
The option can not be set to the values 1 through 1279.
If it is set to a non-zero value that value will be used
to determine the size of the fragments.
There are no other checks on the value of the option.

When packets are sent and the option has been set to a non-zero value
an attempt is made to fragment to the min of that value and the length of
the packet sent.
If this value exceeds the path MTU or link MTU an ETOOBIG error may be returned
(or the packet silently dropped) and an IPV6_PATHMTU ancillary data item sent
to the application (assuming it has enabled IPV6_RECVPATHMTU). [Note that some
implementations, in particular those based on STREAMS, might not be able to
return an error to a sendto/sendmsg call when the error is generated by the
protocols in the stack. Hence the "may" above. In any case, ETOOBIG wouldn't
indicate the correct max size to use.] A portable application which sets
IPV6_USEMTU to a value other than 0 or 1280 must enable IPV6_RECVPATHMTU and
use those notifications to modify either the size of the packets being sent or
the value of the IPV6_USEMTU option.

One question that remains is should we specify the use of
this with TCP? 
Would it make sense for a busy DNS server (which some times fall back to
sending using TCP) to be able to tell TCP to send 1280 byte packets
even if the link MTU is larger? That would save time and resources when it
is just sending back e.g. 2k of data.

   Erik

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 5 Apr 2000 22:18:14 -0700 (PDT)

> If the user sets this option with a value of 1, use the minimum 

Having the application just set it to 1280 in that case seems more
logical - less special values to track.

> do path mtu discovery (starting with the first hop) itself.

Good point.
Has anybody implemented this for IPv6?
Or do we need to invent something new? (IPV6_DONTFRAGMENT option?)

The IPv4 traceroutes I've seen do this depend on being able to
set the DF flag with raw sockets (when IP_HDRINCL is set).
We don't have anything similar in IPv6.

  Erik

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 5 Apr 2000 22:23:53 -0700 (PDT)

> do path mtu discovery (starting with the first hop) itself.

I just realized this can be done using the IPV6_USEMTU as I specified
it in an earlier email.

The application enables IPV6_RECVPATHMTU and sets IPV6_USEMTU to 65535
(or maxint).

This means that it will receive an IPV6_PATHMTU ancillary data item
(and maybe an ETOOBIG) as long as it it sending packets larger
than the link MTU (or larger than a cached path MTU - is
the path MTU interaction an issue here? Should the kernel
just check the link MTU and ignore the path MTU for this type of
traceroute?)

   Erik

---

From: Francis Dupont <[email-masked]>
Date: Thu, 06 Apr 2000 11:26:10 +0200

Good point.

=> fragmentation is disabled in my code on one of these three conditions:
 - forwarding (both unicast and multicast)
 - internal don't-frag flag (used by IPv6 over IPv6 as an option
   and by TCP (in order to get path MTU discovery))
 - raw output (ie. output on a SOCK_RAW socket) with the IP_HDRINCL
   option set (this is used by traceroute6).

=> my IP_HDRINCL is not standard (:-) then I am in favor of a don't fragment
socket option.

=> I agree

[email-masked]

PS: an alternative is to make draft-ietf-ipngwg-bsd-frag-01.txt alive again.

---

From: Vladislav Yasevich <[email-masked]>
Date: Thu, 06 Apr 2000 10:27:25 -0400

> I just realized this can be done using the IPV6_USEMTU as I specified

I agree.

> (and maybe an ETOOBIG) 

This is one point that I was uncertain about in handling IPV6_RECVPATHMTU.
Assuming the application uses 2 sockets (one to send and one to receive)
it would never know to adjust the size of the packets without something
like ETOOBIG on the sendto/sendmsg (btw, this is a case with traceroute6
as it's using UDP instead of RAW sockets).

So, it sounds like a sending error would be needed where possible (as you
said, its not available in streams).

>			as long as it it sending packets larger

Path MTU interaction is not really an issue as any application that does
this,  would know the mtu from IPV6_PATHMTU ancillary data and would
reset IPV6_USEMTU with that value.  (in case of traceroute, it can
process ICMP6_PACKET_TOO_BIG to get the value).

-vlad