Subject: Re: Confirm consensus to adopt draft-hinden-6man-mtu-option

From: Ole Troan <[email-masked]>
Date: Thu, 1 Aug 2019 11:00:41 +0200

Hi Nick,

[...]

> Despite intense pressure on vendors to parse EHs in the fast path since ipv6 was defined 20 years ago, there is an awful lot of hardware out there which doesn't, and there aren't compelling reasons to think this will change any time in the future due to the performance impact of handling the complexity of EHs at high speed.

This is the first hop by hop option that has some general usefulness. (Perhaps in addition to the router altert.).
You could argue that there has been a chicken and egg problem with HBH headers, and that this draft hatches the egg. ;-)

Implementation of this draft requires changes in the forwarding path, but it is a lot fewer lines of code, than the current drop packet and send a token bucket rate limited ICMP message back. And it only need to be supported on devices connected to smaller MTU links, which may be in a different part of the network than the highest performing routers. Or not.

Does it solve the PMTUD issue by itself? No. It's going to be yet one more signal to the transport layer.
Will it become successful? Well, that's part of what the experiment.
I certainly don't see any harm in trying to improve PMTUD, and this is one of the few ideas that has come up.
Do I share your gloom and doom. Well, certainly. But that applies to a lot of the other stuff we're doing too. ;-)

Best regards,
Ole

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 1 Aug 2019 12:51:41 +0100

Hi Ole,

Ole Troan wrote on 01/08/2019 10:00:
> Implementation of this draft requires changes in the forwarding path,
> but it is a lot fewer lines of code, than the current drop packet and
> send a token bucket rate limited ICMP message back.

The complexity approach is different though.  Both mechanisms compare 
the length of the incoming packet against the mtu of the egress 
interface.  The current mechanism punts to a control plane, where 
programming is typically straightforward even if resources are limited. 
This draft needs to unroll the EH chain for data plane traffic, and then 
write into it - which is an expensive proposition, regardless of the 
recommendation in rfc8200 that hbh headers should be first in the list. 
This may be workable if you're forwarding traffic using NPUs, but if 
it's ASICs, then that's a different situation.

> Does it solve the PMTUD issue by itself? No. It's going to be yet one
> more signal to the transport layer. Will it become successful? Well,
> that's part of what the experiment. I certainly don't see any harm in
> trying to improve PMTUD, and this is one of the few ideas that has
> come up. Do I share your gloom and doom. Well, certainly. But that
> applies to a lot of the other stuff we're doing too. ;-)

Yup, totally get this.  It's just very late in the day to make 
suggestions about such a basic part of packet forwarding functionality, 
particularly when it uses a mechanism that we know is fraught with problems.

Nick

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 1 Aug 2019 14:37:24 +0200 (CEST)

> I certainly don't see any harm in trying to improve PMTUD, and this is one of the few ideas that has come up.

Has any ASIC/NPU experts been consulted on the complexity/usefulness of 
the proposed approach? If they have, what was their response?

What's the odds that the proposed HBH options could be implemented with 
reasonable effort in existing/planned hardware?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Ole Troan <[email-masked]>
Date: Thu, 1 Aug 2019 15:49:06 +0200

Hi Nick,

> The complexity approach is different though.  Both mechanisms compare the length of the incoming packet against the mtu of the egress interface.  The current mechanism punts to a control plane, where programming is typically straightforward even if resources are limited. This draft needs to unroll the EH chain for data plane traffic, and then write into it - which is an expensive proposition, regardless of the recommendation in rfc8200 that hbh headers should be first in the list. This may be workable if you're forwarding traffic using NPUs, but if it's ASICs, then that's a different situation.
> 
> Yup, totally get this.  It's just very late in the day to make suggestions about such a basic part of packet forwarding functionality, particularly when it uses a mechanism that we know is fraught with problems.

Yes, I also see these points and agree with them.
It's a judgement we have to make... 

Best regards,
Ole

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 1 Aug 2019 14:47:59 +0000

> particularly when it uses a mechanism that we know is fraught with problems.

RFC1063 was an experiment that was ended prematurely because they believed
it was late in the day back then (1988 - 1990). But, history shows that it was actually
rather early, and if the experiment had been allowed to continue we may have
had an MTU-robust Internet today instead of one based on fragile ICMP messaging.

I would not want to look back on these archives 30 years from now and lament
that path MTU is still broken because we thought back in 2019 that it was too
late to do anything about it.

Fred

---

From: Philip Homburg <[email-masked]>
Date: Thu, 01 Aug 2019 17:21:12 +0200

>late to do anything about it.

In my opinion it is better to discover the PMTU using probing without
relying on the network (for protocols with session state, otherwise limit to
1280). There should be a clear message to transport protocols that the network
cannot provide PMTU information. And the network cannot support fragmented 
packets either.

However, even if it is unlikely that this draft will actually solve the 
PMTU problem, it is worth having it as a stable protocol description.

---

From: Michael Richardson <[email-masked]>
Date: Thu, 01 Aug 2019 12:22:09 -0400

> And it only need to
    > be supported on devices connected to smaller MTU links, which may be in
    > a different part of the network than the highest performing routers. Or
    > not.

This is a really important observation, that I wanted to +1.
In the PPPoE BNG that I've worked on, it would not be a problem to do this.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Tom Herbert <[email-masked]>
Date: Thu, 1 Aug 2019 14:21:51 -0700

On Wed, Jul 31, 2019 at 3:34 PM Nick Hilliard <[email-masked]> wrote:
>
> Ole Troan wrote on 25/07/2019 15:17:
> > If you _object to adopting this document, please respond to the mailing list.
> > Editoril suggestions can be sent to the authors.
>
> the usefulness of this approach to pmtu detection is based on the
> assertion in section 6.1:
>
> >    o  The design of this Hop-by-Hop Option makes it feasible to be
> >       implemented within the fast path of a router, because the required
> >       processing is simple.
>
> The problem is the word "feasible", which is not a word that means
> anything in this context.  For an IETF contributor, it means that the
> practical implications of this EH can be put to one side while the rest
> of the draft can be discussed; for a silicon designer or asic
> programmer, it means the specification can be ignored due to the cost /
> performance constraints they operate within.
>
> Despite intense pressure on vendors to parse EHs in the fast path since
> ipv6 was defined 20 years ago, there is an awful lot of hardware out
> there which doesn't, and there aren't compelling reasons to think this
> will change any time in the future due to the performance impact of
> handling the complexity of EHs at high speed.
>
Nick,

I believe a few things have changed over the past twenty years that
are pertinent:

- IPv6 has finally achieved significant deployment and maturity which
allows us to start looking at how to leverage "advanced features" like
EH that aren't in IPv4.
- The long deployment path of IPv6 has taught us lessons about
deploying new protocol at scale. We are cognizant of the fact that EH
will not be able to traverse all corners of the Internet, so we can
take that into account and build in appropriate fallbacks or
deployment in limited domains.
- RFC8200 relaxed the requirement that all intermediate nodes need to
process HBH options. This was critical to making HBH options feasible.
So if a device, say a core router, is only interested in forwarding
performance all it needs to do is ignore the HBH EH. This declares the
minimal "fix" to make HBH deployable.
- There are other developments to make EH feasible. For instance,
RFC8504 allows receivers to apply reasonable limits on extension
header processing in order to thwart some DOS attacks.
- EH is the extensibility mechanism of IPv6. Given the perception that
EH is not yet deployable, there have been a number of alternative
attempts to make an extensible signaling protocol in like SPUD,
intermediate devices parsing and changing TCP header & options,
various methods of putting signals into UDP payload for
encapsulation-- all of these have issues of some sort with protocol
ossification, correctness, feasibility, etc. HBH options is the only
truly correct and extensible method for host to network signaling in
IP protocols.
-  PMTU HBH is not the only use case of EH that is being proposed,
there are actually several and some of which are being proposed by the
router vendors themselves (e.g. IOAM, APN6, SRv6, SRv6+). Segement
routing is particularly interesting given the amount of investment and
effort being exerted make it happen-- I believe that we will see
significant deployment of that. SRv6 is at least an order of magnitude
more complex than something like the PMTUD option, so if SR is
feasible then PMTUD (IOAM and the rest) are feasible.
- There is a strong trend in the industry towards programmble devices
with P4 (routers), eBPF (hosts, SmartNICs), etc. This faciliates fast
development and bug fix turn around compared to fixed function ASIC
solutions.
- The complexity of the "fast path" has grown by several orders of
magnitudes. Routers, especially those on the edge, commonly perform
DPI, connection tracking, ACLs, etc. For the complexity to process EH
is incremental to existing complexity, and in fact may reduce
complexity in long (e.g. EH may obviate need for DPI in some cases).

Tom
> Unfortunately, this pushes this draft into the category of aspirational
> rather than providing a workable approach to handling ipv6 pmtu
> detection.  I wish this weren't the case.
>
> As a related observation, rfc7872 suggests that HBH headers are
> associated with high rates of packet drops.
>
> Sorry to take such a pessimistic view of the draft but I don't think
> it's workable and consequently don't think it's worth pushing towards
> rfc status, even experimental.
>
> Nick
>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 2 Aug 2019 01:23:26 +0100

Tom,

just to follow up on a couple of your points:

> feasible then PMTUD (IOAM and the rest) are feasible.

I agree that if SRv6 takes off, it could possibly push better support 
for EHs in ipv6.

> complexity in long (e.g. EH may obviate need for DPI in some cases).

IPv6 was defined in an era when software forwarding was mostly handled 
on generic CPUs, i.e. there was plenty of scope even at the time for 
writing code to perform arbitrary packet mangling operations.  We may 
have more complexity now, but it would have been arguably easier to 
program that complexity in forwarding engines of the 1990s because they 
were generally CPU based rather than today's mix of CPU forwarding, NPU 
based routers, programmable ASICs and simplistic ASICs.

The issue here is that everything has a cost - nothing is free. The 
costs associated with EHs are what have caused them not to be supported 
properly over the last 20 years. Even the requirement to support v6 
frags wasn't enough to overcome the historical reticence to implement 
proper EH support in many cases.  Looking at this from a pseudo-economic 
model, the cost drivers haven't changed much, so it's hard to see how 
the outcome is going to be much different in years to come.

Nick

---

From: Tom Herbert <[email-masked]>
Date: Thu, 1 Aug 2019 19:11:00 -0700

Nick,

>From a macro viewpoint of the Internet that is true to some degree,
but unlike twenty years ago we now have huge providers and data center
operators that have significant leverage with vendors to support
features they find useful, and they are starting to run IPv6. These
are where the first serious deployment of EH will occur. Providers
like this don't necessarily need ubiquitous support for EH across all
possible vendors, just the ones they deploy. So assuming such
deployment happens then it becomes a competitive disadvantage not
support EH so the economic motivations are resolved. I do believe that
both segment routing and IOAM will be sufficiently compelling to
operators and providers so that it's pretty likely this will play out.

Making EH ubiquitous over the Internet is still relevant and a little
more challenging. Since there's no controlling entity, we have to deal
with incremental adoption and do things like "happy eyeballs" for EH.
In reality though, there's nothing special about EH in this regard,
there's a lot of protocols we might want to use over the Internet that
might not work consistently (e.g. SCTP, DCCP, but even QUIC and IPv6
still don't have ubiquitous support).

Tom

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 2 Aug 2019 15:43:55 +0100

Tom Herbert wrote on 02/08/2019 03:11:
> features they find useful, and they are starting to run IPv6. 
We certainly do and there's a rich history of scaled-up deployment 
scenarios like this pushing certain technologies forward, e.g vxlan. 
SRv6 is one approach to SR, so industry pressure to push this sort of 
functionality out will depend on how the market approaches SR in 
general.  That said, there are plenty of interesting protocols which 
never got widescale market traction, despite them solving interesting 
and difficult problems, e.g. TRILL, LISP, etc.

All we can categorically state at this point is that past EH support has 
been fraught, despite core ipv6 functionality depending on them (e.g. 
frags, ipsec).  Any hopes for improved EH support in future are 
unfortunately that - aspirations and hopes.  I just question the wisdom 
of lumping more important functionality into a protocol mechanism with 
such serious historical deployment problems and no guarantee of better 
support in future.

Nick

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 2 Aug 2019 15:12:42 +0000

Hi Nick,

> such serious historical deployment problems and no guarantee of better

I understand your point, but do you see a better way of implementing MTU
determination that does not rely on ICMP messaging or large packet path
probing? If not, then we need to confront the unfortunate truth that hard
decisions will have to be made.

We should first start by agreeing that accommodating MTU diversity in the
Internet is an important goal. So that, 30 years from now we can reflect back
on these discussions and say that we were the generation that made hard
decisions to improve the Internet experience for our children and
grandchildren.

Fred

> -----Original Message-----
> From: ipv6 [mailto:[email-masked]] On Behalf Of Nick Hilliard
> Sent: Friday, August 02, 2019 7:44 AM
> To: Tom Herbert <[email-masked]>
> Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
> Subject: Re: Confirm consensus to adopt draft-hinden-6man-mtu-option
>

---

From: Philip Homburg <[email-masked]>
Date: Fri, 02 Aug 2019 17:26:06 +0200

>Internet is an important goal. 

I assume that you mean MTUs larger than 1500.

If the 'ethernet' MTU remains effectively limited to 1500, then I don't see
the point of going beyond 1280. Too much effort for too little gain.

But the key thing is that if we want to have larger MTUs, we have to make sure
that ideally only the involved end systems need to be aware. Less ideal, if
all routers on the path need to do something special.

Where it goes wrong is that basically all systems on the internet need to do
the right thing. And that's just not not going to work.

---

From: Tom Herbert <[email-masked]>
Date: Fri, 2 Aug 2019 09:06:29 -0700

> the point of going beyond 1280. Too much effort for too little gain.
Philip,

Some larger operators, e.g. I believe Google is one of them, are
moving to larger than 1500 byte MTUs.

> the right thing. And that's just not not going to work.

The only "right thing" we need routers on the Internet to do is too
ignore HBH EH instead of dropping packets with them or relegating
packets to some slow path. The PMTU option as well as any other HBH
option we might define are optional to support at middleboxes and
don't require specific support. We know that trying to force a
unversal support requirement is impractical.

Tom

---

From: Nick Hilliard <[email-masked]>
Date: Sat, 3 Aug 2019 23:55:31 +0100

> decisions to improve the Internet experience for our children and

to answer your second question first, yes, we all want the internet to 
work and that includes MTU diversity.  Some things are hard.  PMTU 
management is one of these things.

It's hard because unlike circuit switched networks, there's no fixed 
path on an IP network between hosts A and B with deterministic 
attributes, and the endpoint devices are largely unaware of what's going 
on in the middle of the path.  Lack of intrinsic awareness of path mtu 
means that this needs to be discovered, and from a high level practical 
point of view, there are two general approaches to handling pMTU 
detection: one is for the hosts to measure it, and the other is for the 
network to provide the information.

It is hard for hosts to measure this sort of thing because the only way 
to handle it is to implement some form of constant measurement 
mechanism.  I.e. transit data and see what is ack'd.  This approach is 
doomed to failure in the general case because it assumes bidirectional 
traffic flow with in-band flow telemetry and a flow stack which can 
handle all this.

 From a network point of view, you can either report failures back to 
the sending host from intermediate points (for example, ICMP PTB / 
fragmentation-requested-but-df-set) or else attempt some form of in-line 
packet rewriting.  That might take the form of TCP MSS rewriting or it 
might be something along the lines of what's proposed in this draft or 
in RFC1063, i.e. create a writable bit in each packet and expect each 
node to record the minimum path mtu along the way.

None of the network-aware approaches work well because to one degree or 
another they all impinge on the notion that the network should be 
stateless and that intermediate devices have line rate capability to 
report this state back to the end points.

For reasons fundamentally associated with the principals above, we never 
really got pmtu management to work properly in ipv4.  As the underlying 
characteristics and assumptions are no different for ipv6, it would be 
brave to assume that the outcome would be any different.

That said, a good starting point for a network-based pmtu detection 
process would be to get the process to depend on a transmission 
mechanism which we knew was inherently reliable.  Because 
draft-hinden-6man-mtu-option depends on something which we know to to 
have unusually poor transmission characteristics (rfc7872 suggests 50% 
packet loss for packets with hbh EHs), we're crippling the future 
prospects of this ever working.

Overall, this is an unusually difficult and subtle problem to tackle. 
If ietf wants to come up with a mechanism for handling it by 
intermediate reporting, the first step would be to start out with a 
reliable transport encapsulation.  HBH EHs do not fit this bill.

I realise that this assessment will come across as being pessimistic to 
the point of defeatist, but I'd respectfully suggest that if PMTU 
management were a tractable problem within the context of how IP traffic 
is handled, we would have solved the problem space long ago, both for 
ipv4 and ipv6.  IOW, no disrespect is intended towards the authors of 
this document: what's been presented is an interesting approach. I just 
don't think it will work in practice.

Nick

---

From: Tom Herbert <[email-masked]>
Date: Sun, 4 Aug 2019 08:10:28 -0700

> prospects of this ever working.

RFC7872 is over three years old now-- it's relevance is fading every
day. This was just a snapshot point in time and doesn't reflect what
is happening in large closed networks (i.e. limited domains). This
point was brought up in 6man WG meeting that we need up to date data.
To me this seems like something that should be continuously measured.

> reliable transport encapsulation.  HBH EHs do not fit this bill.
IETF did come up with a mechanism to allow intermediate nodes to
report information-- this is exactly what modifiable HBH options are
for. The mechanism is generic, extensible, and transport protocol
agnostic. But, if HBH options don't "fit the bill" then obvious
question to be asked is "What, exactly, would fit the bill?"

Tom

---

From: Michael Richardson <[email-masked]>
Date: Sun, 04 Aug 2019 15:33:37 -0400

> traffic flow with in-band flow telemetry and a flow stack which can

Yes, but I think that we need to do exactly this.
     draft-hopps-ipsecme-iptfs-01

would benefit greatly from this, and ought to be able to do that measurement regularly.

    > prospects of this ever working.

It doesn't have to work everywhere, it just has to work at the places where
the MTU becomes smaller.   If the minimum MTU is 1280, then systems (100G+ )
which want to do bigger MTUs all the way into and between the data centers,
then the "downshift" systems need to provide this HBH processing.
Does it seems reasonable to put the onus here?  With IPv4, the minimum MTU
was significantly lower than ethernet (1/3), and on slow links (19.2 SLIP
links..) using smaller MTUs was initially a good idea.

    > Overall, this is an unusually difficult and subtle problem to
    > tackle. If ietf wants to come up with a mechanism for handling it by

What would this look like?

---

From: Mark Smith <[email-masked]>
Date: Mon, 5 Aug 2019 06:23:59 +1000

> traffic flow with in-band flow telemetry and a flow stack which can

TCP is doing this sort of measurement and recovery all of the time. It
dynamically adapts to the changing characteristics of the network, be
it packet loss or changing latency between the end-points.

The Internet can be imagined to be a black box, with unreliable fault
and status (ICMP) indicator lights on the side. Packets sent into the
black box by a host may not come out. The indicator lights may not
flash if there is a fault.

Hosts have to assume packet loss can occur, and need to recover from
it themselves if desired. Possibly unreliable ICMP messages from the
network are a hint that can improve recovery and other operations.
They cannot be relied upon.

Packet loss due to path MTU changes is just another cause in the set
of possible causes of packet loss that a transport layer needs to
recover from, should it want to.

Detection and recovery of loss due to MTU changes has been specified
in RFC 4821, "Packetization Layer Path MTU Discovery".

It is available in Linux, although not enabled by default.

Windows has had RFC 4821 since Vista, and it has been enabled by
default since then. There used to be a Microsoft Knowledge Base
article about it, however they seem to have deleted it. The URL was
https://support.microsoft.com/en-us/kb/925280, from which I got from
https://lists.opensuse.org/opensuse-bugs/2016-03/msg02376.html, a Suse
request to enable it by default under Linux.

<snip>
> report this state back to the end points.

<snip>

> mechanism which we knew was inherently reliable.

Stateless packet networks will never be inherently reliable.

Stateful or circuit switched packet networks mostly weren't either.
You have to go back as far as X.25 to see both link-layer error
detection and recovery through retransmission.

Even with "reliable networks", it is still better for hosts to be
performing their own error checking and recovery, because the
"reliable network" will not back it's claimed 100% reliability with
any form of compensation if proves not to be reliable. A host should
know whether the "reliable network" is being reliable by testing and
measuring it.

<snip>

> don't think it will work in practice.

The trap to avoid is thinking that because a mechanism won't be
perfect, it is therefore better to do nothing. Useful improvement is
the goal. Nothing is ever perfect, there are always trade-offs being
made.

---

From: "Manfredi (US), Albert E" <[email-masked]>
Date: Sun, 4 Aug 2019 21:11:08 +0000

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Mark Smith

TCP for packet and window size, and RTP/RTCP (and other streaming protocols) for update rate and encoder settings, do this. Perhaps interestingly, they accomplish this measurement and adjustment feat without involving mechanisms in routers along the path. I have to believe that it is because they employ this end to end approach, that these protocols have been solving the problem for decades, already. Maybe this is the right way to go about it, in packet switched networks, in which the path can change at a moment's notice, and where path symmetry is never guaranteed. Rather than attempting to introduce circuit-switched paradigms.

Bert

---

From: Mark Smith <[email-masked]>
Date: Mon, 5 Aug 2019 16:46:13 +1000

> TCP for packet and window size, and RTP/RTCP (and other streaming protocols) for update rate and encoder settings, do this. Perhaps interestingly, they accomplish this measurement and adjustment fe…

It might seem that the packet switched network model is more recent
than circuit switching, and designed to overcome circuit switching
limitations usually caused by state.

I think the packet switching model is inspired by something much older
than that - the physical package and letter postal system that
apparently goes back to 2400 BC.

I think evidence that suggests this is the use of the term "letter" to
describe packets in early pre-RFC documents - Internet Engineering
Notes or IENs.

For example, IEN 5, "TCP Version 2 Specification", March 1977,
(https://www.rfc-editor.org/ien/ien5.pdf):

"1. Introduction"
...
"Processes exchange finite length LETTERS as a way of communicating;"

So I think when ever we have a packet switching network problem, it
can be worth seeing if and how it has been handled in the traditional
physical mail postal system.

For example, when the postal system wants to add information to an
existing letter, it doesn't open up the letter's envelope and insert
the postal information (which is actually against the law in many if
not all countries).

Instead, the postal system encapsulates the original letter in a new
envelope, adding its own new source and destination and other
information. The original letter's integrity is preserved, obeying the
law, and the because the new envelop has a different destination to
the inner envelope, it is far less likely that the postal system's
added information will not be removed, and accidentally sent to the
inner envelope's recipient.

If  letter sender doesn't want to trust just the law to preserve their
letter's integrity while being transported in the postal service, they
can also encrypt the letter's contents. We do the same thing with
packets to enforce packet integrity if we don't trust the network to
be law (protocol) abiding regarding packet handling.

Regards,
Mark.

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Mon, 5 Aug 2019 14:47:02 +0000

Hi Nick,

> From: Nick Hilliard [mailto:[email-masked]]
> Sent: Saturday, August 03, 2019 3:56 PM
> To: Templin (US), Fred L <[email-masked]>
> Cc: Tom Herbert <[email-masked]>; 6man WG <[email-masked]>; 6man Chairs <[email-masked]>

About being intractable, a bit more history is that the RFC1063 experiment got
underway at roughly the same time that 10Mbps Ethernet was the standard
and FDDI and ATM were just coming online. The proponents of those link
technologies believed at the time that achieving higher data rates would only
be possible by increasing the MTU (~4500 for FDDI and 9180 for ATM). That
means that they needed some form of robust path MTU discovery.

But, the RFC1063 experiment was not terminated due to failures or flawed
concepts; it was derailed by market forces that wanted to sell FDDI and ATM
gear *now* and not wait for some academic experiment to conclude. Thus
was borne RFC1191 which prematurely ended RFC1063 thanks to a push
from industry.

We all know how the timeline evolved from there. With the advent of 100Mbps
Ethernet, there was proof that higher data rates could be achieved without
increasing the frame size and FDDI and ATM largely fell by the wayside. We
have enjoyed ever greater increases in Ethernet data rates since then even at
1500 MTUs, with the promise of even greater using the same technologies if we
can somehow unlock the potential of 9K.

I think I see you agreeing in your note Nick that it would be great if the network
could somehow tag packets with the minimum MTU value, but your hesitation
is the experience we have had with the proposed mechanism so far. But, I see
it as a timely meeting of opportunity and need, and history has shown that the
Internet robustly responds to such challenges.

Today, thanks to the ingenuity of the community, we have IP fragmentation
and reassembly at line rates, TCP segmentation offload, congestion control,
hardware checksums and countless other innovations that are now considered
integral elements of the Internet fabric. So, I don't see why we couldn't
reasonably expect innovation to unfold in this case also.

Thanks - Fred

---

From: Philip Homburg <[email-masked]>
Date: Mon, 05 Aug 2019 17:03:36 +0200

> The only "right thing" we need routers on the Internet to do is
> too ignore HBH EH instead of dropping packets with them or relegating
> packets to some slow path. 

First, there is an obvious deployment problem. It is not attractive for
hosts to send HBH EHs if they get dropped on a large percentage of the 
paths. In the short run this will seriously complicate host processing.
In addition to ICMPs, there will be dropped EHs, ignored EHs, and remote 
hosts that don't know what to do.

At the same time, it is very costly to build fast path processing for HBH EHs
(assuming there is none at the moment) only to support some future internet
where everybody will support this option.

And then we can safely assume that some percentage of paths, so 5% or 10%
will continue to do the wrong thing. There is no way you can get this in
every last firewall.

That said, if we find a way to define this for PMTUs >> 1500, then it is
easier to say it will only work if all devices on the path support it.

Note that there is also
https://tools.ietf.org/id/draft-van-beijnum-multi-mtu-04.txt
to deal with the lack of support for >1500 by IEEE.

---

From: Fernando Gont <[email-masked]>
Date: Mon, 5 Aug 2019 18:12:23 +0300

On 25/7/19 17:17, Ole Troan wrote:
> At the 6MAN session at the Montreal IETF there was a positive hum to adopt the experimental <draft-hinden-6man-mtu-option>
> as a 6MAN working group document:
> 
> Title : IPv6 Minimum Path MTU Hop-by-Hop Option
> Author : R. Hinden, G. Fairhurst
> Filename : draft-hinden-6man-mtu-option-02
> Pages : 14
> Date : 2019-07-05
>  
> https://datatracker.ietf.org/doc/draft-hinden-6man-mtu-option/ 
> 
> This is a call to confirm this consensus.
> This confirmation call will end on 8 August 2019.

I provided my review of this document here:
https://mailarchive.ietf.org/arch/msg/ipv6/_Jqj5UHzqvTZZ6E5L_DUg4dE-hE

As noted in my review, I believe it is quite unlikely that this will
provide an improvement over ICMP-based PMTUD (RFC1981), and I'm
concerned that based on RFC7872, it will actually break way more.

My main question is: Since PMTUD work has traditionally been done in the
transport area, has there been anything going on in the transport area
that led to this document? --- as far as I understand, the transport
area's view of PMTUD is that it should be done with RFC4821 (at least
when it comes to blackhole detection).

I don't mind the experiment, but I'm curious about the motivation, so to
speak.

Thanks!

Cheers,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Mon, 5 Aug 2019 17:52:12 +0000

Hi Fernando,

Even in scenarios where ICMP PTBs are being delivered the way RFC1981 intended,
in some networks it may be unacceptable that large messages are dropped in the
network in the first place. For example, all packets that come off an airplane should
be treated as potentially safety-of-flight critical and are to be delivered to the final
destination with the highest possible assurance. If there were an MTU restriction
on the path, though, the packet would be dropped and not delivered. So, even if
the ICMP PTB got back to the aircraft with high reliability, safety-of-flight critical
data could be lost with significant delay before it could be retransmitted. Or, in
the worst case if the too-large packet were the last message the aircraft ever
transmitted then the loss would be unacceptable.

The two solutions are to either restrict the aircraft to using 1280 for all link types
and  for all time, or to allow some way for the network to inform the aircraft of
the path MTU without risk of data loss. The proposed approach would do the
latter, so IMHO it is worth investigating.

Fred

> From: ipv6 [mailto:[email-masked]] On Behalf Of Fernando Gont
> Sent: Monday, August 05, 2019 8:12 AM
> To: Ole Troan <[email-masked]>; 6man WG <[email-masked]>
> Cc: [email-masked]; 6man Chairs <[email-masked]>

---

From: Fred Baker <[email-masked]>
Date: Mon, 5 Aug 2019 11:39:30 -0700

> On Aug 5, 2019, at 10:52 AM, Templin (US), Fred L <[email-masked]> wrote:
> 
> the path MTU without risk of data loss.

If you want a zero-risk approach, I'll note that you're on a best-effort network. "Best effort" and "zero risk" are at odds, completely. If nothing else, the reason we have CRCs on messages is because they might not even arrive, for reasons entirely unrelated to the protocol stacks.

That would lead you to an approach such as RFC 4821, not the imposition of a reliability requirement on a network that is by definition unreliable.

---

From: Fernando Gont <[email-masked]>
Date: Mon, 5 Aug 2019 22:22:50 +0300

On 5/8/19 20:52, Templin (US), Fred L wrote:
> network in the first place. 

Well, you are using a best-effort network, right?

> For example, all packets that come off an airplane should

The signal itself can be lost. So if the delay involved in an ICMP PTB
getting back is unacceptable, I'd assume that the signal packet being
dropped is also unacceptable?

> The proposed approach would do the latter, so IMHO it is worth investigating.

FWIW, I'm mostly trying to understand the underlying motivation rather
than opposing to the work or xperiment.

Thanks,

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Mon, 5 Aug 2019 19:27:26 +0000

Hi Fred,

> From: Fred Baker [mailto:[email-masked]]
> Sent: Monday, August 05, 2019 11:40 AM
> Cc: Fernando Gont <[email-masked]>; Ole Troan <[email-masked]>; 6man WG <[email-masked]>; tsv-
> [email-masked]; 6man Chairs <[email-masked]>
> If you want a zero-risk approach, I'll note that you're on a best-effort network. "Best effort" and "zero risk" are at odds, completely. If
> nothing else, the reason we have CRCs on messages is because they might not even arrive, for reasons entirely unrelated to the
> protocol stacks.

ICAO has a metric of "Required Communications Performance (RCP)" which asks for
the network to deliver N 9's of reliability (N dependent on the severity of failure for
a particular communications service type). So, they are in reality asking for a
"better-than-best-effort" service. 

> That would lead you to an approach such as RFC 4821, not the imposition of a reliability requirement on a network that is by definition

Packet replication for redundant transmission over multiple links in hopes that at least
one copy of the packet will arrive is a technique ICAO is entertaining to meet their RCP
goals, yes. But, that would be different than RFC4821.

Bottom line is that packets coming off an airplane are precious commodities. They
are expensive in terms of available network capacity and cost, and they may carry
one-time-only safety-of-flight critical information.

Fred

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Mon, 5 Aug 2019 19:35:08 +0000

Hi Fernando,

> From: Fernando Gont [mailto:[email-masked]]
> Sent: Monday, August 05, 2019 12:23 PM
> To: Templin (US), Fred L <[email-masked]>; Ole Troan <[email-masked]>; 6man WG <[email-masked]>

For aviation, we have a goal of making it "better-than-best-effort". Dropping
good data due to a link size restriction seems worse than best effort.

> dropped is also unacceptable?

ICAO believes the network can be made to do better than best effort. For sure,
not all packets are created equal - those that come off airplane links that may be
as low-end as 32Kbps may need to be handled more carefully than those that
come from 100Gbps Ethernet.
 
> than opposing to the work or xperiment.

OK. This is what I see in the aviation domain, which is perhaps only one aspect
of the motivation, but IMHO an important one.

Thanks - Fred

---

From: David Farmer <[email-masked]>
Date: Mon, 5 Aug 2019 15:06:17 -0500

I support the adoption of this draft and conducting the experiment it
describes.

However, given the limited scope of the draft to within Data Centers and
inter-Data Center Link, isn't it possible to ensure proper implementation
of the current PMTU mechanisms and Packet-to-Big ICMP messages within the
defined limited scope? If this is the case, what are the expected
advantages of this approach over the current PMTU mechanisms within the
limited domain specified?

Furthermore, most if not all of the R&E world implement what we call 9K
clean infrastructure. That is, we specify network infrastructure capable of
using 9K or larger MTU, and we ensure the proper functioning of
Packet-to-Big ICMP messages.  Within the R&E world, 9K paths are quite
common at least for the backbone and interdomain infrastructure.  In my
opinion, PMTU isn't the biggest limiting factor for the use of large MTU
paths in our community. In my opinion, what limits the use of large MTU
paths within the R&E domain is that edge networks (LANs) do not operate
with mix MTU, and when a smaller MTU host is accidentally introduced into a
large MTU LAN there are silent failures. This means we can only implement
9K MTU on the edge networks when we know for sure all devices on a network
are capable of and properly configured for 9K MTU. This requirement has
proved to be very difficult to ensure in practice, especially over extended
periods of time.

So while I believe additional work on the PMTU problem is needed, I believe
as much or more work is needed on best practices for large MTU LANs and the
use of the RA MTU Option, even better would be per host MTUs on the LAN to
support a mixed MTU LAN. But even if we don't go as far as mixed MTU LANs,
other issues that need to be explored are, do hosts support and honor the
RA MTU option, what do host do if the MTU exceeds their interface MTU
capability, etc..? This should probably a different draft, but if there is
to be effective end-to-end use of larger MTU paths we can not ignore the
significant issue on the first and last hop LANs.

Thanks.

On Thu, Jul 25, 2019 at 9:17 AM Ole Troan <[email-masked]> wrote:

> At the 6MAN session at the Montreal IETF there was a positive hum to adopt
> the experimental <draft-hinden-6man-mtu-option>
> Best Regards,
> Ole, 6man co-chair

-- 
===============================================
David Farmer               Email:[email-masked]
Networking & Telecommunication Services
Office of Information Technology
University of Minnesota
2218 University Ave SE        Phone: 612-626-0815
Minneapolis, MN 55414-3029   Cell: 612-812-9952
===============================================

---

From: Bob Hinden <[email-masked]>
Date: Mon, 5 Aug 2019 13:34:57 -0700

David,

> On Aug 5, 2019, at 1:06 PM, David Farmer <[email-masked]> wrote:
> 
> I support the adoption of this draft and conducting the experiment it describes.

Thanks!

A few comments from the discussion.

The goal of the authors is to conduct experiments to learn what is the best thing to do, what works, and what does not work.   We understand it’s a hard problem from multiple levels.   The draft tries to make this clear.

This is a collaboration between the Internet and Transport areas, neither of the authors think there is a perfect solution, or that it can be done at only one layer.   This needs to be part of a set of transport mechanisms for sure.  The more information the transport layer can get from the Internet layer the better.

Our short term goal of having it adopted by 6man is to make it easier to get an early IANA allocation to facilitate the experiments.   Using the current experimental code point is fine in the lab and/or hackathon, but not so good if we want to run it on the Internet.   If the draft is adopted, we will request an early allocation.

I hope something useful will come out of this work.   The current state of PMTUD is poor, I think it is worth trying to do something better.   It will be hard, but most things worth doing are hard.

Bob

> 
> However, given the limited scope of the draft to within Data Centers and inter-Data Center Link, isn't it possible to ensure proper implementation of the current PMTU mechanisms and Packet-to-Big ICMP messages within the defined limited scope? If this is the case, what are the expected advantages of this approach over the current PMTU mechanisms within the limited domain specified?
> 
> Furthermore, most if not all of the R&E world implement what we call 9K clean infrastructure. That is, we specify network infrastructure capable of using 9K or larger MTU, and we ensure the proper functioning of Packet-to-Big ICMP messages.  Within the R&E world, 9K paths are quite common at least for the backbone and interdomain infrastructure.  In my opinion, PMTU isn't the biggest limiting factor for the use of large MTU paths in our community. In my opinion, what limits the use of large MTU paths within the R&E domain is that edge networks (LANs) do not operate with mix MTU, and when a smaller MTU host is accidentally introduced into a large MTU LAN there are silent failures. This means we can only implement 9K MTU on the edge networks when we know for sure all devices on a network are capable of and properly configured for 9K MTU. This requirement has proved to be very difficult to ensure in practice, especially over extended periods of time.
> 
> So while I believe additional work on the PMTU problem is needed, I believe as much or more work is needed on best practices for large MTU LANs and the use of the RA MTU Option, even better would be per host MTUs on the LAN to support a mixed MTU LAN. But even if we don't go as far as mixed MTU LANs, other issues that need to be explored are, do hosts support and honor the RA MTU option, what do host do if the MTU exceeds their interface MTU capability, etc..? This should probably a different draft, but if there is to be effective end-to-end use of larger MTU paths we can not ignore the significant issue on the first and last hop LANs.
> 
> Thanks.
> 
> 
>

---

From: Fernando Gont <[email-masked]>
Date: Mon, 5 Aug 2019 22:48:33 +0300

> good data due to a link size restriction seems worse than best effort.

That would be my definition of "best effort" when you are trying to use
an MTU larger than the minimum MTU of such network.

In which case I'd argue that you should deploy MTUs of the required size
where needed, such that you can "safely" use those packets sizes.

In any case, reliability on top of an unreliable network does come at a
price.

> of the motivation, but IMHO an important one.

If such networks need large MTUs, why don't they use a requirement along
those lines? (require a minimum MTU larger than that required in the v6
specs, and make sure nodes employ MTUs that are no larger than such
minimum one?).

It would seem to me that the only environment in which this technique
might ward is in a controlled domain. And if you control the domain, why
not simply raise the MTU in the first place, and use that minimum MTU?

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Mon, 5 Aug 2019 21:11:58 +0000

Hi Fernando,

With aviation data links, one size does not fit all. There is a wide variety of links,
including cellular, satcom, terrestrial, air-to-air, etc. And, that is just in today's
current practice with who knows what else may come in the future.

So, it is not a matter of identifying one size that works well for all - it is a goal of
accommodating MTU diversity (just as we should do for the Internet itself).

Thanks - Fred

> Sent: Monday, August 05, 2019 12:49 PM

---

From: David Farmer <[email-masked]>
Date: Mon, 5 Aug 2019 17:04:39 -0500

On Mon, Aug 5, 2019 at 3:35 PM Bob Hinden <[email-masked]> wrote:

> The goal of the authors is to conduct experiments to learn what is the
> best thing to do, what works, and what does not work.   We understand it’s
> a hard problem from multiple levels.   The draft tries to make this clear.
>
> This is a collaboration between the Internet and Transport areas, neither
> of the authors think there is a perfect solution, or that it can be done at
> only one layer.   This needs to be part of a set of transport mechanisms
> for sure.  The more information the transport layer can get from the
> Internet layer the better.
>
> Our short term goal of having it adopted by 6man is to make it easier to
> get an early IANA allocation to facilitate the experiments.   Using the
> current experimental code point is fine in the lab and/or hackathon, but
> not so good if we want to run it on the Internet.   If the draft is
> adopted, we will request an early allocation.
>
> I hope something useful will come out of this work.   The current state of
> PMTUD is poor, I think it is worth trying to do something better.   It will
> be hard, but most things worth doing are hard.
>

I agree that the general state of PMTUD on the Internet is poor, but there
are limited domains where the state of PMTUD as currently specified is
mostly functional, the R&E world is one of those domains, and with the
scope defined for this draft it is quite possible to implement such a
domain with functional PMTUD as currently specified. So, part of the
experiment needs to compare this new solution to the current solution in
limited domains where the current solution is functional, especially if
this solution is to ever go beyond the experimental stage.

Furthermore, the end-to-end use of larger MTUs is an end-to-end problem and
focusing on PMTUD to the exclusion of the first and last hop LAN problems
is doomed to failure. The point is even if all routers between two hosts
implement this HBH EH perfectly if large MTUs (>1500) on the LANs at each
end are not specified and interoperable how is this going to work? I'm not
aware of BCPs for Large MTU LANs and I don't know of IPv6 test suites that
test the interoperability of hosts on Large MTU LANs.

So if you want to do 9k MTUs between Data Centers we need a definition for
9K MTU on LANs between hosts that is consistently interoperable, which I
don't think exists.

Thanks.

> > So while I believe additional work on the PMTU problem is needed, I
> believe as much or more work is needed on best practices for large MTU LANs
> and the use of the RA MTU Option, even better would be per host MTUs on the
> LAN to support a mixed MTU LAN. But even if we don't go as far as mixed MTU
> LANs, other issues that need to be explored are, do hosts support and honor
> the RA MTU option, what do host do if the MTU exceeds their interface MTU
> > At the 6MAN session at the Montreal IETF there was a positive hum to
> adopt the experimental <draft-hinden-6man-mtu-option>

---

From: Fernando Gont <[email-masked]>
Date: Tue, 6 Aug 2019 00:22:01 +0300

> accommodating MTU diversity (just as we should do for the Internet itself).

Well... the possible extra delay that you get when an oversized packet
gets dropped is the price you pay for it.

If you want to accommodate a wide range of technologies, you have to
settle for the lowest common denominator -- which includes the
possibility that your packets get dropped, and you do get delays, even
non-deterministically.

---

From: Michael Richardson <[email-masked]>
Date: Mon, 05 Aug 2019 18:53:38 -0400

Mark Smith <[email-masked]> wrote:
    > Packet loss due to path MTU changes is just another cause in the set of
    > possible causes of packet loss that a transport layer needs to recover
    > from, should it want to.

    > Detection and recovery of loss due to MTU changes has been specified in
    > RFC 4821, "Packetization Layer Path MTU Discovery".

    > It is available in Linux, although not enabled by default.

I have been turning it on everywhere I can as my IPv6 IPsec tunnel router
(giving my IPv6 everywhere) being a one-armed router, seems not to emit ICMPs!
In any case, they also get lost.

I have been trying to gather enough large scale evidence that it works to
convince distros to turn it on.  I've been told that Google and Facebook do
not even bother with PTMU; they just send at 1280, as the loss of a probe
packet costs too much in terms of transit-opportunities.

    > default since then. There used to be a Microsoft Knowledge Base article
    > about it, however they seem to have deleted it. The URL was

Ah, it is good news that this request is there, but clearly after three
years, SuSE is still not convinced either.  I wonder it Microsoft has enough evidence.

---

From: Tom Herbert <[email-masked]>
Date: Mon, 5 Aug 2019 16:56:56 -0700

> packet costs too much in terms of transit-opportunities.

Michael,

On external facing serers that's probably true, but internal to the
data center there are motivations to use greater then 1500 bytes MTU.
As I understand it, Google recently started using MTU a little over 4K
since it's much more convenient to perform send and receive operations
in units of pages (i.e. 4096 bytes). In a sprawling data center at
that scale, updating the network to instantaneously support a larger
MTU doesn't happen, so it's likely they'd need some sort of PMTU
discovery (even if it's just temporary while the network nodes are
updated). Also, since the data center is a closed and managed network
(limited domain), use of things like ICMP, fragmentation, and
extension headers are viable options for deployment.

Tom

---

From: Ole Troan <[email-masked]>
Date: Tue, 6 Aug 2019 09:10:37 +0200

David,

> I agree that the general state of PMTUD on the Internet is poor, but there are limited domains where the state of PMTUD as currently specified is mostly functional, the R&E world is one of those domains, and with the scope defined for this draft it is quite possible to implement such a domain with functional PMTUD as currently specified. So, part of the experiment needs to compare this new solution to the current solution in limited domains where the current solution is functional, especially if this solution is to ever go beyond the experimental stage.
> 
> Furthermore, the end-to-end use of larger MTUs is an end-to-end problem and focusing on PMTUD to the exclusion of the first and last hop LAN problems is doomed to failure. The point is even if all routers between two hosts implement this HBH EH perfectly if large MTUs (>1500) on the LANs at each end are not specified and interoperable how is this going to work? I'm not aware of BCPs for Large MTU LANs and I don't know of IPv6 test suites that test the interoperability of hosts on Large MTU LANs.
> 
> So if you want to do 9k MTUs between Data Centers we need a definition for 9K MTU on LANs between hosts that is consistently interoperable, which I don't think exists.

As a side-note it might be worth looking at https://tools.ietf.org/html/draft-van-beijnum-multi-mtu-05
One could imagine using the HBH MTU option in NS/NA exchanges between directly connected nodes too.
But that would still only be a hint, probing would be necessary.

Cheers,
Ole

---

From: "john leddy.net" <[email-masked]>
Date: Tue, 6 Aug 2019 13:43:17 -0400 (EDT)

Mark,

I don't understand the Postal argument you make.

If the analogy of letter and packet is correct, then the capability to add and process information about the handling of the packet by intermediate parties is a gap in the current capabilities for delivery of packets over the Internet today.  

https://en.wikipedia.org/wiki/Postal_marking

"A postal marking is any kind of annotation applied to a letter by a postal service. The most common types are postmarks and cancellations;" 
"Less common types include forwarding addresses, routing annotations, warnings, postage due notices and explanations, such as for damaged or delayed mail and censored or inspected mail. A key part of postal history is the identification of postal markings, their purpose, and period of use."

"Service marks provide information to the sender, recipient, or another post office. Advice marks notify about forwarding, missending, letters received in bad condition, letters received too late for delivery by a certain time, or the reason for a delay in mail delivery. (For example, a letter may be marked "snowbank" if snow accumulation not cleared by the potential recipient, or for whatever other reason, makes it difficult or impossible for the carrier to deliver the mail.) Dead letter offices would use various markings to keep track of their progress in finding the addressee, such as a notation that the letter had been advertised in the local newspaper. The tracking process for registered mail may entail multiple marks, notations and backstamps."

"Auxiliary marks are applied by an organization other than the postal administration. For instance, 19th century mail delivery often relied on a mix of private ships, steamboats, stagecoaches, railroads, and other transportation organizations to transport mail. Many of these organizations applied their own markings to each item, sometimes saying simply "STEAMSHIP" or some such, while others had elaborate designs. Similar routing notations were also used in the early days of airmail."

John

> On August 5, 2019 at 2:46 AM Mark Smith <[email-masked]> wrote:
> 
> 
> ....

---

From: Mark Smith <[email-masked]>
Date: Wed, 7 Aug 2019 17:26:15 +1000

> "Service marks provide information to the sender, recipient, or another post office. Advice marks notify about forwarding, missending, letters received in bad condition, letters received too late f…

None of those of those actions open up the envelope and insert new
payload (insert new sheets of paper) or modify the existing sheets of
paper.

The fixed IPv6 header + HbH EH is the equivalent of a letter's
envelope, and what follows is equivalent to letter envelope's payload.

As RFC 8200 says, other than the HbH EH,

"Extension headers (except for the Hop-by-Hop Options header) are not
   processed, inserted, or deleted by any node along a packet's delivery
   path, until the packet reaches the node (or each of the set of nodes,
   in the case of multicast) identified in the Destination Address field
   of the IPv6 header."

The packet payload is not intended to be processed by and in the
network either, which is a sender's expectation. That expectation can
be enforced by the sender authenticating/encrypting the EHs and
payload.

The increase in payload encryption we're seeing indicates senders
aren't happy that the network isn't meeting this service expectation.
Absence of encryption doesn't mean that the sender is permitting
payload or other immutable information modification.

The equivalent of those postal system messages and markings on the
envelope are either the mutable or conditionally mutable fields in the
IPv6 header of the Flow Label, Hop Limit and Traffic Class, or ICMP
messages.

> "Auxiliary marks are applied by an organization other than the postal administration. For instance, 19th century mail delivery often relied on a mix of private ships, steamboats, stagecoaches, rail…

In our model mapping, these are all lower layer encapsulations.
private ships ~= token ring, steamboats ~= ethernet, railroads ~= PPP
etc. The markings etc. here are the link-layer field values, messages
etc.

Regards,
Mark.

---

From: Ole Troan <[email-masked]>
Date: Thu, 8 Aug 2019 22:07:01 +0200

This document has now been adopted.
Can the authors please post a new revision of the document as draft-ietf-6man-mtu-option.

Best regards,
Ole, 6man co-chair

> On 25 Jul 2019, at 16:17, Ole Troan <[email-masked]> wrote:
>