Subject: [v6ops] Comments on draft-vasilenko-v6ops-ipv6-oversized-analysis

From: Jen Linkova <[email-masked]>
Date: Fri, 30 Jul 2021 15:31:14 +1000

Hi Eduard,

I'd like to elaborate a bit on my comment I made during the v6ops session today.

1) RFC8200 defines the minimum value for the reassembly buffer to 1500
bytes. and says: "An upper-layer protocol or application that depends
on IPv6
fragmentation to send packets larger than the MTU of a path should
   not send packets larger than 1500 octets unless it has assurance that
   the destination is capable of reassembling packets of that larger size."

But it doesn't define what counts as assurance.
I have reasons to believe that most OSes are capable of reassembling
much bigger packets. I tested MacOS 11, Ubuntu and Win10 - all of them
are  capable of assembling up to at least 65k bytes. So my first
question is: is the issue of "pre-fragmented packet can only be 1500
bytes" observed in real life? Like getting EMSGSIZE on packets just
slightly bigger than 1500 bytes?
(Interestingly enough, the minimum reassembly size for IPv4 MUST be
576, but also "SHOULD be either configurable or indefinite, and SHOULD
be greater than or equal to the MTU of the connected network(s)." I'm
curious why for IPv6 we just say "1500" but that discussion belongs to
6man).

2) This is a question which has been bothering me for a while, esp.
when we discuss draft-ietf-6man-mtu-option...How common are scenarios
when two ECMP have different MTUs? The only practical scenario I can
imagine is "a network has two ISP uplinks". I'd expect most networks
using overlays - especially overlays with IPv6 extension headers - to
have unified MTU settings on their links - but it looks like I have a
lack of imagination, otherwise we'd not be having this discussion now.

-- 
SY, Jen Linkova aka Furry

---

From: Vasilenko Eduard <[email-masked]>
Date: Fri, 30 Jul 2021 10:40:14 +0000

Hi Jen,

1a. It is a very small part of the draft because it has the conclusion that the host is far from stress 220B, but after some number of years, it should be revisited. Because if we would see big headers overhead from hosts directly then we should not fully rely on what vendors are doing (big reassemble buffer), but have a more consistent requirement in IPv6 specification. I am sure that IoT devices could not have 64k reassembly buffer:-)
1b. I did assume that if the problem is forwarded by the draft into the future then it does not need to investigate it deeply. I need time for more investigation.

2. My list when ECMP could change PMTU would be:
- as you said: service may cross different or many Carriers
- 3rd party rented links (still popular for non-PTT carriers in METRO/Aggregation)
- new EH is added in the middle with a different overhead. iOAM, token for QoS or Security - IETF is creasy now in defining new EH-related staff. It is a little speculation because not much implemented yet in real networks but we hope that IETF is working not for the garbage basket, right? As I said on the call: more and more stuff is programmed into the packet, not to the forwarding table.
- SR in principle could add a different label stack for TE purposes. SRH is from 72B to 216B with not much predictable distribution inside for different paths. The recent spike of MTU-related drafts is related to this - it did happen already, it is not speculation, about 50 SRv6 networks implemented globally.

I have seen ECMP: 4*8*16*8*4=2^14 just for LAG then It has additional *16 LSP planes to split fat flows (to improve TE results). It is probably the worst case. It was difficult to troubleshoot - iOAM makes sense.

Look to recent Huston statistics (research for fragmentation) how many spikes exist in different packet sizes.
https://labs.apnic.net/presentations/store/2021-05-20-v6frag.pdf
https://blog.apnic.net/2021/04/23/ipv6-fragmentation-loss-in-2021/
Probably something else exists.

Eduard
-----Original Message-----
From: Jen Linkova [mailto:[email-masked]] 
Sent: Friday, July 30, 2021 8:31 AM
To: Vasilenko Eduard <[email-masked]>; V6 Ops List <[email-masked]>
Subject: Comments on draft-vasilenko-v6ops-ipv6-oversized-analysis

Hi Eduard,

1) RFC8200 defines the minimum value for the reassembly buffer to 1500 bytes. and says: "An upper-layer protocol or application that depends on IPv6 fragmentation to send packets larger than the MTU of a path should
   not send packets larger than 1500 octets unless it has assurance that
   the destination is capable of reassembling packets of that larger size."

But it doesn't define what counts as assurance.
I have reasons to believe that most OSes are capable of reassembling much bigger packets. I tested MacOS 11, Ubuntu and Win10 - all of them are  capable of assembling up to at least 65k bytes. So my first question is: is the issue of "pre-fragmented packet can only be 1500 bytes" observed in real life? Like getting EMSGSIZE on packets just slightly bigger than 1500 bytes?
(Interestingly enough, the minimum reassembly size for IPv4 MUST be 576, but also "SHOULD be either configurable or indefinite, and SHOULD be greater than or equal to the MTU of the connected network(s)." I'm curious why for IPv6 we just say "1500" but that discussion belongs to 6man).

2) This is a question which has been bothering me for a while, esp.
when we discuss draft-ietf-6man-mtu-option...How common are scenarios when two ECMP have different MTUs? The only practical scenario I can imagine is "a network has two ISP uplinks". I'd expect most networks using overlays - especially overlays with IPv6 extension headers - to have unified MTU settings on their links - but it looks like I have a lack of imagination, otherwise we'd not be having this discussion now.

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 31 Jul 2021 09:43:11 +1200

>  new EH is added in the middle

That would be a major violation of RFC8200.

Regards
   Brian

On 30-Jul-21 22:40, Vasilenko Eduard wrote:
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
>

---

From: Vasilenko Eduard <[email-masked]>
Date: Fri, 30 Jul 2021 21:47:14 +0000

But how iOAM should operate? APN? QoS or Security tokens?
Yes, I know that it is violation of RFC 8200, but looks like nobody does pay an attention to this requirement.
Eduard.
-----Original Message-----
From: Brian E Carpenter [mailto:[email-masked]] 
Sent: Saturday, July 31, 2021 12:43 AM
To: Vasilenko Eduard <[email-masked]>; Jen Linkova <[email-masked]>; V6 Ops List <[email-masked]>
Subject: Re: [v6ops] Comments on draft-vasilenko-v6ops-ipv6-oversized-analysis

Regards
   Brian

On 30-Jul-21 22:40, Vasilenko Eduard wrote:
> Hi Jen,
> 
> 1a. It is a very small part of the draft because it has the conclusion 
> that the host is far from stress 220B, but after some number of years, it should be revisited. Because if we would see big headers overhead from hosts directly then we should not fully rely on what vendors are doing (big reassemble buffer), but have a more consistent requirement in IPv6 specification. I am sure that IoT devices could not have 64k reassembly buffer:-) 1b. I did assume that if the problem is forwarded by the draft into the future then it does not need to investigate it deeply. I need time for more investigation.
>

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 31 Jul 2021 11:19:50 +1200

> But how iOAM should operate? APN? QoS or Security tokens?

Encapsulation, as for SRV6. Which of course also affects the MTU discussion.

   Brian

---

From: Mark Smith <[email-masked]>
Date: Sat, 31 Jul 2021 09:45:46 +1000

On Sat, 31 Jul 2021 at 07:47, Vasilenko Eduard
<[email-masked]> wrote:
>
> But how iOAM should operate?

IOAM over IPv6 doesn't do that, it only uses encapsulation to add IOAM
to an existing packet.
https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-ipv6-options-05

APN? QoS or Security tokens?
> Yes, I know that it is violation of RFC 8200, but looks like nobody does pay an attention to this requirement.

I expect 6man and v6ops will continue to pay attention to this requirement.

So if your intent is to have either 6man or v6ops to eventually
publish your work as RFC, you'd best comply with it.

OTOH, taking it to another non-IPv6 WG can get the violation that you
want. That has been allowed by the IESG and is therefore now
encouraged.

---

From: Vasilenko Eduard <[email-masked]>
Date: Sat, 31 Jul 2021 19:02:56 +0000

Hi Mark,
Hi Brian,
You are right, I am wrong. I did more investigation and found:
https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-ipv6-options-06
"C6 Compliance with [RFC8200] requires OAM data to be encapsulated instead of header/option insertion directly into in-flight packets using the original IPv6 header."
I need to update draft because +40B  improve use case.
Eduard
-----Original Message-----
From: Mark Smith [mailto:[email-masked]] 
Sent: Saturday, July 31, 2021 2:46 AM
To: Vasilenko Eduard <[email-masked]>
Cc: Brian E Carpenter <[email-masked]>; Jen Linkova <[email-masked]>; V6 Ops List <[email-masked]>
Subject: Re: [v6ops] Comments on draft-vasilenko-v6ops-ipv6-oversized-analysis

> But how iOAM should operate?

IOAM over IPv6 doesn't do that, it only uses encapsulation to add IOAM to an existing packet.
https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-ipv6-options-05

So if your intent is to have either 6man or v6ops to eventually publish your work as RFC, you'd best comply with it.

OTOH, taking it to another non-IPv6 WG can get the violation that you want. That has been allowed by the IESG and is therefore now encouraged.

> To: Vasilenko Eduard <[email-masked]>; Jen Linkova 
> <[email-masked]>; V6 Ops List <[email-masked]>
> Subject: Re: [v6ops] Comments on 
> draft-vasilenko-v6ops-ipv6-oversized-analysis
>
> > 1a. It is a very small part of the draft because it has the 
> > conclusion that the host is far from stress 220B, but after some number of years, it should be revisited. Because if we would see big headers overhead from hosts directly then we should not fully rely on what vendors are doing (big reassemble buffer), but have a more consistent requirement in IPv6 specification. I am sure that IoT devices could not have 64k reassembly buffer:-) 1b. I did assume that if the problem is forwarded by the draft into the future then it does not need to investigate it deeply. I need time for more investigation.
> >