Subject: [RRG] A data point on transit MTU size

From: Brian E Carpenter <[email-masked]>
Date: Tue, 23 Sep 2008 09:49:24 +1200

We've argued here about whether it's reasonable to assume >1500 byte
MTU on transit links, when running a LISP-type solution.

Here's a data point about the real world, as far as Internet exchange
points at the southern end go:

http://list.waikato.ac.nz/pipermail/nznog/2008-September/014471.html

    Brian

--
to unsubscribe send a message to [email-masked] with the
word 'unsubscribe' in a single line as the message text body.
archive: <http://psg.com/lists/rrg/> & ftp://psg.com/pub/lists/rrg

---

From: Dino Farinacci <[email-masked]>
Date: Mon, 22 Sep 2008 15:13:18 -0700

> MTU on transit links, when running a LISP-type solution.

And if the assumption doesn't hold, then you fragment before  
encapsulate. That is specified in the main LISP spec.

Dino

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 23 Sep 2008 07:38:01 -0700

Dino, 

>-----Original Message-----
>From: Dino Farinacci [mailto:[email-masked]] 
>Sent: Monday, September 22, 2008 3:13 PM
>To: Brian E Carpenter
>Cc: RRG
>Subject: Re: [RRG] A data point on transit MTU size
>

1) You can't use IPv4 fragmentation if DF=1.
2) You can't send at high data rates if you use
   IPv4 fragmentation.

IMHO, LISP should be SEALed. I will be here and ready
to talk about it whenever you are.

Fred
[email-masked]

>> to unsubscribe send a message to [email-masked] with the word 
>> 'unsubscribe' in a single line as the message text body.

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 23 Sep 2008 08:10:32 -0700

> 1) You can't use IPv4 fragmentation if DF=1.

Well, you could.

> 2) You can't send at high data rates if you use

Yes, you can. That is implementation dependent.

> to talk about it whenever you are.

There is no reason to change our position.

Bigger fish to fry,
Dino

---

From: "Templin, Fred L" <[email-masked]>
Date: Tue, 23 Sep 2008 08:25:20 -0700

>Sent: Tuesday, September 23, 2008 8:11 AM
>To: Templin, Fred L
>Cc: Brian E Carpenter; RRG
>>>> We've argued here about whether it's reasonable to assume 
>>1500 byte 

And break RFC791?

>Yes, you can. That is implementation dependent.

But sub-optimal implementations risk data corruption, which
could be very bad.

>> IMHO, LISP should be SEALed. I will be here and ready to 
>talk about it 
>> whenever you are.
>

In the lisp-interest archives, it was shown how incorporation
of SEAL into LISP would in fact be better from an efficiency
standpoint in that the header size remains the same plus you
gain 6 additional Locator Reach Bits. That is in addition
to the advantage that the path MTU issues are put to rest.

>Bigger fish to fry,

Eh?

Fred

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 24 Sep 2008 09:32:16 +1200

Fred,

>>> 1) You can't use IPv4 fragmentation if DF=1.

Why would DF be set on the outer (LISP-generated) header?
LISP can simply forbid that by specification.

It's irrelevant whether DF is set on the inner (end-host-generated)
header. LISP is acting as a layer 2 as far as that header is concerned.
There's no violation of 791, since the packet will be reassembled
before it's decapsulated.

> But sub-optimal implementations risk data corruption, which

Yes, we agree that sub-optimal implementations are a bad idea.
We also agree, I trust, that writing perfect reassembly code
is fairly hard (although I speak from 20 year old experience).
In particular the RFC 4459 and 4963 issues have to be handled.
I don't have an opinion whether SEAL is necessary for that,
or whether LISP can just rely on basic frag.

   Brian

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 23 Sep 2008 14:52:32 -0700

> or whether LISP can just rely on basic frag.

The code for reassembly isn't hard, it's allocating the buffers to  
store the arrived fragments. That's nearly impossible in hardware.

Dino

---

From: "Tony Li" <[email-masked]>
Date: Tue, 23 Sep 2008 23:48:54 -0700

|The code for reassembly isn't hard, it's allocating the buffers to  
|store the arrived fragments. That's nearly impossible in hardware.

Patently nonsense.  Packet reassembly in hardware is very much
straightforward if one is willing to burn large buffers to do so.  Imagine
taking an implemention in C and simply converting it to Verilog.  Not at all
out of the question.

In fact, a subset of this problem has been solved for a very long time.
Remember ATM?  Remember a SAR chip?  That's reassembly of fixed size packet
fragments.

Tony

---

From: Dino Farinacci <[email-masked]>
Date: Wed, 24 Sep 2008 00:56:21 -0700

> straightforward if one is willing to burn large buffers to do so.   

The willing part is the impossible part.

> taking an implemention in C and simply converting it to Verilog.   

It certainly is.

> Remember ATM?  Remember a SAR chip?  That's reassembly of fixed size  

Having fixed size per standard makes it much simpler. Don't have that  
for IP.

Dino

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 24 Sep 2008 11:15:02 -0700

>Sent: Wednesday, September 24, 2008 12:56 AM
>To: [email-masked]
>Cc: 'Brian E Carpenter'; Templin, Fred L; 'RRG'

I'm not convinced as to how "large" the buffers need to
be? AFAICT, the size depends on the number of simultaneous
reassemblies in progress plus the degree to which the
network mis-orders packets plus the degree to which stale
reassemblies are garbage-collected. This, plus SEAL puts
a firm cap of 2KB as the largest size an ETR will ever
have to reassemble.

Think of ETR reassembly in a LISP/SEAL world as an
indication of discomfort ("pain" would be too strong
a word). The way to migrate toward greater degrees of
comfort is to incrementally identify and weed out the
marginal links. 

>> Remember ATM?  Remember a SAR chip?  That's reassembly of fixed size 
>> packet fragments.
>
>Having fixed size per standard makes it much simpler. Don't 
>have that for IP.

The most we will ever ask the ETR to reassemble is 2KB. Ever.

Fred

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Wed, 1 Oct 2008 15:35:14 +0100

On 24 sep 2008, at 19:15, Templin, Fred L wrote:

> reassemblies in progress

Right. But:

1. This could be a large number. Assume every full size packet is  
fragmented then the number of fragments held at any given time is  
pretty much number of ITRs we talk to / 2

2. It's possible to exhaust limits in this area for DoS purposes

> The most we will ever ask the ETR to reassemble is 2KB. Ever.

Obviously we want the system to be able to handle packets bigger than  
2k if hardware limitations permit, but the only reason for reassembly  
would be the case where the inner packet is 1500 bytes DF=1 so you  
would never have to reassemble more than two fragments, can require  
that those arrive in order, too. Another optimization: require that  
the first fragment is 1024 or 800 bytes or so rather than 1500, so the  
reassembly buffer only needs to be 1k per session.

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 1 Oct 2008 08:29:35 -0700

Iljitsch, 

>From: Iljitsch van Beijnum [mailto:[email-masked]] 
>Sent: Wednesday, October 01, 2008 7:35 AM
>To: Templin, Fred L
>Cc: Dino Farinacci; [email-masked]; Brian E Carpenter; RRG

OK, but please check out my other messages regarding
fragmentation/segmentation as an indication that there
are marginal links on the path in need of replacement.

We don't want segmentation/reassembly as a steady-state
condition; we only want it to bridge the gap between
what we have today and a future jumbo-clean Internet.
(Also as a hot-standby in case someone fat fingers a
link MTU configuration or inserts a 90's-era piece of
networking gear.)

>reassembly buffer only needs to be 1k per session.

So, require that the largest segment be 1KB when we have
to segment? I'm not sure that an ETR would be able to
manage its buffers efficiently in order to take advantage
of that given that it still needs to receive unfragmented
MTU-sized packets, but neither can I guarantee that that
would not be the case. So, I don't see a problem with
what you are suggesting.

Fred

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Thu, 2 Oct 2008 18:06:03 +0100

> what we have today and a future jumbo-clean Internet.

Unfortunately, things don't always work out that way. If you create  
two ways to do something, a good one and an easier one, the easier one  
will remain in use forever, pretty much.

> would not be the case. So, I don't see a problem with

The main thing is that we can restrict the way in which fragmentation  
happens so we don't have to solve the general case.

But I think the maximum number of sessions that must be supported and  
the DoS risks are the big problems.