Subject: Re: I-D Action:draft-hui-6man-rpl-routing-header-00.txt

From: Erik Nordmark <[email-masked]>
Date: Tue, 01 Jun 2010 02:02:22 -0700

On 05/31/10 09:06 AM, culler wrote:
> Since we route WITHIN a RPL network, rather than THROUGH a RPL network,
> it would seem that the case mentioned does not arise. Right?

It can still happen.
If the sender is outside of the RPL network and sends a packet, and then 
at the edge of the RPL network the routing header is added and the 
packet is forwarded. If then a downstream RPL router sees that the 
packet is to big, it will generate an ICMP "packet too big" and send it 
to the source of the packet.

    Erik

>
>
> On May 30, 2010, at 7:56 AM, Erik Nordmark wrote:
>
>>
>> The draft says:
>> A RPL Router MAY insert a Type 4 Routing header if one does not
>> already exist. The conditions for inserting a Type 4 Routing header
>> are out of scope of this document.
>>
>> Having routers insert headers in packets they forward is know to be
>> problematic in general, since it breaks path MTU discovery.
>> Taking an example of a host which sends a packet which is 1500 bytes,
>> and then receives an ICMP packet too big error saying "please limit
>> packets to 1500 bytes", will not change the host's behavior. But that
>> can happen if some router grows the packet to be more than 1500 bytes.
>>
>> Thus in general, the only safe way to insert headers in a packet is to
>> have the "inserter" put an extra IP header, with itself as a source,
>> then the inserted header, and then the original packet.
>> That extra IP header ensures that any ICMP error go back to the
>> "inserter", which can then compensate for its insertion before sending
>> the ICMP error towards the sender of the original packet.
>>
>> Thus I think there is a question for 6man whether we think the
>> constrained environment of ROLL is such that we can relax this. But it
>> does require that all the routers at the edge of the ROLL network
>> adjust the ICMP errors they forward to compensate for any inserted RH4
>> headers.
>>
>> Erik
>>
>
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: Erik Nordmark <[email-masked]>
Date: Tue, 01 Jun 2010 02:11:16 -0700

On 05/31/10 11:03 AM, Vishwas Manral wrote:
> Hi Erik,
>
> The first thing is IPv6 MTU discovery is optional and the only
> requirement is that the Minimum MTU is satisfied.

Yes, but 1280 plus a routing header will exceed 1280.
Thus the minimum MTU by itself doesn't save you.

> Also in cases like LLN the Routing paths may change so even after
> discovery the Packet too big may come anyway. Besides PMTU issues you
> mention are no different than a lot of cases such as tunnelling and
> transition mechanisms.

There is a very big difference.
Tunneling means that the entity which added the tunneling headers is in 
the source IP address field in the outer header. Thus ICMP errors will 
be sent packet to the tunnel entry point. It can then compensate for the 
added headers by sending an ICMP packet too big with a smaller MTU back 
to the original source. This how IPvx in IPvy tunneling is done. See for 
instance RFC 4213 and RFC 2473.

The big difference is that when a router does something to grow the size 
of the packet (for instance, inserting a routing header) and it does not 
add an outer IP header with its source address, then the ICMP errors 
will go back to the original sender without being adjusted for the added 
header.

> For MTU discovery in cases when padding is done, the right MTU will be
> found as the data packets will undergo the same behavior as the PMTU
> discovery message.

Huh?

The ICMP "packet too big" error contains a MTU size. There is nothing in 
the normal IP forwarding path that changes that number.

     Erik

> Thanks,
> Vishwas
>
> On Sun, May 30, 2010 at 7:56 AM, Erik Nordmark<[email-masked]>  wrote:
>>
>> The draft says:
>> Taking an example of a host which sends a packet which is 1500 bytes, and
>> then receives an ICMP packet too big error saying "please limit packets to
>> 1500 bytes", will not change the host's behavior. But that can happen if
>> some router grows the packet to be more than 1500 bytes.
>>
>> Thus in general, the only safe way to insert headers in a packet is to have
>> the "inserter" put an extra IP header, with itself as a source, then the
>> inserted header, and then the original packet.
>> That extra IP header ensures that any ICMP error go back to the "inserter",
>> which can then compensate for its insertion before sending the ICMP error
>> towards the sender of the original packet.
>>
>> Thus I think there is a question for 6man whether we think the constrained
>> environment of ROLL is such that we can relax this. But it does require that
>> all the routers at the edge of the ROLL network adjust the ICMP errors they
>> forward to compensate for any inserted RH4 headers.
>>
>>    Erik
>>

---

From: Ralph Droms <[email-masked]>
Date: Tue, 1 Jun 2010 08:16:41 -0400

I think the packet too big problem can also occur with P2P traffic, if I understand correctly that one proposed solution is to forward all P2P traffic through an edge router...

- Ralph

On Jun 1, 2010, at 5:02 AM, Erik Nordmark <[email-masked]> wrote:

> If the sender is outside of the RPL network and sends a packet, and then at the edge of the RPL network the routing header is added and the packet is forwarded. If then a downstream RPL router sees that the packet is to big, it will generate an ICMP "packet too big" and send it to the source of the packet.
> 
>   Erik
> 
>> 
>>

---

From: Vishwas Manral <[email-masked]>
Date: Tue, 1 Jun 2010 21:50:59 -0700

Hi Erik,

Sorry for the late response.

One thing I wanted to clarify was that not everytime we add data do we
change the source address. IPsec transport mode is a case in point. I
guess others have pointed out other cases. Can you explain your views
on the same?

Here is how ESP is done:

                     BEFORE APPLYING ESP
             ---------------------------------------
       IPv6  |             | ext hdrs |     |      |
             | orig IP hdr |if present| TCP | Data |
             ---------------------------------------

                      AFTER APPLYING ESP
             ---------------------------------------------------------
       IPv6  | orig |hop-by-hop,dest*,|   |dest|   |    | ESP   | ESP|
             |IP hdr|routing,fragment.|ESP|opt*|TCP|Data|Trailer| ICV|
             ---------------------------------------------------------
                                          |<--- encryption ---->|
                                      |<------ integrity ------>|

> Thus the minimum MTU by itself doesn't save you.
One thing to consider is 802.15.4 has an MTU of only 127 octets. There
is a fragmentation and reassembly layer below the IP layer to satisfy
the MTU at each IP hop (just read RFC 4944). Once that is present in
my view it is not a problem to reassemble a larger packet. 802.15.4
experts would however be better suited to answer the question.

> Tunneling means that the entity which added the tunneling headers is
> in the source IP address field in the outer header. Thus ICMP errors
> will be sent packet to the tunnel entry point. It can then compensate
> for the added headers by sending an ICMP packet too big with a
> smaller MTU back to the original source. This how IPvx in IPvy
> tunneling is done. See for instance RFC 4213 and RFC 2473.
You seem to take things literally. Can you explain the behavior in the
case of IPsec transport mode where no new IP header is added and how
it is different?

> The big difference is that when a router does something to grow the size of
> the packet (for instance, inserting a routing header) and it does not add an
> outer IP header with its source address, then the ICMP errors will go back
> to the original sender without being adjusted for the added header.

Thanks,
Vishwas

>>> have
>>> "inserter",
>>> constrained

---

From: Vishwas Manral <[email-masked]>
Date: Tue, 1 Jun 2010 22:32:37 -0700

Hi Erik,

One more thing. The ICMP to the source is one issue which has been
discussed in the earlier mail and you have rightly pointed.

I am already trying to work with Jonathan to get the solution for
this. One of the ways is when using the new header we can have the
address[0] contain the source address of the device which last
modified the packet. ICMP can then be sent to this address.

Regarding PMTUD have a look at RFC 4821 which extends RFC 1981 and you
will understand what I mean by my previous mail.

Thanks,
Vishwas

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 02 Jun 2010 01:35:15 -0700

> guess others have pointed out other cases. Can you explain your views

The IPsec transport vs. tunnel mode is somewhat decoupled from whether 
or not the packet is encapsulated with an outer IP header. It can be 
used when the packet is encapsulated, although it is more common to use 
tunnel mode in that case. But you are thinking of the non-encapsulated 
case, which can only be done on the sending system i.e. on the system 
that is in the IPv6 source address field.

>                       BEFORE APPLYING ESP

Here the orig IP hdr has a source address which is assigned to the 
system which is originating the TCP packet *and* applying ESP to the packet.
Thus the way to handle path MTU and TCP MSS calculations is a local 
matter to that stack's implementation.

In no case can be above type of application of ESP be done in a router 
along the path.

> experts would however be better suited to answer the question.

I don't see what it has to do with the details of 15.4.

What might be helpful is that the MTU for 4944 is defined to be no more 
than the IPv6 minumum. RFC 4944 says
    The MTU size for IPv6 packets over IEEE 802.15.4 is 1280 octets.

Thus assuming ROLL is never applied to anything but RFC 4944 networks 
(which may or may not be a valid assumption), then we know that once a 
ROLL router has added the RH4 (and verified that the result is less than 
1280 bytes) then there can be no subsequent issue with exceeding the MTU 
since subsequent ROLL routers will have an interface MTU of 1280 bytes. 
But this is not the case if ROLL is ever applied to networks that have 
an interface MTU larger than 1280.

Note that when a ROLL router wants to add RH4 it does have to check that 
the result wouldn't exceed 1280 bytes (the outgoing interface MTU). If 
it would, then the IPv6 packet would need to be fragmented by the ROLL 
router before RH4 is added to each fragment.

However, IPv6 routers can only fragment when they originate packets 
according to RFC 2460. And fragmentation elsewhere would be risky since 
it requires "inventing" the 32 bit fragment IDs that the original sender 
own.

Thus I'm having a hard time seeing how what you are proposing fits with 
the IP architecture. The sane way is to have the ingress ROLL router add 
an IPv6 header and RH4 to the original packet, and then fragment that if 
the result exceeds the interface or tunnel MTU. FWIW this is exactly how 
router applies IPsec to a forwarded packet.

> case of IPsec transport mode where no new IP header is added and how

See above.

>> to the original sender without being adjusted for the added header.

We can talk about it on the phone - that might be more efficient than 
these emails.

    Erik

---

From: Ralph Droms <[email-masked]>
Date: Wed, 2 Jun 2010 08:07:37 -0400

Erik - assume, for discussion, the MTU within the RPL network is  
uniform (1500 bytes), and the RPL network includes only non-storing  
routers, so all the forwarding headers are inserted by the RPL edge  
router.  Could that edge router manage the ICMP messages to the source  
correctly; e.g., if the edge router needs 64 bytes for routing  
headers, it sends the ICMP message with "please limit packets to 1436  
bytes"?

- Ralph

On Jun 1, 2010, at 5:02 AM 6/1/10, Erik Nordmark wrote:

> If the sender is outside of the RPL network and sends a packet, and  
> then at the edge of the RPL network the routing header is added and  
> the packet is forwarded. If then a downstream RPL router sees that  
> the packet is to big, it will generate an ICMP "packet too big" and  
> send it to the source of the packet.
>
>   Erik
>
>>
>>

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 02 Jun 2010 05:16:17 -0700

On 06/ 2/10 05:07 AM, Ralph Droms wrote:
> Erik - assume, for discussion, the MTU within the RPL network is uniform
> (1500 bytes), and the RPL network includes only non-storing routers, so
> all the forwarding headers are inserted by the RPL edge router.  Could
> that edge router manage the ICMP messages to the source correctly; e.g.,
> if the edge router needs 64 bytes for routing headers, it sends the ICMP
> message with "please limit packets to 1436 bytes"?

It is actually a bit worse, since RFC 4944 says that the 6lowpan 
interface MTU is 1280. Thus any router inserting things, or 
encapsulating things, would have to fragment the IPv6 packets. In the 
case of inserting RH4 without encapsulating with an extra IPv6 header, 
this means "proxy fragmentation" i.e., a router pretending to fragment 
on behalf of the original host.

I don't know if we want to be that "creative".

In the case of a uniform ROLL network with a fixed MTU sufficiently 
large where all the devices at the "edge" of that network know the 
special rules, then I think it would work to have the edge uniformly 
insert RH4 in packets going "in", remove RH4 in packets going "out", and 
redoing the ICMP errors going "out". Redoing the ICMP errors is 
presumably similar to what is described in the tunneling RFCs - not only 
the mtu field but also the "packet in error" should be redone to remove 
all traces of RH4.

    Erik

---

From: Vishwas Manral <[email-masked]>
Date: Wed, 2 Jun 2010 08:15:36 -0700

Hi Erik,

I agree (again) that the Source address and ICMP part is a problem and
that needs to be solved. We can discuss this offline like you have
mentioned (maybe over the weekend) and send the conclusions to the
group.

That said I have been trying to think of solutions to actually solve
the issue (instead of finding cases where the same issue may occur
:)), if the ROLL edge node did IP in IP tunnelling to the destination
and always fragmented packets say greater than say 1000 bytes. Would
that solve the issue?

Also I am not sure, if we can do it but if 15.4 said the IPv6 MTU is
1500 instead of 1280, wouldn't that solve the issue too?

Thanks,
Vishwas

On Wed, Jun 2, 2010 at 1:35 AM, Erik Nordmark <[email-masked]> wrote:
> The IPsec transport vs. tunnel mode is somewhat decoupled from whether or
> not the packet is encapsulated with an outer IP header. It can be used when
> the packet is encapsulated, although it is more common to use tunnel mode in
> that case. But you are thinking of the non-encapsulated case, which can only
> be done on the sending system i.e. on the system that is in the IPv6 source
> address field.
>
>> Here is how ESP is done:
>>
>>                      BEFORE APPLYING ESP
> Here the orig IP hdr has a source address which is assigned to the system
> which is originating the TCP packet *and* applying ESP to the packet.
> Thus the way to handle path MTU and TCP MSS calculations is a local matter
> to that stack's implementation.
>
> In no case can be above type of application of ESP be done in a router along
> the path.
>
>>
> What might be helpful is that the MTU for 4944 is defined to be no more than
> the IPv6 minumum. RFC 4944 says
> Thus assuming ROLL is never applied to anything but RFC 4944 networks (which
> may or may not be a valid assumption), then we know that once a ROLL router
> has added the RH4 (and verified that the result is less than 1280 bytes)
> then there can be no subsequent issue with exceeding the MTU since
> subsequent ROLL routers will have an interface MTU of 1280 bytes. But this
> is not the case if ROLL is ever applied to networks that have an interface
> MTU larger than 1280.
>
> Note that when a ROLL router wants to add RH4 it does have to check that the
> result wouldn't exceed 1280 bytes (the outgoing interface MTU). If it would,
> then the IPv6 packet would need to be fragmented by the ROLL router before
> RH4 is added to each fragment.
>
> according to RFC 2460. And fragmentation elsewhere would be risky since it
> requires "inventing" the 32 bit fragment IDs that the original sender own.
>
> Thus I'm having a hard time seeing how what you are proposing fits with the
> IP architecture. The sane way is to have the ingress ROLL router add an IPv6
> header and RH4 to the original packet, and then fragment that if the result
> exceeds the interface or tunnel MTU. FWIW this is exactly how router applies
> IPsec to a forwarded packet.
>
>>> of
> We can talk about it on the phone - that might be more efficient than these
> emails.
>
>   Erik
>

---

From: Erik Nordmark <[email-masked]>
Date: Wed, 02 Jun 2010 08:38:14 -0700

> mentioned (maybe over the weekend) and send the conclusions to the

There are two issues that have been identified in this thread:
  - MTU-related issues - ICMP packet too big and other errors
  - Need to fragment 1280 byte packets as part of adding RH4

It makes sense to think about approaches that would address both.

> and always fragmented packets say greater than say 1000 bytes. Would

Yes, if you do ROLL edge-to-edge IP in IP tunnels then you can reuse 10+ 
years of understanding of how to handle MTU, fragmentation etc.

> 1500 instead of 1280, wouldn't that solve the issue too?

No change needed in 15.4. The question is whether it would help if RFC 
4944 was revised to require offering more than 1280 bytes of MTU to IP.

I think that would help remove the fragmentation issue since a 1280 byte 
packet would not need to be fragmented any more (assuming 1280+RH4 would 
be less than the that required MTU.)

But one would still have some way to handle the ICMP error issue. 
Requiring that all ROLL edge router rewrite the ICMP errors would be one 
way.

    Erik

---

From: Vishwas Manral <[email-masked]>
Date: Wed, 2 Jun 2010 09:18:06 -0700

Hi Erik,

Great if converging on solutions was so much easier, we should have
tried that instead of why not to solve the issues. :)

On Wed, Jun 2, 2010 at 8:38 AM, Erik Nordmark >> That said I have been
trying to think of solutions to actually solve
>> If the ROLL edge node did IP in IP tunnelling to the destination
I agree. So we have some understanding here. It is edge to edge or
edge to within the domain which is the case here. Another thing is we
can have the MTU changes even with the IP in IP tunnels.

> No change needed in 15.4. The question is whether it would help if RFC 4944
> was revised to require offering more than 1280 bytes of MTU to IP.
>
> packet would not need to be fragmented any more (assuming 1280+RH4 would be
> less than the that required MTU.)
I agree. That is the easiest solution and causes no new issues as the
lower layer allows the fragmentation. I agree we still need to solve
the ICMP issues in this case.

> But one would still have some way to handle the ICMP error issue. Requiring
> that all ROLL edge router rewrite the ICMP errors would be one way.
That is correct.

Thanks,
Vishwas

---

From: JP Vasseur <[email-masked]>
Date: Thu, 3 Jun 2010 10:03:49 +0200

> It makes sense to think about approaches that would address both.

Yes and thanks to all for the fruitful discussions.

> Yes, if you do ROLL edge-to-edge IP in IP tunnels then you can reuse  
> 10+ years of understanding of how to handle MTU, fragmentation etc.
>
>
> No change needed in 15.4. The question is whether it would help if  
> RFC 4944 was revised to require offering more than 1280 bytes of MTU  
> to IP.
>
> I think that would help remove the fragmentation issue since a 1280  
> byte packet would not need to be fragmented any more (assuming  
> 1280+RH4 would be less than the that required MTU.)
>
> Requiring that all ROLL edge router rewrite the ICMP errors would be  
> one way.
>

Just a clarification, ROLL is not tied to 15.4 and could run on many  
other media with various MTU.

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 03 Jun 2010 02:10:00 -0700

> other media with various MTU.

That was what I feared.

We can use the outlined approach as long as we can require such media to 
support a MTU for IP packets that is slightly larger than 1280.
Thus we'd need a maximum size for the amount the ROLL ingress would add 
to the packet (RH4 and the HBH), and then require that the ROLL links 
support at least 1280 plus that max. Essentially a ROLL_MIN_MTU.

That would ensure that the ROLL ingress would never need to fragment a 
1280 byte packet. And for packets larger than 1280, the rewriting of the 
ICMP errors at the ROLL boundary would make path MTU discovery work.

But if we can't mandate such a ROLL_MIN_MTU, then the sensible approach 
would seem to be to do IP in IP encapsulation at the ROLL ingress.

    Erik

---

From: Jonathan Hui <[email-masked]>
Date: Thu, 3 Jun 2010 21:30:11 -0700

Hi Erik,

On Jun 3, 2010, at 2:10 AM, Erik Nordmark wrote:

> We can use the outlined approach as long as we can require such  
> media to support a MTU for IP packets that is slightly larger than  
> 1280.
> Thus we'd need a maximum size for the amount the ROLL ingress would  
> add to the packet (RH4 and the HBH), and then require that the ROLL  
> links support at least 1280 plus that max. Essentially a ROLL_MIN_MTU.
>
> That would ensure that the ROLL ingress would never need to fragment  
> a 1280 byte packet. And for packets larger than 1280, the rewriting  
> of the ICMP errors at the ROLL boundary would make path MTU  
> discovery work.
>
> But if we can't mandate such a ROLL_MIN_MTU, then the sensible  
> approach would seem to be to do IP in IP encapsulation at the ROLL  
> ingress.

Thanks for all your thoughts on this subject.  The two options are  
clear.

Of the two choices, I prefer to specify a ROLL_MIN_MTU and avoid IP in  
IP encapsulation just to support routing.  The former approach seems  
to be less onerous on resource constrained nodes that are expected  
within a RPL network.  It avoids one possible source of IP-layer  
fragmentation.  Of course, as you noted, the edge router must not  
allow any side effects of RH4 to escape the RPL network, which is  
really the intended use of RH4 anyway.

It's also important that we include the rpl-option overhead in  
ROLL_MIN_MTU, so we'll have to specify a maximum for both the rpl- 
option and rpl-routing-header (as you noted in the Anaheim meeting).

In 6LoWPAN networks, while RFC 4944 indicates a MTU of 1280 bytes, the  
fragmentation mechanism does support up to 2048 bytes so that is  
comforting.  So far, I'm not aware of any existing links intended for  
use with RPL that cannot support an MTU that is a bit larger than 1280  
bytes.

If people think specifying a ROLL_MIN_MTU is reasonable, we'll submit  
an update to the rpl-routing-header draft that incorporates these  
changes and the several other ones brought up over the past week.

--
Jonathan Hui

---

From: JP Vasseur <[email-masked]>
Date: Fri, 4 Jun 2010 07:33:11 +0200

Thanks again for the feed-back. Does the WG support this approach ?

Thanks.

JP.

On Jun 4, 2010, at 6:30 AM, Jonathan Hui wrote:

>> That would ensure that the ROLL ingress would never need to  
>> fragment a 1280 byte packet. And for packets larger than 1280, the  
>> rewriting of the ICMP errors at the ROLL boundary would make path  
>> MTU discovery work.
>>
> Of the two choices, I prefer to specify a ROLL_MIN_MTU and avoid IP  
> in IP encapsulation just to support routing.  The former approach  
> seems to be less onerous on resource constrained nodes that are  
> expected within a RPL network.  It avoids one possible source of IP- 
> layer fragmentation.  Of course, as you noted, the edge router must  
> not allow any side effects of RH4 to escape the RPL network, which  
> is really the intended use of RH4 anyway.
>
> In 6LoWPAN networks, while RFC 4944 indicates a MTU of 1280 bytes,  
> the fragmentation mechanism does support up to 2048 bytes so that is  
> comforting.  So far, I'm not aware of any existing links intended  
> for use with RPL that cannot support an MTU that is a bit larger  
> than 1280 bytes.
>
> If people think specifying a ROLL_MIN_MTU is reasonable, we'll  
> submit an update to the rpl-routing-header draft that incorporates  
> these changes and the several other ones brought up over the past  
> week.
>
> --
> Jonathan Hui
>

---

From: JP Vasseur <[email-masked]>
Date: Fri, 4 Jun 2010 07:35:35 +0200

Thanks.

JP.

---

From: Vishwas Manral <[email-masked]>
Date: Thu, 3 Jun 2010 22:40:52 -0700

Hi JP,

The approach is good, we however still need to solve the ICMP message
being sent to the source not the edge of the RPL network.

We can probably use the addr[0] approach which was discussed earlier.

Thanks,
Vishwas

On Thu, Jun 3, 2010 at 10:35 PM, JP Vasseur <[email-masked]> wrote:
>> Thanks for all your thoughts on this subject.  The two options are clear.
>>
>> Of the two choices, I prefer to specify a ROLL_MIN_MTU and avoid IP in IP
>> encapsulation just to support routing.  The former approach seems to be less
>> onerous on resource constrained nodes that are expected within a RPL
>> network.  It avoids one possible source of IP-layer fragmentation.  Of
>> course, as you noted, the edge router must not allow any side effects of RH4
>> to escape the RPL network, which is really the intended use of RH4 anyway.
>>
>> ROLL_MIN_MTU, so we'll have to specify a maximum for both the rpl-option and
>> rpl-routing-header (as you noted in the Anaheim meeting).
>>
>> fragmentation mechanism does support up to 2048 bytes so that is comforting.
>>  So far, I'm not aware of any existing links intended for use with RPL that
>> cannot support an MTU that is a bit larger than 1280 bytes.
>>
>> If people think specifying a ROLL_MIN_MTU is reasonable, we'll submit an
>> update to the rpl-routing-header draft that incorporates these changes and
>> the several other ones brought up over the past week.
>>
>> --
>> Jonathan Hui
>>

---

From: Carsten Bormann <[email-masked]>
Date: Fri, 4 Jun 2010 07:44:43 +0200

On Jun 4, 2010, at 06:30, Jonathan Hui wrote:

> ROLL_MIN_MTU

I personally think it would not be a major disaster to increase the MTU requirement for 6LRs beyond RFC 4944's 1280, as long as hosts can stay at 1280.
(If this is what ROLL decides should be done, let's have a separate discussion on any implications in 6lowpan.)

Gruesse, Carsten

---

From: Jonathan Hui <[email-masked]>
Date: Thu, 3 Jun 2010 23:02:47 -0700

On Jun 3, 2010, at 10:44 PM, Carsten Bormann wrote:

> I personally think it would not be a major disaster to increase the  
> MTU requirement for 6LRs beyond RFC 4944's 1280, as long as hosts  
> can stay at 1280.
> (If this is what ROLL decides should be done, let's have a separate  
> discussion on any implications in 6lowpan.)

Since we're on this subject, how does this relate to the requirement  
in RFC 2460 that all nodes must be able to receive datagrams of up to  
1500 bytes (after reassembly if IP-layer fragmentation is used)?  What  
is the value of keeping the MTU requirement at 1280 for hosts if they  
are still required to receive a 1500 byte datagram?  Or is is that we  
have relaxed the 1500 byte requirement for hosts in a 6lowpan network?

If a 6lowpan host needs to receive a 1500 byte datagram, I'd prefer to  
let the 6lowpan layer handle the fragmentation needs rather than both  
link and IP-layer fragmentation working simultaneously to deliver a  
1500 byte datagram.

If a 6lowpan host only needs to receive a 1280 byte datagram, then I  
agree that keeping the 1280 byte MTU for 6lowpan hosts is useful.

--
Jonathan Hui

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 4 Jun 2010 08:05:04 -0700

Hi Jonathan,

The RFC is very clear:

1. A node must be able to accept a fragmented packet that, after
   reassembly, is as large as 1500 octets.  A node is permitted to
   accept fragmented packets that reassemble to more than 1500   octets.

2. IPv6 requires that every link in the internet have an MTU of 1280
   octets or greater.

3. Links that have a configurable MTU (for example, PPP links [RFC-
   1661]) must be configured to have an MTU of at least 1280 octets; it
   is recommended that they be configured with an MTU of 1500 octets

The first case is after fragmentation, which the second case is
without fragmentation.

The thing you have is for 15.4 you could have had an MTU greater than
1280 (because the lower layer can support it), which means that you
have to fragment a packet whenever any header is attached to the
packet. In your case because there is no tunnel you have to go in for
IP-in-IP tunnelling at the edge and also fragmentation.

Thanks,
Vishwas

On Thu, Jun 3, 2010 at 11:02 PM, Jonathan Hui <[email-masked]> wrote:
>> I personally think it would not be a major disaster to increase the MTU
>> requirement for 6LRs beyond RFC 4944's 1280, as long as hosts can stay at
>> 1280.
> Since we're on this subject, how does this relate to the requirement in RFC
> 2460 that all nodes must be able to receive datagrams of up to 1500 bytes
> (after reassembly if IP-layer fragmentation is used)?  What is the value of
> keeping the MTU requirement at 1280 for hosts if they are still required to
> receive a 1500 byte datagram?  Or is is that we have relaxed the 1500 byte
> requirement for hosts in a 6lowpan network?
>
> If a 6lowpan host needs to receive a 1500 byte datagram, I'd prefer to let
> the 6lowpan layer handle the fragmentation needs rather than both link and
> IP-layer fragmentation working simultaneously to deliver a 1500 byte
> datagram.
>
> If a 6lowpan host only needs to receive a 1280 byte datagram, then I agree
> that keeping the 1280 byte MTU for 6lowpan hosts is useful.
>
> --
> Jonathan Hui
>

---

From: Jonathan Hui <[email-masked]>
Date: Fri, 4 Jun 2010 08:12:39 -0700

Hi Vishwas,

I understand that these are the requirements stated in RFC 2460.  That  
is not in question.

What I wanted was a bit more explanation on why Carsten finds it  
beneficial to keep the MTU at 1280 for 6lowpan hosts.  6lowpan needs  
to provide link-layer fragmentation anyway, so why add scenarios that  
could make IP-layer fragmentation more likely in 6lowpan networks?

--
Jonathan Hui

---

From: Jonathan Hui <[email-masked]>
Date: Fri, 4 Jun 2010 08:16:21 -0700

Hi Vishwas,

If we go down the route of specifying a RH4_MIN_MTU, I'm not so sure  
we want to send the ICMP message to any place other than the source.   
Destination unreachable messages should go to the source.  Packet too  
big messages should also go back to the source (after the RPL edge  
router does its magic to remove any side effects of RH4 headers).

Sending the ICMP message back to the source would maintain the same  
prevalent behavior within IP today.

--
Jonathan Hui

On Jun 3, 2010, at 10:40 PM, Vishwas Manral wrote:

>>>> media to
>>>> fragment a
>>>> approach
>>> option and
>>> the
>>> comforting.
>>> submit an

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 4 Jun 2010 08:21:34 -0700

Hi Jonathan,

If an intermediate device fiddles with a packet, it should get the
ICMP message.  So if the ICMP message states error in RH4 header the
ingress may not be able to do much.

In a tunnel case the following happens:

    It is possible that one of the routers along the tunnel interior
   might encounter an error while processing the datagram, causing it to
   return an ICMP [RFC-792] error message to the encapsulator at the IP
   Source of the tunnel.  Unfortunately, ICMP only requires IP routers
   to return 8 bytes (64 bits) of the datagram beyond the IP header.
   This is not enough to include the entire encapsulated header.  Thus,
   it is not generally possible for an encapsulating router to
   immediately reflect an ICMP message from the interior of a tunnel
   back to the originating host.

   However, by carefully maintaining "soft state" about its tunnels, the
   encapsulator can return accurate ICMP messages in most cases.

You can have the same behavior for ICMP in your case too is what I
have been trying to say.

Thanks,
Vishwas

On Fri, Jun 4, 2010 at 8:16 AM, Jonathan Hui <[email-masked]> wrote:
>
> Hi Vishwas,
>
> If we go down the route of specifying a RH4_MIN_MTU, I'm not so sure we want
> to send the ICMP message to any place other than the source.  Destination
> unreachable messages should go to the source.  Packet too big messages
> should also go back to the source (after the RPL edge router does its magic
> to remove any side effects of RH4 headers).
>
>>>> IP

---

From: Jonathan Hui <[email-masked]>
Date: Fri, 4 Jun 2010 08:29:19 -0700

Hi Vishwas,

But by specifying a RH4_MIN_MTU, we avoid the need for tunnels.   
Without tunnels, I'm not sure there's a good use case for ICMP errors  
caused by RH4 headers to go back to the inserting node rather than the  
source.  If the packet can't be delivered because the source route is  
bad, that's not any different than a routing failure.  If the packet  
can't be delivered because it is too big, then a packet too big  
message with MTU set to RH4_MIN_MTU - MAX_RH4_SIZE should be sent to  
the source.  Are there specific ICMP errors you have in mind that  
should be sent to the inserting node rather than the source?

--
Jonathan Hui

On Jun 4, 2010, at 8:21 AM, Vishwas Manral wrote:

>>>>>> media
>>>>>> to
>>>>>> 1280 byte packet. And for packets larger than 1280, the  
>>>>>> rewriting of
>>>>>> the
>>>>> encapsulation just to support routing.  The former approach  
>>>>> seems to be
>>>>> less
>>>>> course, as you noted, the edge router must not allow any side  
>>>>> effects of
>>>>> RH4
>>>>> ROLL_MIN_MTU, so we'll have to specify a maximum for both the  
>>>>> rpl-option
>>>>> and
>>>>>  So far, I'm not aware of any existing links intended for use  
>>>>> with RPL
>>>>> that
>>>>> update to the rpl-routing-header draft that incorporates these  
>>>>> changes
>>>>> and

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 4 Jun 2010 08:37:56 -0700

Hi Jonathan,

I agree changing the MTU to a higher value say 1500 solves the MTU issues.

The concern I have is any checks on the RH4 header by an intermediate
device fail (for example loop detected), it is not caused by the
source but by the intermediate device (also it means that RH4 header
related information may be propagated outside the domain).

Another thing we can do is send ICMP to source as well as RPL edge
(but this causes an amplification 1 packet leads to 2 reply messages).

Thanks,
Vishwas

On Fri, Jun 4, 2010 at 8:29 AM, Jonathan Hui <[email-masked]> wrote:
>
> Hi Vishwas,
>
> But by specifying a RH4_MIN_MTU, we avoid the need for tunnels.  Without
> tunnels, I'm not sure there's a good use case for ICMP errors caused by RH4
> headers to go back to the inserting node rather than the source.  If the
> packet can't be delivered because the source route is bad, that's not any
> different than a routing failure.  If the packet can't be delivered because
> it is too big, then a packet too big message with MTU set to RH4_MIN_MTU -
> MAX_RH4_SIZE should be sent to the source.  Are there specific ICMP errors
> you have in mind that should be sent to the inserting node rather than the
> source?
>
> --
> Jonathan Hui
>
>>>>>>> add
>>>>>>> a
>>>>>> encapsulation just to support routing.  The former approach seems to
>>>>>> be
>>>>>> less
>>>>>> course, as you noted, the edge router must not allow any side effects
>>>>>> of
>>>>>> RH4
>>>>>> an

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 4 Jun 2010 08:45:03 -0700

Hi Jonathan,

I agree with you. We are on the same page regarding this.

Thanks,
Vishwas

On Fri, Jun 4, 2010 at 8:12 AM, Jonathan Hui <[email-masked]> wrote:
>
> Hi Vishwas,
>
> I understand that these are the requirements stated in RFC 2460.  That is
> not in question.
>
> What I wanted was a bit more explanation on why Carsten finds it beneficial
> to keep the MTU at 1280 for 6lowpan hosts.  6lowpan needs to provide
> link-layer fragmentation anyway, so why add scenarios that could make
> IP-layer fragmentation more likely in 6lowpan networks?
>
> --
> Jonathan Hui
>
>>> let
>>> agree

---

From: Vishwas Manral <[email-masked]>
Date: Fri, 4 Jun 2010 10:32:58 -0700

Hi Jonathan,

I was just thinking and the following came to my mind.
What do you do for a packet going out of the RPL domain? You probably
have to remove the RH4 header when the packet has to go out of the
domain.

If the above is an issue, it is best solved with IP-in-IP tunnels. So
the outer IP header with the RH4 header will be removed by the edge
node and all will be good. I think you should still consider the
solution of IP-in-IP in the RPL domain, even after increasing the MTU
size.

Thanks,
Vishwas