Subject: Re: [Roll] Flow Label: 12 bits mutable and 8 bits immutable

From: Erik Nordmark <[email-masked]>
Date: Tue, 14 Sep 2010 19:35:43 -0700

On 08/11/10 04:41 PM, Philip Levis wrote:

> Basically, RPL puts up to two pieces information in packets that it
> routes. The first is which routing topology this packet should be
> routed on: RPL supports multiple parallel topologies, e.g., one
> optimized for low latency and a second optimized for low energy. This
> information is called the RPLInstanceId.
>
> The second is information about the "cost" of the forwarder's route
> to the destination. This hop-by-hop "cost" information is called
> Rank. Rank is needed to detect routing loops: if a router receives a
> packet whose prior router's Rank is less than or equal to its own,
> Rank is not strictly decreasing and there may be a routing loop.
>
> Being able to embed this information in the flow label, rather than
> requiring additional headers and IP-in-IP, would greatly reduce
> header overhead. In RPL, we're often concerned about very short, such
> as 10 byte, packets, so this could be a big savings.
>
> The challenge is that the RPLInstanceId is 8 bits and Rank is 16
> bits.

I realize that taking 48 bytes to carry 3 bytes of content (IP-in-IP 
plus a min size option is 48 bytes) seems a bit silly.

Has anybody discussed adding a header with just the 3 bytes you need 
*before* the IP header?
That avoids the overhead. The downside is that you need a new code point 
(for demux) in the different layer2s that you want to run this on.
But it seems a lot simpler than trying to overload some existing IPv6 
header field.

    Erik

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 15 Sep 2010 07:06:54 +0200

> Has anybody discussed adding a header with just the 3 bytes you need *before* the IP header?

Yes.  Ever since you proposed pretty much that at a previous IETF meeting, I've been thinking that architecturally it makes a lot of sense to think about ROLL as a sub-IP protocol.

> The downside is that you need a new code point (for demux) in the different layer2s that you want to run this on.

And that is exactly the non-starter about the general proposal.
It is not acceptable to require a new spec for each of the many link layers we want to run ROLL on, in particular for those who don't really care that much about the overhead.

However, it would be pretty easy to put something in 6lowpan to carry those 3 bytes.
(Consider it an advanced form of header compression for the 48-byte IP-in-IP thing, if you don't like the sub-IP thinking.)
Consult http://tools.ietf.org/html/draft-bormann-6lowpan-ext-hdr-00 for a sample base design.
Such a simple extension may actually be a preferable way to carry ROLL in 6lowpan.

Gruesse, Carsten

---

From: Rémi Després <[email-masked]>
Date: Wed, 15 Sep 2010 12:32:09 +0200

Le 15 sept. 2010 à 04:35, Erik Nordmark a écrit :

> ... Has anybody discussed adding a header with just the 3 bytes you need *before* the IP header?
> That avoids the overhead.

In www.ietf.org/mail-archive/web/ipv6/current/msg12204, I essentially proposed that (quote below).

> The downside is that you need a new code point (for demux) in the different layer2s that you want to run this on.

If this header could be made mandatory for RPL interfaces, this wouldn't even be necessary.

> But it seems a lot simpler than trying to overload some existing IPv6 header field.

Full agreement.

Regards,
RD

QUOTE:
<<<
De : Rémi Després <[email-masked]>
> Date : 12 août 2010 14:54:46 HAEC
> ...
>> There is certainly a strong benefit in Low Power Lossy Networks to make
>> the flow label mutable.
> 
> How strong the benefit isn't clear to me.
> Since RPL networks are recognized as such by hosts, it seems easy to add a short field, just before the IP header, to contain whatever is useful for this kind of network.
> Mixing functions with those of FLs would thus be avoided.

---

From: Fred Baker <[email-masked]>
Date: Wed, 15 Sep 2010 09:19:22 -0500

So that might use the mesh network header part of the 6lowpan header?

On Sep 15, 2010, at 12:06 AM, Carsten Bormann wrote:

> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------

---

From: JP Vasseur <[email-masked]>
Date: Wed, 15 Sep 2010 13:03:04 -0700

> So that might use the mesh network header part of the 6lowpan header?

We would need be using the compressed ID header

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 15 Sep 2010 21:06:28 +0200

> So that might use the mesh network header part of the 6lowpan header?

That would be a bit more radical, I think (and there is no place to put a rank or instance ID in RFC 4944).
But the effect is similar, as the ROLL-specific information would precede the (compressed) IP header.

The difference is in handling fragmentation: in mesh-under, each fragment is l2-routed separately; here, we route entire IP packets.
(The relationship of this extension header with fragmentation headers needs to be defined, but this seems quite simple to me: It only ever would appear on first fragments.  If you want to do the per-fragment forwarding trick instead of reassemble-before-foward, the ROLL extension header information just goes into the tag cache together with the IP header.)

Gruesse, Carsten

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 16 Sep 2010 14:07:36 +0200

> > The challenge is that the RPLInstanceId is 8 bits and Rank is 16
bits.
This much of the challenge can be absorbed by RPL. In most reasonable
cases, we could can make the floor(Rank) 8 or less bits, and that's all
we need to accomplish the purpose of this header.

> I realize that taking 48 bytes to carry 3 bytes of content (IP-in-IP
plus a min size
> option is 48 bytes) seems a bit silly.
Most certainly. Some people will say that we can live without IP in IP.
Other that we can make sure we always emit the compressed form which
hopefully would not be a lot more than the original bits.

> Has anybody discussed adding a header with just the 3 bytes you need

We've discussed using forms of tag switching from the beginning of the
WG but there must be an IP based reference model before we start doing
the tricks.

As it goes, RPL till v07 was using the flow label, using a mutable but
recoverable model. Let's see whether 6MAN allows us to make the field
mutable in the first place.

Pascal

> -----Original Message-----
> From: Erik Nordmark [mailto:[email-masked]]
> Sent: Wednesday, September 15, 2010 4:36 AM
> To: Philip Levis
> Cc: Pascal Thubert (pthubert); Carsten Bormann; [email-masked];
[email-masked];
> [email-masked]; Brian E Carpenter
> Subject: Re: [Roll] Flow Label: 12 bits mutable and 8 bits immutable
> 
to
> > the destination. This hop-by-hop "cost" information is called Rank.
> > Rank is needed to detect routing loops: if a router receives a
packet
> > whose prior router's Rank is less than or equal to its own, Rank is
> > not strictly decreasing and there may be a routing loop.
> >
header
> > overhead. In RPL, we're often concerned about very short, such as 10
> > byte, packets, so this could be a big savings.
> >
bits.
> I realize that taking 48 bytes to carry 3 bytes of content (IP-in-IP
plus a min size
> option is 48 bytes) seems a bit silly.
> 
> That avoids the overhead. The downside is that you need a new code
point (for
> demux) in the different layer2s that you want to run this on.
header
> field.

---

From: Erik Nordmark <[email-masked]>
Date: Thu, 16 Sep 2010 09:43:42 -0700

> WG but there must be an IP based reference model before we start doing

You can have a reference model which uses IP-in-IP plus a destination 
options to carry the 3 bytes, and then have an optimization for low 
speed networks like 6lowpan that sticks those bytes somewhere else as 
Carsten suggested.

Architecturally, in both cases, we'd end up with the 3 bytes before the 
(inner) IP header.

    Erik

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 16 Sep 2010 20:19:25 +0200

Hi erik,

If we compress IP in IP and a HbH header, we can hardly make that 0 bits
overhead.  The FL allows a total cost of 20 bits, whereas the
compression will end up with 5 or 6 octets minimum...

Pascal

> Sent: Thursday, September 16, 2010 6:44 PM
> To: Pascal Thubert (pthubert)
> Cc: Philip Levis; Carsten Bormann; [email-masked]; [email-masked];
options
> to carry the 3 bytes, and then have an optimization for low speed
networks like
> 6lowpan that sticks those bytes somewhere else as Carsten suggested.
>

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 15 Sep 2010 08:51:20 +0200

Gruesse, Carsten

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 17 Sep 2010 15:55:16 -0700

> compression will end up with 5 or 6 octets minimum...

You misunderstood my suggestion.

    Erik

---

From: JP Vasseur <[email-masked]>
Date: Sat, 18 Sep 2010 02:22:46 +0200

> Yes.  Ever since you proposed pretty much that at a previous IETF meeting, I've been thinking that architecturally it makes a lot of sense to think about ROLL as a sub-IP protocol.

Architecturally RPL is not a "sub-IP" protocol, but lies at the networking layer. Remember that RPL is link layer "agnostic" and no assumption is made on the link layer.

> Such a simple extension may actually be a preferable way to carry ROLL in 6lowpan.

"preferable" in which sense ?

Thanks.

JP.

> 
> Gruesse, Carsten
> 
> _______________________________________________
> Roll mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/roll

---

From: Rémi Després <[email-masked]>
Date: Sat, 18 Sep 2010 16:53:02 +0200

> "preferable" in which sense ?

At least, IMHO, in that it eliminates the motivation to interfere with the current discussion on improving flow-label utilization.

Regards,
RD

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 18 Sep 2010 15:43:11 +0200

On Sep 18, 2010, at 02:22, JP Vasseur wrote:

> Architecturally RPL is not a "sub-IP" protocol, 

You are right, I was too terse: it contains one.
Architecturally, the instance-id acts a lot like a VLAN tag, and the rank is something that happens "alongside" IP forwarding -- whether that's below, above, or sideways depends on your vantage point.
But we probably don't need to discuss the fine architectural points here.
(After thirty years, I'm a bit tired of trying to compress complex reality into "layers".)

My point is not about labelling RPL as anything, but about making use of knowledge we already have.
Erik's comments have been pretty eye-opening here for me.

> "preferable" in which sense ?

In the sense of good performance and minimal impact.
Again, just think of it as a 6lowpan-optimized form of header-compressing the 56-byte*) IP-in-IP/extension-header wrapper that other link layers will happily carry.

Gruesse, Carsten

*) (Note that 6lowpan-HC as is will probably already compress that wrapper to approximately 14 bytes -- I haven't done a fully worked example yet, though, as the RPL option processing is not yet fully defined, and there is a lot of complexity to get down to those ~ 14 bytes.)

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 21 Sep 2010 14:08:14 +0200

Hi Rémi:

It would not. 

We'll be very glad that 6LoWPAN compresses RPL optimally. But RPL being layer 2 agnostic cannot depend on 6LoWPAN. 
Header and IP in IP insertion is problematic on any network, be it for the MTU issues only.

The FL for RPL discussion illustrates that there can be multiple valuable usages of the field by the network. 
It mostly shows that tying the usage to local balancing only is probably short sighted.

Cheers,

Pascal

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Rémi
> Després
> Sent: Saturday, September 18, 2010 4:53 PM
> To: JP Vasseur
> Cc: IPv6 WG; ROLL WG
> >> (Consider it an advanced form of header compression for the 48-byte
> >> IP-in-IP thing, if you don't like the sub-IP thinking.) Consult
> http://tools.ietf.org/html/draft-bormann-6lowpan-ext-hdr-00 for a sample base
> design.
> At least, IMHO, in that it eliminates the motivation to interfere with the current
> discussion on improving flow-label utilization.
> 
> Regards,
> RD
> 
>

---

From: Rémi Després <[email-masked]>
Date: Tue, 21 Sep 2010 14:27:31 +0200

Le 21 sept. 2010 à 14:08, Pascal Thubert (pthubert) a écrit :

> Header and IP in IP insertion is problematic on any network,

Couldn't the additional header start with e.g. a 0xF so that it can be distinguished from an IPvX header.
(These 4 bits, plus the 20 that would be sufficient if FLs were to be used, would make a nice 3 octets.)
It would then be layer-2 agnostic. 

> be it for the MTU issues only.

Why?
Isn't each traversed link authorized to have its link MTU?

Cheers,
RD

---

From: Michael Richardson <[email-masked]>
Date: Tue, 21 Sep 2010 09:48:39 -0400

>>>>> "Pascal" == Pascal Thubert <(pthubert)" <[email-masked]>> writes:
    Pascal> Hi Rémi:

    Pascal> We'll be very glad that 6LoWPAN compresses RPL
    Pascal> optimally. But RPL being layer 2 agnostic cannot depend on

Only when the security is provide by the layer 2 can the layer 2 do any
compression.   Otherwise, the encryption of the RPL messages renders the
compression useless. 

Saying that RPL is layer-2 agnostic doesn't mean that it will perform
well on all layer-2s. the interface between layer 2 and layer 3 needs to
be carefully managed.

If you want layer-2 agnostic compression, it has to be done before the
encryption, and that means at layer-3.

-- 
]       He who is tired of Weird Al is tired of life!           |  firewalls  [
]   Michael Richardson, Sandelman Software Works, Ottawa, ON    |net architect[
] [email-masked] http://www.sandelman.ottawa.on.ca/ |device driver[
   Kyoto Plus: watch the video <http://www.youtube.com/watch?v=kzx1ycLXQSE>
	               then sign the petition.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 21 Sep 2010 16:49:38 +0200

It is. 

But the real world tells us that various MTUs over various paths is problematic. In particular it creates states in the hosts which can be subject to attacks. Middle box happen to defeat PMTUD. Encapsulation for insertion will probably hit the router performance unless there's special silicon for that operation, which current routers obviously do not have. Etc...

Pascal

> From: Rémi Després [mailto:[email-masked]]
> Sent: Tuesday, September 21, 2010 2:28 PM
> Cc: JP Vasseur; IPv6 WG; ROLL WG
> > We'll be very glad that 6LoWPAN compresses RPL optimally. But RPL being
> layer 2 agnostic cannot depend on 6LoWPAN.
> (These 4 bits, plus the 20 that would be sufficient if FLs were to be used, would
> make a nice 3 octets.) It would then be layer-2 agnostic.
> 
> >> From: [email-masked] [mailto:[email-masked]] On Behalf
> >> Of Rémi Després
> >>>> However, it would be pretty easy to put something in 6lowpan to
> >>>> carry those
> >> 3 bytes.
> >> http://tools.ietf.org/html/draft-bormann-6lowpan-ext-hdr-00 for a
> >> sample base design.
> >>>> Such a simple extension may actually be a preferable way to carry
> >>>> ROLL in
> >> 6lowpan.
> >>>
> >> At least, IMHO, in that it eliminates the motivation to interfere
> >> with the current discussion on improving flow-label utilization.
> >>
> >> Regards,
> >> RD
> >>
> >>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 21 Sep 2010 17:05:04 +0200

You are correct, Carsten:

This also relates to the concept that was evoked earlier in the thread
to allow an implicit format on a RPL link. There's no such thing as a
RPL link. As you say, RPL control itself does not exhibit the HbH header
since most of it is 1 hop - link local. And then a link upon which RPL
is active can be shared with other activities that are not RPL related
at all. If the HbH is present, then the route lookup will be done along
a VRF that is indexed by the instanceID. Other tagging might come in the
future. And then there's the default RIB when the router has no better
indication in the packet.

I like your vlan analogy. It is quite usual in practice to map vlans
onto vrfs. Since we are doing route over, we are quite naturally
integrating the concepts. For the time being and considering the
constraints that apply to RPL, we found that one octet was the right
trade off to represent an instance. We'll probably see a day, though,
where the concept generalizes, and at that time we'll probably need 4
octets at least. In particular, RPL or something very much like RPL
would probably be very interesting in hierarchical access domains like
DSL where 1:1 vlans apply today.

Take care,

Pascal

> From: Carsten Bormann [mailto:[email-masked]]
> Sent: Tuesday, September 21, 2010 4:18 PM
> To: Michael Richardson
> Cc: Pascal Thubert (pthubert); ROLL WG; IPv6 WG
> On Sep 21, 2010, at 15:48, Michael Richardson wrote:
> 
> I think Pascal was talking about the encapsulation RPL provides for
data packets
> (RPL option, draft-ietf-6man-rpl-option-00.txt), not about actual RPL
messages.
> Do you think the encapsulation information will be encrypted?
> 
> Gruesse, Carsten

---

From: Rémi Després <[email-masked]>
Date: Tue, 21 Sep 2010 17:10:24 +0200

Le 21 sept. 2010 à 16:49, Pascal Thubert (pthubert) a écrit :

> It is. 
> 
> But the real world tells us that various MTUs over various paths is problematic.

Isn't this independent from RPL, and from whatever MTU one applies locally?

Actually, this is a nice feature of IPv6 that every network under an administrative control has to make sure it supports 1280 + whatever headers are needed to traverse.
Hosts, except for on-link neighbors, should better send with a 1280 MTU.
(This constraint will disappear when PMTUD becomes reliable, but it may take quite a while.)

> In particular it creates states in the hosts which can be subject to attacks. Middle box happen to defeat PMTUD.

> Encapsulation for insertion will probably hit the router performance unless there's special silicon for that operation,

The change of size only appears at the border between RPL and ordinary IP, right?

> which current routers obviously do not have. Etc...

Etc. Etc.
It looks like you insist on destroying FL compatibility with what it has been specified for.

IMHO, working on RPL solutions without interfering with FLs has a better chance to permit consensus, both for RPL and for FLs.

RD

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 21 Sep 2010 16:18:22 +0200

> compression.   Otherwise, the encryption of the RPL messages renders the

I think Pascal was talking about the encapsulation RPL provides for data packets (RPL option, draft-ietf-6man-rpl-option-00.txt), not about actual RPL messages.  Do you think the encapsulation information will be encrypted?

Gruesse, Carsten

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 24 Sep 2010 10:47:07 +0200

> It looks like you insist on destroying FL compatibility with what it has been

What was it defined for, indeed?

The story starts in the mid-90s and is contiguous with the early Ypsilon work on flow switching. It's quite natural that at that time, the concept of flow was integrated in the IPv6 header, in a same fashion that byte alignment in enforced in the headers since that was also a concern of the day.

While IPv6 adopted the wait-and-see position, flow switching evolved into tag switching and then MPLS. From an IPv6 perspective, what can we see after all this waiting? That the real value of the tag has nothing to do with the initial MP (IPv4 has been hegemonic ever since) or the LS (IP routing in silicon has rendered the performance discussion obsolete more than ten years ago).

The real value that we can clearly observe today appeared with the applications to VPN and TE.  IOW, the value of the tag derives from the capability in the router to select a path that's not the one the main RIB would have indicated. Modern networks need VPN, TE and MTR capabilities and MPLS bloomed in a niche that IPv4, with no mean to do this tagging, could simply not start to address.

What does that tell us? Probably that the use of the flow label as a tag to influence the routing of the packet (eg a VRF index) is not so far from the initial intent but simply its natural evolution. Clearly, tags are employed in the core in such a way that a simple 20 bit field in the header can never suffice to emulate. But OTOH, at the edge of the network, it is still possible to place a tag in the flow label that will be mapped into a MPLS label at the border. This is quite naturally the route-over version of the mapping of a 1Q tag into a label that we commonly do today.

Pascal
http://www.xtranormal.com/watch/7011357/

> Sent: Tuesday, September 21, 2010 5:10 PM
> Actually, this is a nice feature of IPv6 that every network under an
> administrative control has to make sure it supports 1280 + whatever headers
> are needed to traverse.
> > Encapsulation for insertion will probably hit the router performance
> > unless there's special silicon for that operation,
> 
> IMHO, working on RPL solutions without interfering with FLs has a better
> chance to permit consensus, both for RPL and for FLs.
> 
> RD
> 
> 
> > Pascal
> >
> >
> >> (These 4 bits, plus the 20 that would be sufficient if FLs were to be
> >> used, would make a nice 3 octets.) It would then be layer-2 agnostic.
> >>
> >>> The FL for RPL discussion illustrates that there can be multiple
> >>> valuable
> >> usages of the field by the network.
> >>> It mostly shows that tying the usage to local balancing only is
> >>> probably short
> >> sighted.
> >>>
> >>> Cheers,
> >>>
> >>> Pascal
> >>>
> >>>> From: [email-masked] [mailto:[email-masked]] On
> >>>> Behalf Of Rémi Després
> >>>>>> (Consider it an advanced form of header compression for the
> >>>>>> 48-byte IP-in-IP thing, if you don't like the sub-IP thinking.)
> >>>>>> Consult

---

From: Rémi Després <[email-masked]>
Date: Fri, 24 Sep 2010 17:01:55 +0200

Thanks Pascal for the historical tutorial.
Having not participated in IETF in the mid 90s, I find it interesting.  
My FL reference to FLs is is essentially based on the "IPv6 Flow Label Specification" of RFC 3697, in 2004.

> What was it defined for, indeed?

RFC 3697 says "The 20-bit Flow Label field in the IPv6 header [IPv6] is used by a source to label packets of a flow".
The RFC doesn't really say for what purpose but, in the absence of any additional signaling that would permit more influence on routing, the purpose seems to just be an influence on load balancing?
Besides, FLs are for this an effective approach because intermediate routing nodes that do load balancing cannot reconstruct 5-tuples for all packets they have to forward.

That's why I believe that, if RFC 3697 is revised, in particular to clarify what intermediate nodes can do with FLs, it should only be without destroying what they are good for.

RD