Subject: [mpls] Some comments on draft-kompella-mpls-entropy-lables

From: Stewart Bryant <[email-masked]>
Date: Wed, 09 Jul 2008 12:07:12 +0100

The Use of Entropy Labels in MPLS Forwarding
                  draft-kompella-mpls-entropy-lables

1.  Introduction

   Load balancing, or multi-pathing, is an attempt to balance traffic
   across a network by allowing the traffic to use several paths, not
   just a single shortest path.  Load balancing has several benefits: it
   eases capacity planning; it can help absorb traffic surges by
   spreading them across several links; it allow better resilience by
   offering alternate paths should a link or node fail.

SB> This last point is not correct as it stands. When a link or
SB> note fails, it is good to have another high capacity path
SB> and if it's ECMP you can use it for FRR, but the path
SB> itself will blackhole until you take action.

  There are three key
   reasons why this is beneficial:

   1.  at the ingress of the LSP, MPLS encapsulation hasn't yet
       occurred, so deep inspection is not necessary;

SB> Sure it's less deep, but it's still the standard tuple and
SB> most folks would call that DPI. However I agree that it
SB> is less deep.

1.1.  Motivation

   Currently, each MPLS LSR along a given path needs to individually
   infer the underlying protocol within a MPLS packet in order to then
   extract appropriate keys from the payload.  Those keys are then used
   as input into a hash algorithm to determine the specific output
   interface on a LSR that is used for that given "microflow".
   Unfortunately, if the MPLS LSR is unable to infer the MPLS packet's
   payload (as is often the case), they typically will resort to using
   the topmost MPLS labels in the MPLS stack as keys to the load-hashing
SB> Do you mean top or bottom?

   algorithm.  The result is an extremely inequitable distribution of
   traffic across multiple equal-cost paths exiting that node, simply
   because the topmost MPLS labels are very coarse-grained forwarding
   labels that typically describe a next-hop, or provide some other type
   of mux/demux forwarding function, and do not describe the granularity
   of the underlying traffic.

2.  Approaches

3.  Entropy Labels

   An entropy label (as used here) is a label:

   1.  that is not used for forwarding;

   2.  that is not signaled; and

   3.  whose only purpose in the label stack is to provide "entropy" to
       improve load balancing.

   Entropy labels are generated by an ingress LSR, based entirely on
   load balancing information.  However, they MUST not have values in
   the reserved label space (0-15).  Entropy labels MUST be at the
   bottom of the label stack, and thus the "end-of-stack" bit in the
   label should be set.  To ensure that they are not used inadvertently
SB> MUST be set
   for forwarding, entropy labels SHOULD have a TTL of 0.
SB> That is a good idea, we will put it in the PW draft.

   Since entropy labels are generated by the ingress LSR, an egress LSR
   MUST be able to tell unambiguously that a given label is an entropy
   label.  This of course depends on the underlying application.  If any
   ambiguity is possible, the label above the entropy label MUST be an
   "entropy label indicator" (ELI), which says that the following label
   is an entropy label.  The ELI may be signaled, or may be a reserved
   label reserved specifically for this purpose.  Fortunately, for many
   applications, the use of entropy labels is unambiguous, and does not
   need an ELI.

SB> I would be worried about using a reserved label for this.
SB> I don't think its actually needed and they are in short supply.
SB> We are going to need a forwarding path that gets context from a
SB> signaled label and discards the following label for a number of the
SB> cases, so we should try to use that method everywhere.

4.  Forwarding and Load Balancing Behaviors for Entropy Labels

   Thus, transit LSRs are almost unaffected by the use of entropy
   labels.  If transit LSRs were programmed to use a subset of the label
   stack, they may have to be reconfigured to use the full stack.  But
   otherwise, no changes are needed.

SB> Surely they just need to use the bottom of stack?

4.3.  Egress LSRs

SB> Should you consider PHP of entropy labels.

6.  Security Considerations

   Having security is a Good Thing.

SB> Firstly it's worth noting in the Security section that care is 
needed not
SB> to make this label some sort of covert channel.
SB>
SB> Secondly kind of the opposite of the above. You called it an entropy
SB> label and we (who independently thought this up) called it a load
SB> balance label. We should pick on or the other. However that causes
SB> to wonder if there is some other use that we will find for that 20
SB> bits? If so perhaps we should call it a context label?
SB> I put that remark down here because it flys in the face of my
SB> first comment about covert channels.

7.  Acknowledgments

   We wish to thank Ulrich Drafz for his contributions, as well as the
   entire "hash label" team for their valuable comments and discussion.

SB> There are so few of us we could probably be named :)



_______________________________________________
mpls mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mpls

---

From: "Vishwas Manral" <[email-masked]>
Date: Wed, 9 Jul 2008 05:47:20 -0700

Hi Stewart/ Kireeti,

I personally find the draft very helpful though I cannot find link to
the draft, this is the exact point I raised on the other drafts too,
the idea of Flow labels being used.
http://www.ietf.org/mail-archive/web/mpls/current/msg02080.html

Kireeti, there are other advantages of flow labels I had mentioned in
my earlier mail, besides ECMP. You may want to include that in your
draft too. Having worked on middle boxes these are very real issues.

Some other things to consider are, reserved labels not being the
bottom of stack always and deriving information of the bottom of stack
from the last label, and deriving information like the packet protocol
(IPv4 or IPv6 based on that label).

Thanks,
Vishwas

On Wed, Jul 9, 2008 at 4:07 AM, Stewart Bryant <[email-masked]> wrote:
> SB> Firstly it's worth noting in the Security section that care is needed
> not

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 09 Jul 2008 14:05:54 +0100

Wim
> WH> If we use TTL=0 and we use PHP on the egress LER with BGP shortcuts,
> the packet on the egress LER will be received with TTL=0. The egress LER
> will drop this packet normally.
>
> I see 3 options for this:
> 1. Either use TTL=1 for the entropy label
> 2. Do something different for BGP shortcuts, introduce a service label
> like is done for 6PE e.g., but this requires changes from today's
> implementations and is always hard.
> 3. Avoid PHP with entropy label , but this changes again current
> deployment models which is always though.
>
> For me option 1 seems the best, unless I miss something.
>
That is a good point. In the case of PWE3 TTL=0 would  work because we 
still have
the PW label after PHP of the PE delivery label. However PWE3  should 
adopt the same
policy as MPLS.

Stewart

---

From: Shane Amante <[email-masked]>
Date: Wed, 09 Jul 2008 09:03:07 -0600

Stewart,

> SB> itself will blackhole until you take action.

Actually, I believe the point is correct.  Specifically, in operation of 
a network, it has been observed that as the number of component-links in 
a link-bundle increases, one tends to experience a higher rate of 
failure of individual component-link members, generally due to subtle 
transmission errors.  Rather than have just 1 component-link 
(experiencing only a momentary transmission problem) taking down a very 
large link-bundle, operators may over-build paths with an extra 
component-link to absorb a [brief] failure of any individual member. 
This tends to be significantly less disruptive to traffic overall.

Second, with respect to node failure, once a failure is detected (either 
through Loss of Light or, perhaps, BFD) it is *automatically* propagated 
via the IGP throughout the network.  Thus, the network should recover 
automatically in a very short period of time.  Finally, operators are 
typically aware that if they are load-balancing traffic over an ECMP 
path to alleviate capacity constraints of a single path, they could have 
(big) problems when one of the multiple paths fails.  Therefore, they 
may plan to load-balance over, say, 3x ECMP paths in order that if any 
one path fails there is only a reduction of 33% of capacity, which may 
be adequate to continue forwarding without any traffic loss.

> SB> most folks would call that DPI. However I agree that it

Although it may be a "standard tuple", the larger problem is *where to 
find it* or how deep in a packet you're going to have to search to find 
something that adequately and accurately describes a "microflow".  For 
example, consider the following very brief list of encapsulations:

IPv4/GRE/IPv4
IPv6/IPv4 (IP-in-IP)
IPv4/Ethernet(Untagged)/MPLS
IPv4/Ethernet(NxVLAN Tags)/MPLS
...

> SB> Do you mean top or bottom?

Perhaps we should remove 'topmost' from that sentence, since 
implementations are free to do as they wish, just as long as they are 
self-consistent.  However, even if they are using the bottomost labels, 
which is better, it still ultimately resorts in "macroflows".

> SB> That is a good idea, we will put it in the PW draft.

I will follow-up on this in the separate thread that spawned after this 
e-mail.

> SB> cases, so we should try to use that method everywhere.

We should discuss this further.  I agree reserved labels are in 
short-supply, however having a reserved label for an ELI would seem to 
be a good use case.  A reserved label could have the advantage that HW 
would be able to quickly recognize it and not attempt to use it for a 
FIB look-up for forwarding.  (There's less that can go wrong with 
forwarding if an ELI is not dynamically signaled).

I'll follow-up on the rest later.

-shane

---

From: George Swallow <[email-masked]>
Date: Wed, 09 Jul 2008 17:50:13 -0400

I disagree with the text below.  It is my understanding that a lot of
equipment does ECMP by looking for EOS, checking the first nibble for 0100b
and if true running the usual IPv4 load-balancing.  Only if it is not 0100b
(and maybe 0110b) does it actual use some portion of the label stack.  This
is why in PW, the first nibble of the CW is 0000b.

It seems to me the only way to be sure that this will work is to define a
new FEC in LDP. This would be equivalent to an IP FEC, but say that you must
load balance on the label stack only.

...George

4.2.  Transit LSR

   Transit LSRs have no change in forwarding behavior.  For load
   balancing, transit LSRs SHOULD use the whole label stack (e.g., for
   computing the load balance hash).  Transit LSRs MAY choose to look
   beyond the label stack for further load balancing information;
   however, if entropy labels are being used, this may not be very
   useful.  In a mixed environment (or for backward compatibility), this
   is the simplest approach.

---

From: "HENDERICKX Wim" <[email-masked]>
Date: Wed, 9 Jul 2008 14:53:38 +0200

Kireeti/Steward

Some remarks in-line regarding TTL=0 for the entropy label

Cheers,
Wim

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Stewart Bryant
Sent: woensdag 9 juli 2008 13:07
To: Kireeti Kompella; Shane Amante; Clarence Filsfils
Cc: [email-masked]
Subject: [mpls] Some comments on draft-kompella-mpls-entropy-lables

1.  Introduction

SB> This last point is not correct as it stands. When a link or note 
SB> fails, it is good to have another high capacity path and if it's 
SB> ECMP you can use it for FRR, but the path itself will blackhole 
SB> until you take action.

  There are three key

SB> Sure it's less deep, but it's still the standard tuple and most 
SB> folks would call that DPI. However I agree that it is less deep.

1.1.  Motivation

2.  Approaches

3.  Entropy Labels

4.3.  Egress LSRs

SB> label and we (who independently thought this up) called it a load 
SB> balance label. We should pick on or the other. However that causes 
SB> to wonder if there is some other use that we will find for that 20 
SB> bits? If so perhaps we should call it a context label?
SB> I put that remark down here because it flys in the face of my first 
SB> comment about covert channels.

7.  Acknowledgments

---

From: "HENDERICKX Wim" <[email-masked]>
Date: Wed, 9 Jul 2008 15:11:39 +0200

Steward,

Agreed, we should define one architecture which works in all scenarios
according to me.
PWE is the priority though, but we should architect it for IP-VPN and
BGP shortcuts, etc.

Are you in favor of TTL=1? The other options might be to much impact for
existing deployments according to me.

Cheers,
Wim

-----Original Message-----
From: Stewart Bryant [mailto:[email-masked]] 
Sent: woensdag 9 juli 2008 15:06
To: HENDERICKX Wim
Cc: Kireeti Kompella; Shane Amante; Clarence Filsfils; [email-masked]
Subject: Re: [mpls] Some comments on draft-kompella-mpls-entropy-lables

Wim
> WH> If we use TTL=0 and we use PHP on the egress LER with BGP 
> WH> shortcuts,
> the packet on the egress LER will be received with TTL=0. The egress 
> LER will drop this packet normally.
>
> 1. Either use TTL=1 for the entropy label 2. Do something different 
> for BGP shortcuts, introduce a service label like is done for 6PE 
> e.g., but this requires changes from today's implementations and is 
> always hard.
That is a good point. In the case of PWE3 TTL=0 would  work because we
still have the PW label after PHP of the PE delivery label. However PWE3
should adopt the same policy as MPLS.

Stewart

---

From: Shane Amante <[email-masked]>
Date: Sun, 13 Jul 2008 21:48:24 -0600

George,

On Jul 9, 2008, at 3:50 PM, George Swallow wrote:
> I disagree with the text below.  It is my understanding that a lot  
> of equipment does ECMP by looking for EOS, checking the first nibble  
> for 0100b and if true running the usual IPv4 load-balancing.  Only  
> if it is not 0100b (and maybe 0110b) does it actual use some portion  
> of the label stack.  This is why in PW, the first nibble of the CW  
> is 0000b.

>
> It seems to me the only way to be sure that this will work is to  
> define a new FEC in LDP. This would be equivalent to an IP FEC, but  
> say that you must load balance on the label stack only.

Although an operator could, at the appropriate point in time,  
(manually) configure the transit LSR's to only pay attention to the  
label stack for load-balancing, this suggestion is likely better for  
backwards compatibility, so we'll take it under advisement.

Thanks,

-shane

---

From: Shane Amante <[email-masked]>
Date: Sun, 13 Jul 2008 22:21:17 -0600

Stewart,

Finishing the second half of this e-mail.  Apologies for the delay.

> SB> to make this label some sort of covert channel.

I'm not sure I understand why you think an entropy label could become  
a 'covert channel'.  Do you mean for the purposes of quickly  
identifying a microflow?  If so, that risk already exists today, (by  
looking for EOS + 0100b, etc.).

> SB> balance label. We should pick on or the other. However that causes

I understand where you're coming from, but one advantage of calling  
this an entropy label is that it's at the Tunnel Label level vs. the  
Load-Balance Label, which is at the PW level.  While "entropy label"  
might not be the coolest name, perhaps it's worth keeping them  
different to avoid confusion down the road?

> SB> first comment about covert channels.

I would advocate a more narrow definition of a entropy-label, unless  
you can think of some actual use cases where we would re-use these 20- 
bits.

> SB> There are so few of us we could probably be named :)

We will do that in the next rev.

Thanks,

-shane