Subject: [mpls] draft-villamizar-mpls-tp-multipath

From: Yong Lucy <[email-masked]>
Date: Thu, 22 Jul 2010 15:19:03 -0500

Hi Curtis,

Thank you for summarizing hashing based multipath solution in this draft.
Share my opinion here.

Draft Text:

   An alternate simple multipath technique uses a table

   generally with a power of two size, and distributes the table entries

   proportionally among component links according to the capacity of

   each component link.

   An adaptive multipath technique is one where the traffic bound to

   each component link is measured and the load split is adjusted

   accordingly.   

End Text

Comment:

1) Adjusting load over component link may cause flow reordering. 

2) If a huge amount of micro flows mix with few large and long live flows,
the load per entry can be out of balance

3) Measure component link load does not provide the info. for table entry
mapping adjustment

Internet traffic pattern today is different from decade or more years ago.
Hashing works well under the condition 

1) # of flows is very large and flow IDs are statistically random; 2) flow
BWs are pretty balanced. 

Otherwise, hashing has a trouble to make even load balance.  Hashing +
bucket mapping brings a room to deal with the unbalance if each bucket load
is measured. 

If the equipment understands flows better rather than just flow ID, more
proactive way can apply to multipath load balance. This is why we suggest
having large flow classification (draft-yong-pwe3-lfc-fat-pw-01)

Regards,

Lucy

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 27 Jul 2010 10:51:11 -0400

> Thank you for summarizing hashing based multipath solution in this draft.

Thanks for reading the draft.

> 1) Adjusting load over component link may cause flow reordering. 

Yes.  That is why adjustments should be minimizaed and the why
adjustments should avoid moving the same traffic back and forth if it
sits on some sort of mathematical boundary.

> the load per entry can be out of balance

For a hash space size of 24 bits, 1/16,000,000 of the tiny flows will
be mixed in the same hash bucket as a large flow.  This is why large
flows can only be sized at max-component BW minus epsilon, where
epsilong depends on the hash space size.

Perhaps you haven't figured out how a very large hash space can be
supported with a modest amount of hardware.  It can.  You might want
to look at the expired optimized multipath drafts in data-tracker.
There is a little more detail there.

> 3) Measure component link load does not provide the info. for table entry

The measurement is much finer that component link.  Again, look at the
expired optimized multipath drafts in data-tracker.

This is a solved problem.  The sections that describe how it was
successfully done a decade ago is intended only to provide an
existance proof that there is a simple solution.  There are other
solutions that make use of the same properties of the hash but build
the hardware somewhat differently than described in the OMP drafts.

> Hashing works well under the condition 

Yes.  I know that.

> 1) # of flows is very large and flow IDs are statistically random; 2) flow

The number of flows observed within a minute interval 15 years ago was
in the millions.  Today in LAGs of tens of 10GbE, it is likely to be
in the billion range.  I am aware of this.  If 2% of the flows are
large then 2% of a billion is also large meaning that they are very
numerous.  Building core equipment support 20 million hardware entries
to handle such a case is not a good suggestion.

> bucket mapping brings a room to deal with the unbalance if each bucket load

You clearly don't understand how this works.  There is a finite
probability that too many large flows end up in the same hash bucket
but with a large hash space this probability becomes extremely small.
The consequence is modestly disruptive, a hash reseed, but the
probability of needing two consecutive reseeds is miniscule.  I've
studied this problem but didn't put anything in the draft.

> having large flow classification (draft-yong-pwe3-lfc-fat-pw-01)

I don't see any support at all for that.  I didn't see any signs of
support for it this morning in PWE.

I also didn't see any considerations for MPLS-TP in your draft.

This draft is about extending a current practice to support a new
requirement - having MPLS act as a server layer that can carry fully
compliant MPLS-TP client layer LSP.

> Lucy

Curtis

---

From: Yong Lucy <[email-masked]>
Date: Tue, 27 Jul 2010 11:02:05 -0500

Cutis,

Please see inline.

> epsilong depends on the hash space size.
[LY] This is true when only hashing is used. 
> There is a little more detail there.
[LY] hashing is stateless and provides the simple way to support the large
hash space. This is not the problem we address. 
> to handle such a case is not a good suggestion.
[LY] If large flow has the notion that its rate is about the threshold, it
is easy to count how many large flows a link can support. Be honest, we
don't see such many large flows in today's network.
For a 10x10GE link,
If 500Mbps is the threshold for large flow, the link can not take more than
200 large flows.
If 50M is the mark, the link can not take more than 2000 large flows.
If 5M is the mark, the link can not take more than 20000 large flows.

[LY] We don't think the flow with 5Mpbs belonging to large flow class. We
need to speak to reality. It seems that we can different ways to count ratio
of large flow ratio.

Regards,
Lucy

Snip

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 27 Jul 2010 15:05:33 -0400

> hash space. This is not the problem we address. 

This thread is not about your draft.  It is about a draft which
extends an existing widely implemented technique used with MPLS and
multipath to support MPLS-TP.

> If 500Mbps is the threshold for large flow, the link can not take more than

500 Mbps x 200 is 100Gb/s is the whole 10x10G.

> If 50M is the mark, the link can not take more than 2000 large flows.

50 Mbps x 2000 is 100Gb/s is the whole 10x10G.

> If 5M is the mark, the link can not take more than 20000 large flows.

5 Mbps x 20000 is 100Gb/s is the whole 10x10G.

So total is 300G.  Is something wrong with your math here?

The large flows come and go.  A flow for an enterprise VPN may be
small all day and then run a database backup.  They are not going to
signal any LSP differently before this happens.  In the core the PW
LSP associated with this flow may be inside another MPLS LSP used to
improve network scalability.  Most likely the PW with be inside an LDP
LSP which is inside a TE LSP in the core.

> need to speak to reality. It seems that we can different ways to count ratio

Once again, this is not about your draft.  Start another thread for
that.  This is about small changes to an existing MPLS technique that
allow MPLS (without the TP) to be used as a server layer for MPLS-TP.
There is no signaling of large or small flows so that topic is
irrelevant to this draft.

Curtis

---

From: Yong Lucy <[email-masked]>
Date: Wed, 28 Jul 2010 03:31:21 -0500

Curtis,

You are right. This is not thread for large flow classification. Sorry.

I agree that the flow size can't be larger than component link size minus
the epsilon.

PS: you missed my point about large flow mark in previous mail.

Regards,
Lucy

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Tuesday, July 27, 2010 2:06 PM
> To: Yong Lucy
> Cc: [email-masked]; [email-masked]
> Subject: Re: draft-villamizar-mpls-tp-multipath
> 
> 
> > > > Internet traffic pattern today is different from decade or more
> years
> > > ago.
> > > > 1) # of flows is very large and flow IDs are statistically random;
2)
> > > flow

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 28 Jul 2010 05:07:15 -0400

In message <[email-masked]>
Yong Lucy writes:
> I agree that the flow size can't be larger than component link size
> minus the epsilon.
>  

OK.  What point did I miss.  Could you please summarize.

Curtis