Subject: Re: LC Comments on "considerations" scaling analysis

From: Eric Rosen <[email-masked]>
Date: Thu, 09 Jul 2009 13:13:50 -0400

If you are going to recommend one scheme over another based on a scalability
analysis, then the analysis must  take into account all the scaling factors.
You do not have the option  of saying "I didn't consider memory utilization,
that's out of scope", or "I didn't consider sparse mode, that's out of scope
(even though it is an important factor in virtualyl every deployment)".

Further,  if  you want  to  draw any  conclusions  whatsoever  based on  the
analysis of a small subset of  the scalability issues, you have to show that
that subset contains the bottleneck issues.  If the MVPN bottleneck is PE-CE
PIM, then you can't say "I'm only considering PE-PE PIM, PE-CE PIM is out of
scope". 

It also helps if the analysis  uses a measure of "resource utilization" that
has some practical meaning.  The sum of the number of cycles expended by all
routers has no practical  meaning at all; if it did, no  one would ever have
heard of such notions as "distributed computing" or "datagram networks".

If  you do  a scalability  analysis without  attending to  any of  this, the
analysis  is simply  irrelevant,  if it  happens  to be  correct within  its
limited scope.

Of course, if you are doing  a purely theoretical analysis, with no intended
practical application, then of course you can focus in on any isolated issue
that you choose.  However, I don't  think the IETF is the proper publication
venue for that.

A  further  issue  is that  the  analysis  provided  in  Appendix A  of  the
"considerations" draft is not correct even within its limited scope, for all
the reasons  I have detailed  in a prior  message.  These mostly have  to do
with the  failure to  properly understand the  effects of  join suppression.
But I'm not sure it's worthwhile to repeat all those points again as part of
the same last call.

I will just  reiterate my request for  Appendix A to be removed  and for any
recommendations depending upon it to be removed.

---

From: Thomas Morin <[email-masked]>
Date: Fri, 10 Jul 2009 17:54:55 +0200

Hi Eric, working group,

> recommendations depending upon it to be removed.

Indeed, your email only consists in reiterating the arguments of your 
previous email on the subject. 
(Ok, not only, I admit: you also use bigger words to sound more convincing.)

What I see is that:
- your claims that about the analysis in Appendix A are abusive (you 
still did not produce any quote of something that would be incorrect, 
and did not address my reply to your claims of incorrectness)
- the scalability-related observations of this analysis are used 
elsewhere in the document among other arguments, not alone, and lead to 
recommendations that are well balanced between theoretical aspects and 
practical aspects ("[recommendation] that implementations support both 
the BGP-based and the full per MPVN PIM peering solutions for PE-PE 
transmission of customer multicast routing until further operational 
experience is gained with both solutions")

So, I don't see a rationale to follow your request.

I have a few more precise comments inlined below (I'll take the time of 
trying to address your different comment, even if  I honestly find them 
quite abusive, and even if experience tells me that you are unlikely to 
reply).

-Thomas

Eric Rosen:
> that's out of scope", 
(I don't get why you say this, memory utilization due to mVPN routing 
state maintenance *is* accounted for in the analysis in Appendix A)

> or "I didn't consider sparse mode, that's out of scope
Please let me remind you that SSM is a subset of PIM Sparse Mode, so the 
scenario through which a scalability comparison is made in Appendix A is 
certainly "sparse mode" ("sparse", if I may insist, as opposed to 
"dense"). The subset of sparse-mode being looked at in this scenarios 
(SPTs) happens to be used in most non-SSM deployments, which is why this 
scenario certainly is relevant beyond SSM alone.

If you think that you can make a useful contribution to the document by 
looking at a non-ASM case, then please go ahead (e.g. something that 
would result in exposing orders of magnitude significantly different 
that the one we obtained).

> PIM, then you can't say "I'm only considering PE-PE PIM, PE-CE PIM is out of
I am not the one saying that PE-CE PIM is out of scope. The working 
group charter and the drafts on mVPN being worked on in the WG, do not 
cover the PE-CE part. The cases where PE-CE scaling is an issue are 
*not* specific to mVPN, and logically happens to be addressed in the PIM 
Working group.

Following your claim, we would end up saying that "since CE-PE is the 
bottleneck, then we don't need anything special on the PE-PE side, and 
then all options for PE-PE mVPN routing can be thus said to scale 
equally well, and well... let's say that this is true even for 
deployments will no particular CE-PE scaling issue". And we would 
totally miss out the fact that (a) for the PE-PE scaling part we have 
different options with different scaling characteristics impacting 
*real* deployment scenarios and (b) that the CE-PE scaling part can find 
solutions on its own.  *That* would be irrelevant.

> heard of such notions as "distributed computing" or "datagram networks".
I totally agree with you that a total across multiple equipments has not 
an interesting meaning in itself.
For instance, the total amount of time a message is processed, across 
all equipements, is in O(#PEx#PEs-joined-to-(S,G)) has no meaning *in 
itself*. It is not a big or small number intrinsically. It is only when 
you compare "the total with option X" and "the total with option Y" that 
you get something carrying a meaning.

And... this is what we do: *comparisons* between comparable numbers.
I stand that the comparison done is meaningful.

(feel free to suggest a better measure, but it seems to me that the 
*comparison* will lead to very similar results)

> analysis  is simply  irrelevant 
>   
"If,..., if ..."
That makes many "if"s, in your email, for such a strong judgment.

> that you choose.  However, I don't  think the IETF is the proper publication
Oh, big words again...

The analysis in appendix A is not purely theoretical, and your claim on 
the relevance of the analysis does not appear to me as founded. So, 
while I don't think I am in a position to discuss of the appropriateness 
of publication by the IETF of a document that I myself co-edit, let me 
just remind that there has been consensus in this working group to adopt 
this document, including Appendix A, and that the only comments on the 
list on this section where made by you, one week before the end of WG 
last call (nearly seven month after WG adoption).

> But I'm not sure it's worthwhile to repeat all those points again as part of
Except that I extensively replied to you to point some mistakes you 
made, and explain that "join suppression" is actually taken into account 
in the analysis. It would be worthwhile if you were able to point at 
something precise that you think is incorrect (ie. quote and show 
why/how incorrect).