Subject: [Teas] Roman Danyliw's Abstain on draft-ietf-teas-native-ip-scenarios-09: (with COMMENT)

From: Roman Danyliw via Datatracker <[email-masked]>
Date: Wed, 02 Oct 2019 19:37:43 -0700

Roman Danyliw has entered the following ballot position for
draft-ietf-teas-native-ip-scenarios-09: Abstain

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-teas-native-ip-scenarios/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

==[ Summary
I am balloting as an ABSTAIN because the primary contribution of this draft
appears to be the simulation results.  However, these results appear to be
already published in greater detail in
http://ieeexplore.ieee.org/document/8657733.

I have no commentary on the utility of the CDDR Scenarios found in Section 3.

==[ Details

I have reservations about the approach taken in Section 4.  As a stand-alone
write-up, it insufficiently describes the simulations in question. 
Furthermore, the content in this section copies verbatim or re-phrases the
source material of these simulations from an already published academic paper
(without citation).  See http://ieeexplore.ieee.org/document/8657733.  In
particular:

-- The entirely of Section 4.1 (text and diagrams) is a cut-and-paste from
Section V.B of the academic paper.

-- The simulation results of Section 4.4 in this draft align with the analysis
in Section V.C of the academic paper.

-- The simulation results of Section 4.5 in this draft align with analysis in
Section V.D of the academic paper.

Without the benefit of the academic paper cited above, I would have had the
following feedback on Section 4:

-- Section 4.  What use case is being simulated isn’t clear.  The text is
Section 3.1 states that “Section 4 of this document describes the simulation
results of this use case”.  However, the topology for the simulation described
in Section 4.2 looks different than the one in Figure 1 (of Section 3.1).

-- Section 4.  A few questions about the simulation design:
Only the results were presented.  How were the simulations constructed and
results evaluated?

Section 4.0 states that this section “illustrate[s the] CCDR algorithm”.  Where
is that algorithm defined?  Is that Section 7 of draft-ietf-teas-pce-native-ip
or http://ieeexplore.ieee.org/document/8657733?

What specific algorithms was used to create the contrasting charts in Figures 7
and 8?

Section 4.2 said that “[t]he number of links connecting one edge node to the
set of core nodes is randomly between 2 to 30, and the total number of links is
more than 20000.”  Section 4.3 said that “[i]n the CCDR simulation, the
dimension of the traffic matrix is 500*500.  About 20% links are overloaded
when the Open Shortest Path First (OSPF) protocol is used in the network.”,
What is the basis of these choices?  Are they representative of a given use
case?

What is the relationship between the Sections 4.2 and 4.3, and the simulation
results in Sections 4.4 and 4.5

Additional feedback includes:

-- Section 5.  The purpose of this section isn’t clear.

-- Missing references
Section 1.  A reference to CCDR is needed.  Also, given that CCDR is the basis
of the simulation, it needs to be normative.

Section 5.  A reference is needed for the IEEE document.

---

From: Aijun Wang <[email-masked]>
Date: Thu, 3 Oct 2019 12:00:07 +0800

Hi, Roman:
Thanks for your review.
The primary aim of this draft is to illustrate the TE scenarios in Native IP network, which is the base of other two documents:

https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04 (solution draft)

https://tools.ietf.org/html/draft-ietf-pce-pcep-extension-native-ip-04 (PCEP extension)

The simulation results in this draft just want to convince the reader the addition gain with the help of global view and central control, based on the optimization results under the similar topology size and traffic matrix pattern as that in the real network.

The authors of this draft are overlapping with the authors of the reference published paper. We have mentioned this reference in the draft. A normative reference to it will be added in the upcoming updated version.

More responses for your comments are inline. Wish these all can convince you for the future vote of this draft.

Thanks in advance 

Aijun Wang
China Telecom

> On Oct 3, 2019, at 10:37, Roman Danyliw via Datatracker <[email-masked]> wrote:
> 
[Aijun Wang]: please see the above explanation.
> Section V.D of the academic paper.
[Aijun Wang]: The authors of theses two documentaries are overlapping. We just abstract the simulation process and results from the academic paper. A normative reference will be added later.

> in Section 4.2 looks different than the one in Figure 1 (of Section 3.1).

[Aijun Wang] Figure 6 in section 4.2 is just the details expression of the “distributed control network” in Figure 1.
> Only the results were presented.  How were the simulations constructed and
[Aijun Wang]: The topology and traffic matrix generated process are described in section 4.2 and section 4.3 respectively. Based on these two information, the algorithm described in the above mentioned paper is run to get the simulation results.
> or http://ieeexplore.ieee.org/document/8657733?
[Aijun Wang] In the IEEE paper.
> What specific algorithms was used to create the contrasting charts in Figures 7
[Aijun Wang]: These are demonstration of the simulation results. The detailed algorithm is described in the above mentioned paper. The key idea in the algorithm is that the controller has the global view of the underlying network, thus can operate/controlled the network in more efficient way.
> What is the basis of these choices?  Are they representative of a given use

[Aijun Wang] The base of these selections is the real network. We have the same size of the network and the similar traffic matrix patterns in our network.
> results in Sections 4.4 and 4.5

[Aijun Wang]: Section 4.2 and 4.3 are input of the simulation process. They are the base requirements data. Section 4.4 and 4.5 are the simulation results of two key scenarios: E2E QoS assurance and network congestion elimination. The topology and background traffic are provided in section 4.2 and 4.3

> -- Section 5.  The purpose of this section isn’t clear.

[Aijun Wang]: It is the summary of the aim of this draft. 

> of the simulation, it needs to be normative.

[Aijun Wang] Will add in coming version.

> Section 5.  A reference is needed for the IEEE document.

[Aijun Wang] Yes