Subject: Re: [Teas] Benjamin Kaduk's Discuss on draft-ietf-teas-native-ip-scenarios-09: (with DISCUSS and COMMENT)

From: Aijun Wang <[email-masked]>
Date: Thu, 3 Oct 2019 16:10:16 +0800

Hi, Benjamin:
Thanks for your review.

On summary, this draft just gives the scenarios that needed for TE in Native IP network, this is absent in current existing IETF documents. The simulation results just demonstrated the applicability of central control under the global view.
This document is the base document of other two drafts:

https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04 (Solution Draft)

https://tools.ietf.org/html/draft-ietf-pce-pcep-extension-native-ip-04(PCEP extension)

There are currently at least three different solutions trying to accomplish the TE necessities in native IP network. This scenarios and simulation draft is just the start points of these documents.

More details responses are inline below. Wish they can convince you for the future vote.

Thanks in advance.

Aijun Wang
China Telecom

>> On Oct 3, 2019, at 13:15, Benjamin Kaduk via Datatracker <[email-masked]> wrote:
> Benjamin Kaduk has entered the following ballot position for
> draft-ietf-teas-native-ip-scenarios-09: Discuss
> 
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
> 
> 
> Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
> for more information about IESG DISCUSS and COMMENT positions.
> 
> 
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-teas-native-ip-scenarios/
> 
> 
> 
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
> 
> I have two points for discussion:
> 
> (1) If this document was subject to the approval requirements for
> standards action, it would basically be suffering from "death by
> abstain"; this seems like a good signal for the IESG to discuss whether
> it makes sense to approve this document even though the more-lenient
> document-action requirements would otherwise let it go forward.
> 
[Aijun Wang]: This document is the cooperation results of authors from operators, research university and the vendor. The style of the document maybe slight different from the traditional IETF documents, but the aim of it same as others: give some useful information to the community based on our efforts.
Wish the relevant responses and explanations can convince the reviewer.

> (2) The document seems incomplete to me.  It has some aspects of being
> all/any of a use-cases document, an architecture document, and an
> applicability analysis, but does not seem to have a complete treatment
> for any of them.  To be clear, there is enough in the document to
> indicate that the topic merits further work, and there are some
> interesting results, but I'm not sure that publication as an RFC is
> appropriate for this document in this form.  Specifically:
[Aijun Wang] It is mainly one native IP TE scenarios description document. The simulation part of this document just want to convince the reader the addition gain of central control under the global view.
The architecture/solutions contents is in above mentioned solutions draft.

> 
> (2a) use-cases: we see the examples of star topology with BRAS/SR and
> the simulated network in Figure 6, but there is not much discussion of
> where these (or similar) scenarios arise in practice, how common they
> are, and how closely the simulation reflects actual usage.

[Aijun Wang]: The star topology and simulated topology are all abstraction from our real network deployment.

> 
> (2b) architecture: a very high-level picture is given ("use a PCE to
> engineer some of the IP traffic on a network and improve overall
> efficiency"), but we don't see much about how PCCs will be involved and
> apply the computed paths or what requirements will need to be met by the
> protocols and components used to instantiate the architecture

[Aijun Wang]: The above contents that you concerned is described in the above mentioned solution and PCEP extension drafts. As explained in previous, this draft focus only the overall scenarios and the simulation results.

> 
> (2c) applicability: we see some scenarios where the proposed technology
> shows drastic improvement over the alternative selected for comparison,
> but there is little to give confidence that this reflects a broad maxima
> that is robust to environmental variations.  Is the alternative selected
> for comparison an appropriate one for the cases in question?  How would
> the propsal react in the face of changes in the environment it runs in,
> such as link or node failure, changes in the baseline usage, or traffic
> spikes?  What timescale can it react in and what level of visibility
> does the PCE need into current conditions in order to be reliable?
[Aijun Wang] The SDN controller get the overall underlying network conditions via BGP-LS/SNMP/Netflow protocol in about 5 minutes intervals. The optimization process for the simulated complex topology and traffic matrix need only tens of seconds. For further strict requirements, it is easy to enhance the central controller’s capability to tackle the changeable environment.

The above concerns has more relations to the solutions, we can add such descriptions in the solutions draft at its “Deployment Considerations” part.(https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04#section-9)
  
> 
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> If we're using a PCE in a native IP network, how do the computed routes
> get applied; are we using source routing or just being careful about
> the prefixes in use?  (Are there going to be any scaling concerns?)
> 

[Aijun Wang] These contents are in the above mentioned solution draft. It is not using the current source routing, but the combination of multiple BGP session and the manipulation of the path to BGP nexthop with the help of PCEP protocol. The scalability is discussed in section 9.1 of solution draft(https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04#section-9.1)

> Section 3.1
> 
> I don't understand what Figure 1 is intending to convey.  Are "Private
> Cloud Site" and "Public Cloud Site" supposed to be separate boxes on the
> edge of the distributed control network?  Why is the "Cloud Based
> Application" in neither of the named clouds?
> 

[Aijun Wang]: The source and destination of the “Cloud-Based Application” are located at the “Private Cloud Site” and “Public Cloud Site”. You can deem these sites as the concepts described in traditional VPN deployment scenarios.

> Section 3.2
> 
>   Network topology within a Metro Area Network (MAN) is generally in a
>   star mode as illustrated in Figure 2, with different devices
> 
> "Generally" within what scope, commercial ISPs?  I know of things that
> could be called MANs that use a different topology.
> 
[Aijun Wang]: In commercial ISP.

> Section 4.1
> 
> nit: several sentences are missing spaces after the full stop.

[Aijun Wang] Will correct them in update version.
> 
> Section 4.2
> 
> Is a fully-linked core of 100 nodes representative of typical
> deployments?  That's a lot of links not going to customers!

[Aijun Wang]: The core nodes can also connects the customers. That is the reasons that the traffic matrix is 500*500, not 100*100

> 
> Section 4.3
> 
>   The traffic matrix is generated based on the link capacity of
>   topology.  [...]
> 
> I don't know how to interpret this statement.
> It does sound like the traffic matrix is generated in a somewhat
> arbitrary fashion, with no stated effort to keep it aligned with
> real-world traffic patterns.
> 
[Aijun Wang] We just keep the overall congestion links ratio is similar with the real network. The arbitration fashion can otherwise certificate the robustness of the simulation process.  

> 
> _______________________________________________
> Teas mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/teas

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 3 Oct 2019 10:39:06 +0200

All,

I think it is useful informational document.

However I would remove PCE elements from it and turned it into requirements
document perhaps also adding few more real life deployment scenarios where
use of end to end IP TE is very important.

Many thx,
R.

On Thu, Oct 3, 2019 at 10:10 AM Aijun Wang <[email-masked]>
wrote:

> 
> Hi, Benjamin:
> Thanks for your review.
>
> On summary, this draft just gives the scenarios that needed for TE in
> Native IP network, this is absent in current existing IETF documents. The
> simulation results just demonstrated the applicability of central control
> under the global view.
> https://tools.ietf.org/html/draft-ietf-pce-pcep-extension-native-ip-04
> <https://tools..ietf.org/html/draft-ietf-pce-pcep-extension-native-ip-04>(PCEP
> extension)
>
> There are currently at least three different solutions trying to
> accomplish the TE necessities in native IP network. This scenarios and
> simulation draft is just the start points of these documents.
>
> [Aijun Wang]: This document is the cooperation results of authors from
> operators, research university and the vendor. The style of the document
> maybe slight different from the traditional IETF documents, but the aim of
> it same as others: give some useful information to the community based on
> our efforts.
> [Aijun Wang] It is mainly one native IP TE scenarios description document.
> The simulation part of this document just want to convince the reader the
> addition gain of central control under the global view.
> [Aijun Wang]: The above contents that you concerned is described in the
> above mentioned solution and PCEP extension drafts. As explained in
> previous, this draft focus only the overall scenarios and the simulation
> results.
>
>
> [Aijun Wang] The SDN controller get the overall underlying network
> conditions via BGP-LS/SNMP/Netflow protocol in about 5 minutes intervals.
> The optimization process for the simulated complex topology and traffic
> matrix need only tens of seconds. For further strict requirements, it is
> easy to enhance the central controller’s capability to tackle the
> changeable environment.
>
> The above concerns has more relations to the solutions, we can add such
> descriptions in the solutions draft at its “Deployment Considerations”
> part.(
> https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04#section-9)
>
>
>
> [Aijun Wang] These contents are in the above mentioned solution draft. It
> is not using the current source routing, but the combination of multiple
> BGP session and the manipulation of the path to BGP nexthop with the help
> of PCEP protocol. The scalability is discussed in section 9.1 of solution
> draft(
> https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04#section-9.1)
>
>
>
> Section 3.1
>
> [Aijun Wang]: The source and destination of the “Cloud-Based Application”
> are located at the “Private Cloud Site” and “Public Cloud Site”. You can
> deem these sites as the concepts described in traditional VPN deployment
> scenarios.
>
> Section 3.2
>
> could be called MANs that use a different topology..
>
> Section 4..1
> [Aijun Wang]: The core nodes can also connects the customers. That is the
> reasons that the traffic matrix is 500*500, not 100*100
>
>
> Section 4.3
>
> [Aijun Wang] We just keep the overall congestion links ratio is similar
> with the real network. The arbitration fashion can otherwise certificate
> the robustness of the simulation process.
>
>

---

From: Aijun Wang <[email-masked]>
Date: Thu, 3 Oct 2019 17:43:03 +0800

Hi, Robert:

Thanks for your comments.
With the pervasive deployment of overlay technologies within the network, the necessity of IP TE solutions will be more attractive than the traditional MPLS TE solutions.
Current draft illustrates the intra-domain and inter-domain TE scenarios. The solutions that can meet these scenarios/requirements  in one unique approach will be more easily deployed in the network.

The PCE is just the representative of SDN controller. We can mitigate the descriptions of it. But without the help of SDN controller and the global view of underlying network, it is almost impossible to achieve the E2E QoS assurance and improve the network efficiency.

Would you like to provide some text for what you want to solve but not covered by this document?

Thanks in advance.

Aijun Wang
China Telecom

> On Oct 3, 2019, at 16:39, Robert Raszuk <[email-masked]> wrote:
> 
> 
> All,
> 
> However I would remove PCE elements from it and turned it into requirements document perhaps also adding few more real life deployment scenarios where use of end to end IP TE is very important.
> 
> Many thx,
> R.
> 
> 
> 
> 
> 
>> On Thu, Oct 3, 2019 at 10:10 AM Aijun Wang <[email-masked]> wrote:
>> 
>> Hi, Benjamin:
>> Thanks for your review.
>>

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 3 Oct 2019 12:05:43 +0200

Hi Aijun.

> With the pervasive deployment of overlay technologies within the network,
the necessity
> of IP TE solutions will be more attractive than the traditional MPLS TE
solutions.

I fully agree. That is why I wrote my draft proposing how IP TE can be
easily constructed and deployed today.

> The PCE is just the representative of SDN controller. We can mitigate the
descriptions of it. But without
> the help of SDN controller and the global view of underlying network, it
is almost impossible to achieve
> the E2E QoS assurance and improve the network efficiency.

I actually disagree with this above statement.

Central control makes a lot of sense within single administrative domain
where the advantage you have is ability to keep real time traffic matrix or
traffic demand and then when you can control all ingress points and manage
100% of the transit traffic according to the plan.

While this is possible in intra-domain cases it is pretty seldom used. For
inter-domain traffic IMO this is impossible simply as you can not either
get the full traffic matrix across N number of underlay ASes nor control
how that traffic flows.

I presented in my draft how I envision to achieve IP TE based on real time
SLA measurements which btw is the default way  of operation in number of
SDWANs today. Some do it better then others but the core idea is still the
same.

Thx a lot,
R.

On Thu, Oct 3, 2019 at 11:43 AM Aijun Wang <[email-masked]>
wrote:

> With the pervasive deployment of overlay technologies within the network,
> the necessity of IP TE solutions will be more attractive than the
> traditional MPLS TE solutions.
> Current draft illustrates the intra-domain and inter-domain TE scenarios.
> The solutions that can meet these scenarios/requirements  in one unique
> approach will be more easily deployed in the network.
>
> The PCE is just the representative of SDN controller. We can mitigate the
> descriptions of it. But without the help of SDN controller and the global
> view of underlying network, it is almost impossible to achieve the E2E QoS
> assurance and improve the network efficiency.
>
> However I would remove PCE elements from it and turned it into
> requirements document perhaps also adding few more real life deployment
> scenarios where use of end to end IP TE is very important.
>
> Many thx,
> R.
>
>
>
>
>
>> [Aijun Wang] It is mainly one native IP TE scenarios description
>> document. The simulation part of this document just want to convince the
>> reader the addition gain of central control under the global view.
>> [Aijun Wang]: The star topology and simulated topology are all
>> abstraction from our real network deployment.
>>
>>

---

From: Aijun Wang <[email-masked]>
Date: Thu, 3 Oct 2019 18:24:45 +0800

Hi, Robert,
Thanks for your comments.
Please see the inline responses below.

Aijun Wang
China Telecom

> On Oct 3, 2019, at 18:06, Robert Raszuk <[email-masked]> wrote:
> 
> Hi Aijun.
> 
[Aijun Wang] I agree.

> get the full traffic matrix across N number of underlay ASes nor control
[Aijun Wang] There are also the inter-domain that under one administration scope can achieve the expected results.

> SDWANs today. Some do it better then others but the core idea is still the

[Aijun Wang] Is there any SDWAN deployment that operated by different operators? As I known, current SDWAN solutions are mainly proprietary protocol.
The vision for E2E QoS assurance that covers different operators is attractive but also challenging. To transmit the SLA measurements via the central control unit and then deploy them in the network respectively is more feasible.

We can discuss such scenarios further in details.

>>> conditions via BGP-LS/SNMP/Netflow protocol in about 5 minutes intervals

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 3 Oct 2019 14:04:53 -0700

Hi Aijun,

> There are currently at least three different solutions trying to accomplish the TE necessities in native IP network. This scenarios and simulation draft is just the start points of these documents.

Okay, so it sounds like the goal of the document is to show that there are
some scenarios/use-cases in which Native IP provides a substantial
improvement from the previous state of the art (or, perhaps, from the other
competing options), and the simulations are just a tool used to augment the
qualitative description with some concrete instantiations.  This does not
match fully with the current Abstract/Introduction, the former of which I
quote from:

                                                       This document
   describes various complex scenarios and simulation results when
   applying a PCE in a native IP network.  This solution, referred to as
   Centralized Control Dynamic Routing (CCDR), integrates the advantage
   of using distributed protocols and the power of a centralized control
   technology.

A pithy summary of that exerpt might be "describes some routing scenarios
and simulation results for CCDR; CCDR combines advantages of distributed
protocols and centralized control".  While those statements seem to be
true, it doesn't say much about the presented scenarios and results
implying that CCDR is the best or even a good solution for the described
scenarios; it's just two facts that don't get tied together.

> More details responses are inline below. Wish they can convince you for the future vote.

I'm pretty sure that we'll need to update the text for me to change my
ballot position, but I also don't forsee significant challenges in doing
so.

> [Aijun Wang]: The star topology and simulated topology are all abstraction from our real network deployment.

That's what I expected, but it can be helpful to state this explicitly.
(On the other hand, this is probably textbook stuff, so minimal coverage is
okay.)

Since we're focusing on the "use-cases" portion, to use my terminology,
then the main thing to focus on would be to convince the reader that the
simulation results are likely to translate well to real-world usage.
Mentioning that the presented results hold across many different (random)
simulations, as Éric suggest, would help with this, especially if some
sense of the nature of the distribution across trials can be included (such
as standard deviations).  Drawing the connection between the real-world
topology and model topology (as mentioned above) also helps with this.

As a side note, I do wonder how the simulation results would look if there
was a "spatial affinity" for the attachments of edge nodes to the core, in
that there is a correlation between which subset of core nodes are attached
to individual edge nodes.  I expect the impact to be minor, since the core
is fully connected in the simulated topology, but can't guarantee it.  But
this is a side note since I don't think running such new simulations would
be necessary in order to get the document into a state where it can be
published.

> [Aijun Wang]: The above contents that you concerned is described in the above mentioned solution and PCEP extension drafts. As explained in previous, this draft focus only the overall scenarios and…

Sure, and that's a fine way to split up the content -- we don't necessarily
need to include the architecture of the solution in this document.

> The above concerns has more relations to the solutions, we can add such descriptions in the solutions draft at its “Deployment Considerations” part.(https://tools.ietf.org/html/draft-ietf-teas-pce-…

I agree that these concerns relate to specific solutions and are best
addressed alongside the solutions draft(s); we don't necessarily need to
have this draft cover this topic.

> [Aijun Wang] These contents are in the above mentioned solution draft. It is not using the current source routing, but the combination of multiple BGP session and the manipulation of the path to BG…

Okay, thanks.

> [Aijun Wang]: The source and destination of the “Cloud-Based Application” are located at the “Private Cloud Site” and “Public Cloud Site”. You can deem these sites as the concepts described in trad…

So the "Cloud-Based Application" has components in both Cloud sites, but
excludes the PCE and distributed control network?  Perhaps extending the
box for it to include parts at both sites, or having smaller boxes at both
sites and a line grouping them together, would be more clear.

> [Aijun Wang]: In commercial ISP.

Okay.  This was implied by the previous discussion of "service provider
network", but I'd suggest being explicit about it here.

> [Aijun Wang] We just keep the overall congestion links ratio is similar with the real network. The arbitration fashion can otherwise certificate the robustness of the simulation process.  

If I understand correctly, there's three important variables for the
simulation: baseline load, the congesion threshold for each link,  and the
distribution of flow bandwidth for the simulated flows.  I don't know if
the simulation is being run with zero baseline load so that all flows are
centrally managed or not, so please say something about that.  For the
other two, sould it be accurate to say something like "the congestion
threshold for the edge/core and intra-core links are set to the fixed value
of 90%" or "the congestion threshold for edge/core and intra-core links are
sampled from a distribution representative of observed real-world
environments", and "the bandwidth requirements of generated flows is from a
distribution representative of observed real-world flow data"?

In short, I think we need to say enough about what was done that someone
else could independently run if not the same experiment, one sufficiently
similar that we expect the results to also be similar.

Thanks,

Ben

---

From: Aijun Wang <[email-masked]>
Date: Fri, 4 Oct 2019 07:21:38 +0800

Hi, Benjamin:
The pithy summary that you proposed is right. The relationship between these two parts is that the simulation parts(4.4 and 4.5) of the document just demonstrated how to solve efficiently the TE scenarios/req  described in 3.1 and 3.4.
The scenarios described in 3.2 and 3.3 can also be solved using the similar strategy but not covered by the current simulation process.
What summary sentences will be appropriate for the above information then? Would you like to give us some hints from the experts viewpoint? 
We can update the abstract in next version to reflect this relationship, together with other responses to you and reviewers’s comments.

Thanks in advance.

Aijun Wang
China Telecom

> On Oct 4, 2019, at 05:05, Benjamin Kaduk <[email-masked]> wrote:
> 
> Hi Aijun,
>

---

From: Benjamin Kaduk <[email-masked]>
Date: Thu, 24 Oct 2019 17:18:59 -0700

Hi Aijun,

I went through the document and attempted to add some more context about
what use-cases/simulations are being presented, analysis of why the
simulation results are expected to be indicative of real-world results, and
to present the clear conclusion that CCDR is expected to provide a generic
solution to the indicated class of problems/scenarios.  (I also made some
editorial/grammar fixes while I was in there.)  However, I am not the
expert on this work, and I probably made some errors along the way -- just
because I say something is true doesn't mean it's true!  Please take a look
at the attached diff to the XML sources and consider which parts of it will
help improve the document.

I look forward to seeing your take on a revised I-D.

Thanks,

Ben

> similar that we expect the results to also be similar.
--- draft-ietf-teas-native-ip-scenarios-10.xml	2019-10-24 15:03:06.193789000 -0700
+++ draft-ietf-teas-native-ip-scenarios-bjk.xml	2019-10-24 17:12:45.500913000 -0700
@@ -14,6 +14,7 @@
 <?rfc inline="yes"?>
 <?rfc compact="yes"?>
 <?rfc subcompact="no"?>
+<!-- version -10 with added modifications by Benjamin Kaduk -->
 <rfc category="info" docName="draft-ietf-teas-native-ip-scenarios-10"
      ipr="trust200902">
   <front>
@@ -153,11 +154,11 @@
 
     <abstract>
       <t>Requirements for providing the End to End(E2E) performance assurance
-      are emerging within the service provider network. While there are
-      various technology solutions, there is no one solution which can fulfill
-      these requirements for a native IP network. One universal (E2E) solution
-      which can cover both intra-domain and inter-domain scenarios is
-      needed.</t>
+      are emerging within service provider networks. While there are
+      various technology solutions, there is no single solution that can fulfill
+      these requirements for a native IP network.  In particular, there is a need
+      for a universal (E2E) solution that is simultaneously applicable for both
+      intra- and inter-domain scenarios.</t>
 
       <t>One feasible E2E traffic engineering solution is the addition of
       central control in a native IP network. This document describes various
@@ -165,7 +166,9 @@
       Computation Element (PCE) in a native IP network. This solution,
       referred to as Centralized Control Dynamic Routing (CCDR), integrates
       the advantage of using distributed protocols and the power of a
-      centralized control technology.</t>
+      centralized control technology, providing traffic engineering for
+      native IP networks in a manner that applies equally to intra- and
+      inter-domain scenarios.</t>
     </abstract>
   </front>
 
@@ -182,7 +185,7 @@
 
       <t>Multiprotocol Label Switching (MPLS) using Traffic Engineering (TE)
       technology (MPLS-TE)<xref target="RFC3209"/>is one solution for traffic
-      engineering network but it introduces an MPLS network and related
+      engineering networks but it introduces an MPLS network and related
       technology which would be an overlay of the IP network. MPLS-TE
       technology is often used for Label Switched Path (LSP) protection and
       complex path set-up within a domain.</t>
@@ -195,7 +198,7 @@
       integrates some advantages of using a distributed protocol and a
       centrally control technology, but it requires the underlying network,
       especially the provider edge router, to do a label push and pop action
-      in-depth, and adds complexity, when coexisting with the Non-Segment
+      in-depth, and adds complexity when coexisting with the Non-Segment
       Routing network. Additionally, it can only maneuver the E2E paths for
       MPLS and IPv6 traffic via different mechanisms.</t>
 
@@ -206,11 +209,17 @@
       defined in this document do not require the additional complexity of
       deterministic properties and so differ from the DetNet use cases.</t>
 
-      <t>This draft describes scenarios for a native IP network that a
-      Centralized Control Dynamic Routing (CCDR) framework can easily solve,
-      without requiring a change of the data plane behavior on the router. It
-      also provides path optimization simulation results to illustrate the
-      applicability of the CCDR framework. </t>
+      <t>This draft describes several scenarios for a native IP network where a
+      Centralized Control Dynamic Routing (CCDR) framework can produce
+      qualitative improvement in efficiency without requiring a change of
+      the data-plane behavior on the router.  Using knowledge of
+      interface-specific prefixes advertised by a router, a central PCE is
+      able to compute a path and give the sender a destination address to
+      use, such that the distributed routing protocol will naturally use the
+      computed path.  Some results from simultations of path optimization
+      are also presented, to concretely illustrate a variety of scenarios
+      where CCDR shows significant improvement over traditional distributed
+      routing protocols.</t>
 
       <t>This draft is the base document of the following two drafts: the
       universal solution draft, which is suitable for intra-domain and
@@ -254,8 +263,10 @@
     </section>
 
     <section title="CCDR Scenarios">
-      <t>The following sections describe various deployment scenarios for
-      applying the CCDR framework.</t>
+      <t>The following sections describe various deployment scenarios where
+      applying the CCDR framework is intuitively expected to produce
+      improvements, based on the macro-scale properties of the framework and
+      the scenario.</t>
 
       <section title="QoS Assurance for Hybrid Cloud-based Application">
         <t>With the emergence of cloud computing technologies, enterprises are
@@ -313,11 +324,11 @@
         connected to different customer types. The traffic from these
         customers is often in a tidal pattern, with the links between the Core
         Router(CR)/Broadband Remote Access Server(BRAS) and CR/Service
-        Router(SR), experiencing congestion in different periods, because the
-        subscribers under BRAS, often use the network at night, and the
-        dedicated line users under SR, often use the network during the
+        Router(SR) experiencing congestion in different periods, because the
+        subscribers under BRAS often use the network at night, and the
+        dedicated line users under SR often use the network during the
         daytime. The link between BRAS/SR and CR must satisfy the maximum
-        traffic volume between them respectively and this causes these links
+        traffic volume between them, respectively, and this causes these links
         often to be under-utilized.</t>
 
         <figure>
@@ -337,8 +348,8 @@
         <t>If we consider connecting the BRAS/SR with a local link loop (which
         is usually lower cost), and control the overall MAN topology with the
         CCDR framework, we can exploit the tidal phenomena between the BRAS/CR
-        and SR/CR links, maximizing the utilization of these links (which are
-        usually higher cost).</t>
+	and SR/CR links, maximizing the utilization of these central trunk
+	links (which are usually higher cost than the local loops).</t>
 
         <t><figure>
             <artwork><![CDATA[                                    +-------+
@@ -384,7 +395,7 @@
 
         <t>A solution for this scenario requires the gathering of NetFlow
         information, analysis of the source/destination AS, and determining
-        what is the main cause of the congested link. After this, the operator
+        what is the main cause of the congested link(s). After this, the operator
         can use the external Border Gateway Protocol(eBGP) sessions to
         schedule the traffic among the different domains according to the
         solution described in CCDR framework.</t>
@@ -392,8 +403,10 @@
 
       <section title="Network Temporal Congestion Elimination">
         <t>In more general situations, there are often temporal congestion
-        within the service provider&rsquo;s network. Such congestion phenomena
-        often appear repeatedly, and if the service provider has methods to
+        within the service provider&rsquo;s network, for example due to
+	daily or weekly periodic bursts, or large events that are scheduled
+	well in advance. Such congestion phenomena
+        often appear regularly, and if the service provider has methods to
         mitigate it, it will certainly improve their network operations
         capabilities and increase satisfaction for their customers. CCDR is
         also suitable for such scenarios, as the controller can schedule
@@ -404,19 +417,27 @@
     </section>
 
     <section anchor="Section-4" title="CCDR Simulation">
-      <t>The following sections describe one case study to illustrate CCDR
-      algorithm, the topology and traffic matrix generation process and the
-      optimization results for E2E QoS assured path and congestion elimination
-      in applied scenarios.</t>
+      <t>The following sections describe a specific case study to illustrate
+      the workings of the CCDR algorithm with concrete paths/metrics, as
+      well as a procedure for generating topology and traffic matrices and
+      the results from simulations applying CCDR for E2E QoS (assured path
+      and congestion elimination) over the generated topologies and traffic
+      matrices. In all cases examined, the CCDR algorithm produces
+      qualitatively significant improvement over the reference (OSPF)
+      algorithm, suggesting that CCDR will have broad applicability.</t>
 
-      <t>The structure and scale of the simulated topology is similar with the
-      real network. Several amounts of traffic matrix are generated to
-      simulate the different congestion condition in network, only one of them
-      is illustrated.</t>
+      <t>The structure and scale of the simulated topology is similar to
+      that of
+      real networks. Multiple different traffic matrices were generated to
+      simulate different congestion conditions in the network, but only one
+      of them is illustrated since the others produce similar results.</t>
 
       <section title="Case Study for CCDR algorithm">
-        <t>Figure 5 depicts the topology of the network for the case study of
-        CCDR algorithm. There are 8 forwarding devices in the network. The
+        <t>In this section we consider a specific network topology for a
+	worked case study, examining the path selected by OSPF and CCDR and
+	evaulating how and why the paths differ.  Figure 5 depicts the
+	topology of the network in question. There are 8
+	forwarding devices in the network. The
         original cost and utilization are marked on it, as shown in the
         figure. For example, the original cost and utilization for the link
         (1,2) are 3 and 50% respectively. There are two flows: f1 and f2. Both
@@ -424,13 +445,15 @@
         assumed that the bandwidth of the link in the network is 10Mb/s. The
         flow rate of f1 is 1Mb/s, and the flow rate of f2 is 2Mb/s. The
         threshold of the link in congestion is 90%.</t>
+<!-- Given that Figure 8 is showing CD percentages over 20, do we need to
+have the congestion threshold here be more like 75% or 80% ? -kaduk -->
 
         <t>If OSPF protocol (ISIS is similar, because it also use the
         Dijstra's algorithm) is applied in the network, which adopts
         Dijkstra's algorithm, the two flows from node 1 to node 8 can only use
         the OSPF path (p1: 1-&gt;2-&gt;3-&gt;8). It is because Dijkstra's
         algorithm mainly considers original cost of the link. Since CCDR
-        considers cost and utilization simultaneously, the same path with OSPF
+        considers cost and utilization simultaneously, the same path as OSPF
         will not be selected due to the severe congestion of the link (2,3).
         In this case, f1 will select the path (p2: 1-&gt;5-&gt;6-&gt;7-&gt;8)
         since the new cost of this path is better than that of OSPF path.
@@ -493,12 +516,15 @@
       </section>
 
       <section title="Topology Simulation">
-        <t>The network topology mainly contains nodes and links information.
-        Nodes used in the simulation have two types: core node and edge node.
-        The core nodes are fully linked to each other. The edge nodes are
-        connected only with some of the core nodes. Figure 6 is a topology
-        example of 4 core nodes and 5 edge nodes. In this CCDR simulation, 100
-        core nodes and 400 edge nodes are generated.</t>
+	<t>Moving on from the specific case study, we now consider a class
+	of networks more representative of real deployments, with a
+	fully-linked core network that serves to connect edge nodes, which themselves
+	connect to only a subset of the core.  An example of such a topology
+	is shown in Figure 6, for the case of 4 core nodes and 5 edge nodes.
+	The CCDR simulations presented in this work use topologies involving
+	100 core nodes and 400 edge nodes.  While the resulting graph does
+	not fit on this page, this scale of network is similar to what is
+	deployed in production environments.</t>
 
         <t><figure>
             <artwork><![CDATA[                                  +----+
@@ -524,26 +550,31 @@
 
                      Figure 6: Topology of Simulation
 ]]></artwork>
-          </figure>The number of links connecting one edge node to the set of
-        core nodes is randomly between 2 to 30, and the total number of links
-        is more than 20000. Each link has a congestion threshold.</t>
+	  </figure>For the simulations, the number of links connecting one
+	    edge node to the set of core nodes is randomly chosen between 2 to
+	    30, and the total number of links is more than 20000. Each link
+	    has a congestion threshold, which can be arbitrarily set to
+	    (e.g.) 90% of the nominal link capacity without affecting the
+	    simulation results.</t>
       </section>
 
       <section title="Traffic Matrix Simulation">
-        <t>The traffic matrix is generated based on the link capacity of
+        <t>For each topology, a traffic matrix is generated based on the
+	link capacities of
         topology. It can result in many kinds of situations, such as
         congestion, mild congestion and non-congestion.</t>
 
         <t>In the CCDR simulation, the dimension of the traffic matrix is
-        500*500. About 20% links are overloaded when the Open Shortest Path
-        First (OSPF) protocol is used in the network.</t>
+        500*500 (100 core nodes plus 400 edge nodes). About 20% of links are overloaded when the Open Shortest Path
+        First (OSPF) protocol is used in the network, for the scenarios bein
+	simulated.</t>
       </section>
 
       <section anchor="Section-4.4" title="CCDR End-to-End Path Optimization">
         <t>The CCDR E2E path optimization is to find the best path which is
-        the lowest in metric value and each link of the path is far below
-        link&rsquo;s threshold. Based on the current state of the network, the
-        PCE within CCDR framework combines the shortest path algorithm with a
+        the lowest in metric value and for which each link of the path is far below
+        link&rsquo;s congestion threshold. Based on the current state of the network, the
+        PCE within the CCDR framework combines the shortest path algorithm with a
         penalty theory of classical optimization and graph theory.</t>
 
         <t>Given a background traffic matrix, which is unscheduled, when a set
@@ -560,6 +591,14 @@
         eye-catching decrease in UID relative to the path chosen based on
         OSPF.</t>
 
+	<t>While real-world results invariably differ from simulations (for
+	example, real-world topologies are likely to exhibit correlation in
+	the attachment patterns for edge nodes to the core, which are not
+	reflected in these results), the dramatic nature of the improvement
+	in UID and the choice of simulated toplogy to resemble real-world
+	conditions suggests that real-world deployments will also experience
+	significant improvement in UID results.</t>
+
         <t><figure>
             <artwork><![CDATA[         +-----------------------------------------------------------+
          |                *                               *    *    *|
@@ -601,16 +640,17 @@
 
       <section anchor="Section-4.5"
                title="Network Temporal Congestion Elimination">
-        <t>Different degrees of network congestion were simulated. The
-        Congestion Degree (CD) is defined as the link utilization beyond its
-        threshold.</t>
+        <t>During the simulations, different degrees of network congestion
+	were considered. To examine the effect of CCDR on link congestion,
+	we consider the Congestion Degree (CD) of a link, defined as the
+	link utilization beyond its threshold.</t>
 
         <t>The CCDR congestion elimination performance is shown in Figure 8.
         The first graph is the CD distribution before the process of
         congestion elimination. The average CD of all congested links is about
         20%. The second graph shown in Figure 8 is the CD distribution after
-        using the congestion elimination process. It shows only 12 links among
-        totally 20000 links exceed the threshold, and all the CD values are
+        using the congestion elimination process. It shows that only 12 links among
+        the total of 20000 links exceed the threshold, and all the CD values are
         less than 3%. Thus, after scheduling of the traffic away from the
         congested paths, the degree of network congestion is greatly
         eliminated and the network utilization is in balance.</t>
@@ -656,31 +696,45 @@
 ]]></artwork>
           </figure></t>
 
-        <t>More detailed information about the algorithm can refer to <xref
+        <t>It is clear that using an active path-computation mechanism that
+	is able to take into account observed link traffic/congestion, the
+	occurrence of congestion events can be greatly reduced.  Only when a
+	preponderance of links in the network are near their congestion
+	threshold will the central controller be unable to find a clear
+	path, as opposed to when a static metric-based procedure is used,
+	which will produce congested paths once a single bottleneck
+	approaches its capacity.
+	More detailed information about the algorithm can be found in <xref
         target="PTCS"/> .</t>
       </section>
     </section>
 
     <section title="CCDR Deployment Consideration">
-      <t>Above CCDR scenarios and simulation results demonstrate that it is
-      feasible to find one general solution to cope with various complex
-      situations. Integrated use of a centralized controller for the more
-      complex optimal path computations in a native IP network results in
+      <t>The above CCDR scenarios and simulation results demonstrate that
+      a single general solution can be found that copes with multiple complex
+      situations. The specific situations considered are not known to have
+      any special properties, so it is expected that the benefits
+      demonstrated will have general applicability. Accordingly, the
+      integrated use of a centralized controller for the more complex
+      optimal path computations in a native IP network should result in
       significant improvements without impacting the underlay network
       infrastructure.</t>
 
-      <t>For intra-domain or inter-domain native IP TE scenario, the
-      deployment of CCDR solution is similar. This universal deployment
-      characteristic can facilitate the operator to tackle their traffic
-      engineering issues in one general manner. To deploy the CCDR solution,
+      <t>For intra-domain or inter-domain native IP TE scenarios, the
+      deployment of a CCDR solution is similar, with the centralized
+      controller being able to compute paths and no changes required to the
+      underlying network infrastructure. This universal deployment
+      characteristic can facilitate a generic traffic engineering solution,
+      where operators do not need to differentiate between intra-domain and
+      inter-domain TE cases.  To deploy the CCDR solution,
       the PCE should collect the underlay network topology dynamically, for
       example via BGP-LS<xref target="RFC7752"/>. It also needs to gather the
       network traffic information periodically from the network management
-      platform. The simulation results show PCE can compute the E2E optimal
-      path within seconds thus it can cope with the change of underlay network
-      in minute scale. More agile requirements needs increase the sample rate
-      of underlay network, also decrease the detection and notification
-      interval of underlay network. The methods to gather and decrease the
+      platform. The simulation results show that the PCE can compute the E2E optimal
+      path within seconds, thus it can cope with the change of underlay network
+      on the scale of minutes. More agile requirements would need to increase the sample rate
+      of underlay network and decrease the detection and notification
+      interval of the underlay network. The methods to gather and decrease the
       latency of these information are out of the scope of this draft.</t>
     </section>

---

From: "Aijun Wang" <[email-masked]>
Date: Fri, 25 Oct 2019 16:39:28 +0800

Hi, Benjamin:

Thanks for your careful review and valuable suggestions. Based on your suggestions, I think the document will be looked more coherent and be easily understood. 
I accepted almost all of your suggestions excepts the following parts:
[Original]
This draft describes several scenarios for a native IP network where a Centralized Control Dynamic Routing (CCDR) framework can produce qualitative improvement in efficiency without requiring a change of the data-plane behavior on the router. Using knowledge of interface-specific prefixes advertised by a router, a central PCE is able to compute a path and give the sender a destination address to use, such that the distributed routing protocol will naturally use the computed path.  Some results from simultations of path optimization are also presented, to concretely illustrate a variety of scenarios where CCDR shows significant improvement over traditional distributed routing protocols.

[New]
This draft describes several scenarios for a native IP network where a Centralized Control Dynamic Routing (CCDR) framework can produce qualitative improvement in efficiency without requiring a change of the data-plane behavior on the router.  Using knowledge of BGP(Border Gateway Protocol) session-specific prefixes advertised by a router, the network topology and the near real time link utilization information from network management systems, a central PCE is able to compute an optimal path and give the underlay routers the destination address to use to reach the BGP nexthop, such that the distributed routing protocol will use the computed path via traditional recursive lookup procedure. Some results from simulations of path optimization are also presented, to concretely illustrate a variety of scenarios where CCDR shows significant improvement over traditional distributed routing protocols.

I have uploaded the updated version at https://tools.ietf.org/html/draft-ietf-teas-native-ip-scenarios-11, please take a look at it again and see whether it is suitable to be forward then.

Thanks again for your efforts.

Best Regards.

Aijun Wang
China Telecom

-----邮件原件-----
发件人: Benjamin Kaduk [mailto:[email-masked]] 
发送时间: 2019年10月25日 8:19
收件人: Aijun Wang
抄送: The IESG; [email-masked]; [email-masked]; [email-masked]; [email-masked]
主题: Re: [Teas] Benjamin Kaduk's Discuss on draft-ietf-teas-native-ip-scenarios-09: (with DISCUSS and COMMENT)

Hi Aijun,

I went through the document and attempted to add some more context about what use-cases/simulations are being presented, analysis of why the simulation results are expected to be indicative of real-world results, and to present the clear conclusion that CCDR is expected to provide a generic solution to the indicated class of problems/scenarios.  (I also made some editorial/grammar fixes while I was in there.)  However, I am not the expert on this work, and I probably made some errors along the way -- just because I say something is true doesn't mean it's true!  Please take a look at the attached diff to the XML sources and consider which parts of it will help improve the document.

Thanks,

Ben

On Thu, Oct 03, 2019 at 02:05:00PM -0700, Benjamin Kaduk wrote:
> Okay, so it sounds like the goal of the document is to show that there 
> are some scenarios/use-cases in which Native IP provides a substantial 
> improvement from the previous state of the art (or, perhaps, from the 
> other competing options), and the simulations are just a tool used to 
> augment the qualitative description with some concrete instantiations.  
> This does not match fully with the current Abstract/Introduction, the 
> former of which I quote from:
> 
>                                                        This document
> A pithy summary of that exerpt might be "describes some routing 
> scenarios and simulation results for CCDR; CCDR combines advantages of 
> distributed protocols and centralized control".  While those 
> statements seem to be true, it doesn't say much about the presented 
> scenarios and results implying that CCDR is the best or even a good 
> solution for the described scenarios; it's just two facts that don't get tied together.
> 
> ballot position, but I also don't forsee significant challenges in 
> doing so.
> 
> > Thanks in advance.
> > 
> > Aijun Wang
> > China Telecom
> > 
> > > When responding, please keep the subject line intact and reply to 
> > > all email addresses included in the To and CC lines. (Feel free to 
> > > cut this introductory paragraph, however.)
> > > 
> > > 
> > > Please refer to 
> > > https://www.ietf.org/iesg/statement/discuss-criteria.html
> > > abstain"; this seems like a good signal for the IESG to discuss 
> > > whether it makes sense to approve this document even though the 
> > > more-lenient document-action requirements would otherwise let it go forward.
> > > 
> > > (2) The document seems incomplete to me.  It has some aspects of 
> > > being all/any of a use-cases document, an architecture document, 
> > > and an applicability analysis, but does not seem to have a 
> > > complete treatment for any of them.  To be clear, there is enough 
> > > in the document to indicate that the topic merits further work, 
> > > and there are some interesting results, but I'm not sure that 
> > > publication as an RFC is appropriate for this document in this form.  Specifically:
> > > (2a) use-cases: we see the examples of star topology with BRAS/SR 
> > > and the simulated network in Figure 6, but there is not much 
> > > discussion of where these (or similar) scenarios arise in 
> > > practice, how common they are, and how closely the simulation reflects actual usage.
> > 
> Since we're focusing on the "use-cases" portion, to use my 
> terminology, then the main thing to focus on would be to convince the 
> reader that the simulation results are likely to translate well to real-world usage.
> Mentioning that the presented results hold across many different 
> (random) simulations, as Éric suggest, would help with this, 
> especially if some sense of the nature of the distribution across 
> trials can be included (such as standard deviations).  Drawing the 
> connection between the real-world topology and model topology (as mentioned above) also helps with this.
> 
> As a side note, I do wonder how the simulation results would look if 
> there was a "spatial affinity" for the attachments of edge nodes to 
> the core, in that there is a correlation between which subset of core 
> nodes are attached to individual edge nodes.  I expect the impact to 
> be minor, since the core is fully connected in the simulated topology, 
> but can't guarantee it.  But this is a side note since I don't think 
> running such new simulations would be necessary in order to get the 
> document into a state where it can be published.
> 
> > > 
> > > (2b) architecture: a very high-level picture is given ("use a PCE 
> > > to engineer some of the IP traffic on a network and improve 
> > > overall efficiency"), but we don't see much about how PCCs will be 
> > > involved and apply the computed paths or what requirements will 
> > > need to be met by the protocols and components used to instantiate 
> > > the architecture
> > 
> Sure, and that's a fine way to split up the content -- we don't 
> necessarily need to include the architecture of the solution in this document.
> 
> > > 
> > > (2c) applicability: we see some scenarios where the proposed 
> > > technology shows drastic improvement over the alternative selected 
> > > for comparison, but there is little to give confidence that this 
> > > reflects a broad maxima that is robust to environmental 
> > > variations.  Is the alternative selected for comparison an 
> > > appropriate one for the cases in question?  How would the propsal 
> > > react in the face of changes in the environment it runs in, such 
> > > as link or node failure, changes in the baseline usage, or traffic 
> > > spikes?  What timescale can it react in and what level of visibility does the PCE need into current conditions in order to be reliable?
> > The above concerns has more relations to the solutions, we can add 
> > such descriptions in the solutions draft at its “Deployment 
> > Considerations” 
> > part.(https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04#s
> > ection-9)
> 
> addressed alongside the solutions draft(s); we don't necessarily need 
> to have this draft cover this topic.
> 
> > > 
> > > If we're using a PCE in a native IP network, how do the computed 
> > > routes get applied; are we using source routing or just being 
> > > careful about the prefixes in use?  (Are there going to be any 
> > > scaling concerns?)
> > > 
> > 
> > [Aijun Wang] These contents are in the above mentioned solution 
> > draft. It is not using the current source routing, but the 
> > combination of multiple BGP session and the manipulation of the path 
> > to BGP nexthop with the help of PCEP protocol. The scalability is 
> > discussed in section 9.1 of solution 
> > draft(https://tools.ietf.org/html/draft-ietf-teas-pce-native-ip-04#s
> > ection-9.1)
> 
> Okay, thanks.
> 
> 
> > 
> > 
> > > Section 3.1
> > > 
> > > I don't understand what Figure 1 is intending to convey.  Are 
> > > "Private Cloud Site" and "Public Cloud Site" supposed to be 
> > > separate boxes on the edge of the distributed control network?  
> > > Why is the "Cloud Based Application" in neither of the named clouds?
> > > 
> > 
> So the "Cloud-Based Application" has components in both Cloud sites, 
> but excludes the PCE and distributed control network?  Perhaps 
> extending the box for it to include parts at both sites, or having 
> smaller boxes at both sites and a line grouping them together, would be more clear.
> 
> > > Section 3.2
> > > 
> > > "Generally" within what scope, commercial ISPs?  I know of things 
> > > that could be called MANs that use a different topology.
> > > 
> Okay.  This was implied by the previous discussion of "service 
> provider network", but I'd suggest being explicit about it here.
> 
> > > Section 4.1
> > > 
> > [Aijun Wang]: The core nodes can also connects the customers. That 
> > is the reasons that the traffic matrix is 500*500, not 100*100
> > 
> > > 
> > > Section 4.3
> > > 
> simulation: baseline load, the congesion threshold for each link,  and 
> the distribution of flow bandwidth for the simulated flows.  I don't 
> know if the simulation is being run with zero baseline load so that 
> all flows are centrally managed or not, so please say something about 
> that.  For the other two, sould it be accurate to say something like 
> "the congestion threshold for the edge/core and intra-core links are 
> set to the fixed value of 90%" or "the congestion threshold for 
> edge/core and intra-core links are sampled from a distribution 
> representative of observed real-world environments", and "the 
> bandwidth requirements of generated flows is from a distribution representative of observed real-world flow data"?
> 
> In short, I think we need to say enough about what was done that 
> someone else could independently run if not the same experiment, one 
> sufficiently similar that we expect the results to also be similar.
> 
> Thanks,
> 
> Ben