Subject: Re: [Detnet] Comments on I-D Action: draft-ietf-detnet-oam-framework-01.txt

From: Balázs Varga A <[email-masked]>
Date: Sun, 6 Jun 2021 10:58:05 +0000

Hi,

As discussed on the last OAM call, please find below some text proposal to update section 2.
Please feel free to take it or modify it. 

Cheers
Bala'zs

ORIGINAL TEXT
2. Role of OAM in DetNet

DetNet networks expect to provide communications with predictable low
packet delay and packet loss. Most critical applications will define
an SLO to be required for the data flows it generates.

To respect strict guarantees, DetNet can use an orchestrator able to
monitor and maintain the network. Typically, a Software-Defined
Network (SDN) controller places DetNet flows in the deployed network
based on their the SLO. Thus, resources have to be provisioned a
priori for the regular operation of the network. OAM represents the
essential elements of the network operation and necessary for OAM
resources that need to be accounted for to maintain the network
operational.

Fault-tolerance also assumes that multiple paths could be provisioned
so that an end-to-end circuit is maintained by adapting to the
existing conditions. The central controller/orchestrator typically
controls the Packet Replication, Elimination, and Ordering Functions
(PREOF) on a node. OAM is expected to support monitoring and
troubleshooting PREOF on a particular node and within the domain.
Note that PREOF can also be controlled by a set of distributed
controllers, in those scenarios where DetNet solutions involve more
than one single central controller.

NEW TEXT
2. Role of OAM in DetNet

DetNet networks expect to provide communications with predictable low
packet delay and packet loss. Most critical applications will define
an SLO to be required for the data flows it generates.

To respect strict guarantees, DetNet can use an orchestrator able to
monitor and maintain the network. Typically, a Software-Defined
Network (SDN) controller places DetNet flows in the deployed network
based on their the SLO. Thus, resources have to be provisioned a
priori for the regular operation of the network. OAM represents the
essential elements of the network operation and necessary for OAM
resources that need to be accounted for to maintain the network
operational.

Many legacy OAM tools can be used in DetNet networks, but they are 
not able to cover all the aspects of deterministic networking. Fulfilling
strict guarantees is essential for DetNet flows, what resulted in new
DetNet specific functionalities which must be covered with OAM as well.
Filling these gaps are inevitable and needs accurate consideration of 
DetNet specifics. Similar to DetNet flows itself, their OAM needs careful 
end-to-end engineering as well.

For example, proper placing of MEPs along the path of a DetNet flow is
not always a trivial task and may require proper design together with the 
design of the service component of a given DetNet flow.

There are several DetNet specific challenges for OAM. Bounded network 
characteristics (e.g., delay, loss) are inseparable service parameters,
therefore PM is a key topic for DetNet. OAM tools needed to prove the
SLO without impacting the data flow characteristics. A further challenge
is the strict resource allocation. Resources used by OAM must be considered
and allocated to avoid disturbing data flow(s). 

The DetNet Working Group has defined two sub-layers: (1) DetNet
service sub-layer, at which a DetNet service (e.g., service
protection) is provided and (2) DetNet forwarding sub-layer, which
optionally provides resource allocation for DetNet flows over paths
provided by the underlying network. OAM mechanisms exist for the 
DetNet forwarding sub-layer, nonetheless, OAM for the service 
sub-layer requires new OAM procedures. These new OAM functions 
must allow, for example, to recognize/discover DetNet relay 
nodes, to get information about their configuration, and to 
check their operation or status.

DetNet service sub-layer functions using a sequence number. That creates
challenge for inserting OAM packets in the DetNet data flow. 

Fault-tolerance also assumes that multiple paths could be provisioned
so that an end-to-end circuit is maintained by adapting to the
existing conditions. The central controller/orchestrator typically
controls the Packet Replication, Elimination, and Ordering Functions
(PREOF) on a node. OAM is expected to support monitoring and
troubleshooting PREOF on a particular node and within the domain.

Note that PREOF can also be controlled by a set of distributed
controllers, in those scenarios where DetNet solutions involve more
than one single central controller.

DetNet forwarding sub-layer is based on legacy technologies and has
a much better coverage regarding OAM. However, the forwarding sub-layer
is terminated at DetNet relay nodes, so end-to-end OAM state of forwarding 
may be created only based on the status of multiple forwarding sub-layer segments
serving a given DetNet flow (e.g., in case of DetNet MPLS, there may be 
no end-to-end LSP below the DetNet PW).
END

-----Original Message-----
From: detnet <[email-masked]> On Behalf Of Balázs Varga A
Sent: Friday, May 21, 2021 3:27 PM
To: [email-masked]
Subject: [Detnet] Comments on I-D Action: draft-ietf-detnet-oam-framework-01.txt

Hi Greg, co-authors,

Many thanks for the update. I have some further comments below.
I would propose to discuss them during the next DetNet OAM call on 25th May if appropriate.

Comments:
Section 1. Introduction
TEXT 
   ... its ability to respect the Service Level
   Objectives (SLO), such as packet delay, delay variation, and packet
   loss ratio, assigned to each data flow.
>Comment1> Please, consider to add here "in-order-delivery" as well. It is also part of the SLO.

Section 1.1.  Terminology
TEXT
   *  OAM entity: a data flow to be monitored for defects and/or its
      performance metrics measured.
>Comment2> This definition is more the object of OAM. Would fit better to define "OAM entity" as a function that provides OAM related actions.

Section 2. Role of OAM in DetNet
General comment on the content.
>Comment3> It should be made clear here that general legacy OAM tools 
>Comment3> can be used for DetNet as well, however some DetNet specifics
extensions may be needed.

>Comment4> Please consider to add here DetNet specifics, i.e., DetNet service sub-layer and forwarding sub-layer. Service sub-layer is service protection (i.e., PREOF) and forwarding sub-layer is for explicit route + dedicated/allocated resources. DetNet defines some new functions or use existing functions with some special requirements. Checking these functions with new or updated OAM tools is part of the framework document.

Section 3.  Operation
>Comment5> General question: should this chapter contain DetNet specific statements?

>Comment6> Specific question: I miss here the performance management related aspects. In case of DetNet, too late is also a failure.
PM should cover: loss, delay, delay variation and in-order-delivery.

Section 3.1. Information Collection
TEXT 
   Also, we can characterize methods of transporting OAM information
   relative to the path of data.
>Comment7> Please consider to extend the terminology. As processing of DetNet flow packets is time critical we may distinguish 4 scenarios:
- inside-flow: OAM information is placed inside a regular DetNet flow data packet
- outside-flow: OAM information is placed in extra packet(s)
- in-band (or in-path): OAM information uses same resources allocated for the monitored DetNet flow
- out-of-band (or out-of-path): OAM information uses different resources than allocated for the monitored DetNet flow

TEXT
   In the latter, information
   is collected in a sequence of follow-up packets that traverse the
   same path as the data packet of the monitored DetNet flow.
>Comment8> This text intends to refer to an out-of-band method. Is this statement correct? Should it say the opposite?

Section 3.2.  Continuity Check
TEXT
   Continuity check is used to monitor the continuity of a path, i.e.,
   that there exists a way to deliver the packets between two endpoints
   A and B.
>Comment9> Why are they called endpoints? Are A and B MEPs? Or A and B can any node along the e2e path, between them we intend to check the continuity?

Section 3.4. Route Tracing
TEXT
   DetNet with IP data plane is NOT RECOMMENDED to use multiple paths or
   links,
>Comment10> Why only IP? ECMP is in general not recommended for DetNet. It is valid for both IP and MPLS data plane.

3.7.  Use of Hybrid OAM in DetNet
General comment on section content.
>Comment11> Would fit better to section 2. Text is more a general statement of what OAM technics used for what in case of DetNet.

TEXT
   Hybrid OAM methods are used in performance monitoring and defined in
   [RFC7799] as:
>Comment12> PM should be covered in a dedicated section, not only in section 3.

Section 4.  Administration
TEXT
   The network SHOULD expose a collection of metrics to support an
   operator making proper decisions, including:
>Comment13> List should be extended. Adding here e.g., service sub-layer specific metrics, like packet processing resources, etc.

TEXT
   *  per virtual circuit to measure ...
>Comment14> This term is not used for DetNet. How is it related to DetNet? Should we use DetNet flow / member flow / compound flow instead?

TEXT
   *  per path to detect misbehaving path when multiple paths are
      applied.
>Comment15> What it intends to say? It sounds strange that a path has 
>Comment15> paths within itself. Does it meant a segment here? (E.g., 
>Comment15> Per segment to detect misbehaving path when multiple paths 
>Comment15> are applied.)

TEXT
   *  per device to detect misbehaving node, when it relays the packets
      of several flows.
>Comment16> "node" --> "function"? What is the relation of device/node?

Section 4.1. Collection of metrics
General comment.
>Comment17> Agree with the content but it is general text not a DetNet specific one. I think it would be good to add rationale, e.g., strict resource allocation/reservation per DetNet flow.

Section 5.1. Replication / Elimination
TEXT
   When multiple paths are reserved between two maintenance endpoints,
   packet replication may be used to introduce redundancy and alleviate
   transmission errors and collisions.
>Comment18> I do not think this is a valid statement. Placing MEPs and PREOF points are orthogonal. They may or may not be in the same node.
>Comment19> "alleviate transmission errors and collisions" are not really the target of PREOF. Are these RAW specifics?

TEXT
   ... Each maintenance endpoint will decide to trigger the packet
   replication, elimination or the ordering process when a set of
   metrics passes a threshold value.
>Comment20> I think such a function is not defined so far in DetNet. Is it something RAW specific? PREOF is not on-demand but something defined before data-flow forwarding starts.

TEXT
   Figure 1.
>Comment21> What is a "root" in the figure?

Section 5.2.  Resource Reservation
TEXT
   ... We need to provide mechanisms to patch the network
   configuration.
>Comment22> What does it mean " to patch the network configuration"? Is this a management function? In DetNet, the service sub-layer functions are envisioned to cope with such situations. However, in DetNet we consider failure of a path, not QoS degradation. Sound like a radio link specific statement ...

Section 5.3.  Soft transition after reconfiguration TEXT
   Since DetNet expects to support real-time flows, DetNet OAM MUST
   support soft-reconfiguration,
>Comment23> Why OAM must support this? Is it not more a functionality implementation characteristics to be able to reconfigure without service interruption?

Section 6. Requirements
TEXT
   1.   It MUST be possible to initiate DetNet OAM session from any
        DetNet node towards another DetNet node(s) within given domain.
>Comment24> Do we need a definition in the Terminology section for DetNet OAM Session? 
(E.g., DetNet OAM session: exchanging OAM data between MEPs, or a better definition ...)

TEXT
   2.   It MUST be possible to initialize DetNet OAM session from a
        centralized controller.
>Comment25> Depending on the previous comment resolution: --> via a centralized controller? The SDNc is not a MEP.

TEXT
   3.   DetNet OAM MUST support proactive and on-demand OAM monitoring
        and measurement methods.
>Comment26> Why only these aspects (monitoring and management) of OAM? Proposed to be deleted and leave "... OAM methods."

TEXT
   5.   DetNet OAM MUST support unidirectional OAM methods, continuity
        check, connectivity verification, and performance measurement.
>Comment27> Is it an exhaustive list? What about e.g., fault localization / characterization?

TEXT
   6.   DetNet OAM MUST support bi-directional OAM methods.  Such OAM
        methods MAY combine in-band monitoring or measurement in the
        forward direction and out-of-bound notification in the reverse
        direction, i.e., from egress to ingress end point of the OAM
        test session.
>Comment28> " egress to ingress end point" Are these the MEPs?

TEXT
   8.   DetNet OAM MUST support Path Maximum Transmission Unit
        discovery.
>Comment29> Why is it DetNet specific? It is a general tool, not?

TEXT
   13.  DetNet OAM MUST support unidirectional performance measurement
        methods.  Calculated performance metrics MUST include but are
        not limited to throughput, packet loss, delay and delay
        variation metrics.  [RFC6374] provides detailed information on
        performance measurement and performance metrics.
>Comment30> How is this point related to requirement 6.5 (also mentioning PM)?
>Comment31> To be discussed whether adding "out-of-order" characteristics as well here.

TEXT
   15.  DetNet OAM MUST support methods to enable survivability of the
        DetNet domain.  These recovery methods MAY use protection
        switching and restoration.
>Comment32> " survivability" term not used in DetNet so far. How it is related to DetNet sub-layers? Is it a forwarding sub-layer functionality?

TEXT
   16.  DetNet OAM MUST support the discovery of Packet Replication,
        Elimination, and Order preservation sub-functions locations in
        the domain.
   17.  DetNet OAM MUST support testing of Packet Replication,
        Elimination, and Order preservation sub-functions in the domain.
>Comment33> "Order" --> "Ordering"
>Comment34>  "sub-functions" --> "functions". These are functions not sub-functions.

Special thanks if You have read so far ... :--)))))

Cheers
Bala'zs

-----Original Message-----
From: detnet <[email-masked]> On Behalf Of [email-masked]
Sent: Wednesday, May 19, 2021 10:02 PM
To: [email-masked]
Cc: [email-masked]
Subject: [Detnet] I-D Action: draft-ietf-detnet-oam-framework-01.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Deterministic Networking WG of the IETF.

        Title           : Framework of Operations, Administration and Maintenance (OAM) for Deterministic Networking (DetNet)
        Authors         : Greg Mirsky
                          Fabrice Theoleyre
                          Georgios Z. Papadopoulos
                          Carlos J. Bernardos
	Filename        : draft-ietf-detnet-oam-framework-01.txt
	Pages           : 14
	Date            : 2021-05-19

Abstract:
   Deterministic Networking (DetNet), as defined in RFC 8655, is aimed
   to provide a bounded end-to-end latency on top of the network
   infrastructure, comprising both Layer 2 bridged and Layer 3 routed
   segments.  This document's primary purpose is to detail the specific
   requirements of the Operation, Administration, and Maintenance (OAM)
   recommended to maintain a deterministic network.  With the
   implementation of the OAM framework in DetNet, an operator will have
   a real-time view of the network infrastructure regarding the
   network's ability to respect the Service Level Objective, such as
   packet delay, delay variation, and packet loss ratio, assigned to
   each data flow.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-detnet-oam-framework/

There is also an HTML version available at:
https://www.ietf.org/archive/id/draft-ietf-detnet-oam-framework-01.html

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-detnet-oam-framework-01

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
detnet mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet

_______________________________________________
detnet mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/detnet

---

From: <[email-masked]>
Date: Mon, 7 Jun 2021 10:46:54 +0800 (CST)

Hi Balazs,

many thanks for the prepared text. I'm working on updating the draft and will have a new version before the meeting to share with everyone. I have to apologize for not joining the meeting this week. I have the BBF Q2 meeting in the London time zone, and it ends at 03:00 PDT (yes, three after midnight).

Regards,

Greg Mirsky

Sr. Standardization Expert
预研标准部/有线研究院/有线产品经营部 Standard Preresearch Dept./Wireline Product R&D Institute/Wireline Product Operation Division

E: [email-masked] 
www.zte.com.cn

Original Mail

Sender: BalázsVargaA
To: [email-masked];
Date: 2021/06/06 03:58
Subject: Re: [Detnet] Comments on I-D Action: draft-ietf-detnet-oam-framework-01.txt

Hi,

Cheers
Bala'zs
 
ORIGINAL TEXT
2. Role of OAM in DetNet

 
 
NEW TEXT
2. Role of OAM in DetNet

 

 

 

 

-----Original Message-----
From: detnet <[email-masked]> On Behalf Of Balázs Varga A
Sent: Friday, May 21, 2021 3:27 PM
To: [email-masked]
Subject: [Detnet] Comments on I-D Action: draft-ietf-detnet-oam-framework-01.txt
 
Hi Greg, co-authors,

Comments:
Section 1. Introduction
TEXT  
 
Section 1.1.  Terminology
TEXT

 
Section 3.  Operation

 
TEXT

 
 
TEXT

TEXT
 
TEXT
 
TEXT

 
TEXT
 
TEXT
   Figure 1.

 
Section 6. Requirements
TEXT
 
TEXT
 
TEXT
 
TEXT
 
TEXT
 
TEXT
 
TEXT
 
TEXT
 
TEXT

Cheers
Bala'zs

 
-----Original Message-----
From: detnet <[email-masked]> On Behalf Of [email-masked]
Sent: Wednesday, May 19, 2021 10:02 PM
To: [email-masked]
Cc: [email-masked]
Subject: [Detnet] I-D Action: draft-ietf-detnet-oam-framework-01.txt

 
Abstract:

---

From: <[email-masked]>
Date: Tue, 8 Jun 2021 04:41:11 +0800 (CST)

Hi Balazs,
again, thank you for the text you've so kindly contributed. I've updated the working version (attached with the diff) according to your suggestion. The working version also includes updates we've been working on to address comments from your review.
I much appreciate it if you share your feedback on these updates.
And my apologies, I'll not join in the next call.

Regards,
Greg Mirsky
------------------Original Mail------------------
Sender: gregory mirsky10211915
To: balazs.a.varga=[email-masked];
CC: [email-masked];
Date: 2021/06/06 19:47
Subject: Re: [Detnet]  Comments on I-D Action: draft-ietf-detnet-oam-framework-01.txt

Hi Balazs,

Regards,
Greg Mirsky
-----Original Message-----
From: detnet <[email-masked]> On Behalf Of Balázs Varga A
Sent: Friday, May 21, 2021 3:27 PM
To: [email-masked]
Subject: [Detnet] Comments on I-D Action: draft-ietf-detnet-oam-framework-01.txt
Hi Greg, co-authors,
.. its ability to respect the Service Level
Section 1.1.  Terminology
TEXT
Section 3.  Operation
TEXT
TEXT
TEXT
TEXT
>Comment19> "alleviate transmission errors and collisions" are not really the target of PREOF. Are these RAW specifics?
TEXT
.. Each maintenance endpoint will decide to trigger the packet
replication, elimination or the ordering process when a set of
metrics passes a threshold value.
TEXT
Figure 1.
Section 5.2.  Resource Reservation
TEXT
.. We need to provide mechanisms to patch the network
configuration.
Section 6. Requirements
TEXT
TEXT
TEXT
TEXT
TEXT
TEXT
TEXT
TEXT

---

From: Fabrice Theoleyre <[email-masked]>
Date: Wed, 9 Jun 2021 14:52:44 +0200

Dear Balazs, 

After the call of Yesterday, please find inline the comments of our last modifications.

>> Comment28> " egress to ingress end point" Are these the MEPs?

FT>> we have yesterday a quite long discussion on this problem (are MEP only the ingress/egress nodes?). In particular, PREOF may require to have nodes in charge of elimination / replication. Thus, We changed the assertion 1:
TEXT: "It MUST be possible to initiate DetNet OAM session from any  DetNet node towards another DetNet node(s) within given domain."

FT>> Balazs argued yesterday that the source and destination may be DetNet nodes. So, we insert a novel requirement
TEXT "It MAY be possible to initiate a direct DetNet OAM session from a  DetNet source towards a DetNet destination within given domain."

FT>> We also discussed the problem of bidirectional DetNet flows. 
FT>> We separated now the requirement for bidirectional vs. unidirectional flows. For unidirectional DetNet flows, the forward direction may be in-band, the inverse direction may be out-of-band.
TEXT: « OAM methods MAY combine in-band monitoring or measurement in the forward direction and out-of-bound notification in the reverse direction, i.e., from egress to ingress end point of the OAM test session.»

FT>> For bidirectional DetNet flows, we MUST support two OAM unidirectional sessions 
TEXT: "DetNet OAM MUST support bidirectional DetNet flows, with a pair of OAM sessions, each of them in charge of one of the two directions. »

@all: Please tell us if it is compliant with what you have possibly in mind for uni/bidirectional DetNet flows.

>> Comment31> To be discussed whether adding "out-of-order" characteristics as well here.

FT>> good suggestion, inserted in the list.

>> Comment32> " survivability" term not used in DetNet so far. How it is related to DetNet sub-layers? Is it a forwarding sub-layer functionality?

FT>> We replaced it by availability to be compliant with RAW.

>> Comment34>  "sub-functions" --> "functions". These are functions not sub-functions.

FT>> Greg seems have already change it.

FT>> We will upload the novel version.

Thank you very much for your constructive feedback. 

Cheers,
Fabrice

---

From: Balázs Varga A <[email-masked]>
Date: Tue, 22 Jun 2021 11:11:55 +0000

Dear Fabrice,
Thanks for the updates, they are fine with me.
Cheers
Bala’zs

From: Fabrice Theoleyre <[email-masked]>
Sent: Wednesday, June 9, 2021 2:53 PM
To: Balázs Varga A <[email-masked]>
Cc: [email-masked]
Subject: Re: [Detnet] Comments on I-D Action: draft-ietf-detnet-oam-framework-01.txt

Dear Balazs,

TEXT

TEXT

TEXT

TEXT

Cheers,
Fabrice