Cycle Mapping Learning Method for Scaling DetNet
Internet-Draft draft-zhu-detnet-cycle-mapping-learning-02
Xiangyang Zhu
ZTE Corporation
No.50 Software Avenue
Nanjing
Jiangsu
211100
China
zhu.xiangyang@zte.com.cn
Jinghai Yu
ZTE Corporation
Nanjing
Jiangsu
China
yu.jinghai@zte.com.cn
Chenqiang Gao
ZTE Corporation
Nanjing
China
gao.chenqiang@zte.com.cn
Quan Xiong
ZTE Corporation
Wuhan
China
xiong.quan@zte.com.cn
2024
Routing
DetNet
The scaling DetNet (Deterministic Networking) technology based on cyclic queuing and scheduling is expected to solve the scalability problem of DetNet. One of the key technologies is to accurately obtain the cyclic mapping relationship between adjacent nodes and the packets can achieve the end-to-end deterministic forwarding. This draft describes the method for nodes to learn the cycle mapping relationship through sending learning messages.
Introduction
[I-D.ietf-detnet-scaling-requirements] proposes the requirements for deterministic forwarding of services in scaling DetNet (Deterministic Networking) networks. Typical characteristics of large-scale networks include long single-hop delay, no time sync, and large network dimensions (typically, greater than 16 hops), etc., these requirements cannot be met by the existing DetNet architecture and technology. In order to provide deterministic services in large-scale networks, [I-D.ietf-detnet-dataplane-taxonomy] has discussed the data plane enhancement solutions in DetNet which include the cycle-based queuing mechanisms such as ECQF [IEEE 802.1Qdv], Multi-CQF [I-D.dang-queuing-with-multiple-cyclic-buffers"], TCQF [I-D.eckert-detnet-tcqf] and and CSQF [I-D.chen-detnet-sr-based-bounded-latency]. During these queuing mechanisms, cycle mapping relationship should be obtained.
For the two adjacent nodes in DetNet domain, one is called upstream node (UN) and the other is downstream node (DN). The cycle-mapping relationship refers to the correspondence between the cycle label carried by the upstream node`s egress packet and the cycle label encapsulated by the downstream node`s egress port. The packet is forwarded under the guidance of the cycle mapping relationship. For example, as shown in Figure 1, if there is a cycle mapping relationship x->z between UN and DN, when the packet with the cycle label x arrives at the downstream node , the downstream node will obtain the downstream egress cycle label according to the mapping relationship x->z, then z will be encapsulated into and the packet will be scheduled and forwarded according to the cycle slot z.
Cycle Forwarding Process

   |  cycle x  |           |
UN +-----------+-----------+
          \
           \   Packet
            \  received with x
           | V         |           |  cycle z  |
DN         +-----------+  ... ...  +-----------+
                                       \
                                        \   Packet
                                         \  sent with z
                                          V

To obtain the cycle mapping relationship, we have many optional methods, for example, one can use control plane for calculation and configure the result to forwarding node, and one can also leave the task to forwarding plane self-learning. In the forwarding plane learning mode, cycle mapping learning messages are exchanged between upstream node and downstream node, and the downstream node have to calculate and store the cycle mapping relationship according to the learning messages.
This draft gives the general idea, problems and corresponding solutions for cycle mapping self-learning. This draft only considers the general learning process of the forwarding plane , and does not consider the underlying protocol (e.g., OAM) and protocol extension format of the learning message, nor does it consider the interaction protocol extension between the control plane and the forwarding plane. The consideration of related extensions will be described in detail in future drafts.
Conventions used in this document
Terminology
UN: Upstream Node
DN: Downstream Node
Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
Cycle Mapping Learning Message
The cycle mapping learning process requires UN to send learning messages to DN. No restriction is put on the detailed encapsulation format, the sending frequency, sending times and how the downstream node can identify the message, for example, it can be configured by the control plane.
At least three optional cycle mapping learning message types can be used.
Special Message
In order to clearly mark the egress cycle slot boundary, the UN can constructs and sends special protocol messages (such as PAUSE, TWAMP) as learning messages at the beginning and the ending of the cycle. In order to avoid occupying too much bandwidth, special messages are sent at a reasonable frequency, for example, sent once a physical cycle. The start time of cycle learning may be the time of system startup, controller configuration time, user triggering and so on. When a learning message arrives at a DN, it can be used to clearly mark the boundary of the UN egress cycle at local, which is called a UN virtual cycle.
Advantages: easy to mark cycle boundaries, out-of-band, flexible and controllable;
Disadvantages: Bandwidth consumption is required, and special message construction brings hardware overhead;
Original DetNet Flows Packets
Before DetNet flow enters the network, the cycle mapping relationship does not work and is meaningless. At the same time, in order to avoid bandwidth consumption in the active mode, learning can be based on actual DetNet flow packets, because when deterministic packets are sent from the source node, they will inevitably carry the egress cycle label of the UN egress. There is no restriction on the packet type, for example, it may be an SRv6 or an SR-MPLS packet.
Advantages: In-band mode can realize packets multiplexing;
Disadvantages: The number and time of sending actual flow packets are uncertain, resulting in uncertain positions of flow packets within the cycle, and cannot be directly used to calibrate the cycle boundary;
Enhanced DetNet Flows Packets
In this manner, the extended field of the flow packet should be encapsulated, and the value of "deviation from the beginning of the cycle slot" is additionally encapsulated, which can be used to clearly identify the position of the flow packet in the UN egress cycle, thereby assisting the DN to determine the virtual cycle boundary where the flow packet is located.
Advantages: In-band mode, multiplexing can be realized, and downstream nodes are easy to learn;
Disadvantage: The purpose of the extra encapsulation field is only for periodic learning, occupying the effective field of the business message. Both encapsulation and decapsulation logic need to be modified, increasing hardware resource consumption.
According to specific scenarios, one of the above methods can be selected for cycle mapping learning messages.
Cycle Mapping Learning Principle
The cycle mapping self-learning process is carried out by the DN, relying on the cycle mapping learning message sent by the UN. It is assumed that the number of queues and cache capacity configured by the UN are sufficient. The functions of packet routing and forwarding are independent from learning process described here and will not be described.
Obtain the Maximum Processing Delay of the DN
According to the latency reference model proposed in the [RFC9320], the node regulation subsystem must be able to absorb the maximum node processing delay. Typically, the processing delay variation scope only depends on the hardware implementation of a specific device, including shaping, table lookup algorithm, the packet counter resources and etc. For a specific network device, the maximum value of processing delay is determined at the product, and usually includes two components, namely the packet length related part and packet length independent part. For instance, packet-length-independent part include forwarding table lookup, CRC check, etc., and the packet length-related part includes packet cache and scheduling and so on. If the processing delay is known and can be queried by the self-learning module, no active OAM probing is required.
Though processing delay may depend on enabled functions (such as shaping, packet slicing) and packet length, the processing delay range can also be obtained through some active detection tools. Specifically, for a given device with enabled function sets, one can randomly send packets that meet the packet length limit and up to wire-speed, and then it is easy to measure the maximum processing delay. Further, the delay value can be configured in a device-specific field , and then can be used by the cycle mapping self-learning module.
The processing delay can also be configured by the control plane as needed. For example, when the control plane needs to control the end-to-end delay of the DetNet domain, it may actively increase the single-hop process delay of the device.
DN Determines the Ingress Cycle of Learning Message
Suppose that the special purpose cycle learning message described in Section 2.1 is sent by UN, it is easy for DN to correctly identify the boundary of UN egress virtual cycle slot. Different from the IEEE802.1Qch mechanism, since the upstream and downstream nodes are not time synchronized and there are long links, the packets carrying the same UN egress cycle label x, may not necessarily fall into the same DN cycle y-1, in fact some packets may also fall into cycle y, as shown in Figure 2.
DN determines ingress cycle

       |  cycle x        |                 |
UN     +-----------------+-----------------+
        \                 \
         \                 \
          \ learning msg    \learning msg
           \ P1 received     \P2 received
            \ in cycle y-1    \in cycle y
   |         V       |         V       |                 |
DN +-----------------+-----------------+-----------------+
            y-1               y                   y+1

In order to correctly obtain the egress cycle number of the downstream node, it is necessary to take the arrival time of the latest packet belonging to UN egress cycle x as the baseline, that is, y as the final receiving cycle. In the implementation, it is necessary to have a table storing the cycle mapping relationship between the UN egress and the DN ingress cycle label. When the first learning message P1 carrying the cycle label x is received, the mapping relationship x->y-1 is obtained and stored. When the second learning message carrying cycle label x is received , UN will obtain x->y relationship and update the table.
DN Determines the Message Sending Cycle
The DN calculates the local ingress->egress port cycle mapping relationship according to the maximum processing delay obtained in Section 4.1, as shown in Figure 3.
DN Determines Egress Cycle

UN      |     cycle x     |                 |
egress  +-----------------+-----------------+
         \                 \
          \P1(x)            \P2(x)
           \                 \
            V                 V
            <--virtual cycle-->
DN ingress+-----------------+--*--------------+ ....+-----------------+
          |                 |  *              |     |                 |
          |                 |  <-max process delay->|                 |
          |    cycle y-1    |     cycle y     |     | *    cycle z    |
DN egress +-----------------+-----------------+ ....+-*---------------+
                                                       \        \
                                                        \P1(z)   \P2(z)
                                                         \        \
                                                          V        V

The calculation method of the egress cycle label of DN is no very simple and intuitive, it needs to consider the cycle slot size, the slot offset of the P2 message falling in the cycle and the maximum processing delay of UN at the same time. Assuming that the cycle slot size is T and the receiving cycle is y, the formula is as follows:
UN egress cycle z=y+ ceil((processing delay-T+offset)/T) + 1
In formula above, ceil means to round-up the division result.
In particular, if (processing delay-T+offset) happens to be an integer multiple of T, in order to avoid the last flow packet of cycle x sent to thez queue cannot be forwarded in time and cause an error, because, for example, hardware implementationprecision, z can be proactively increased by 1 for redundant configuration .
Then can get the DN ingress and egress cycle mapping relationship y->z.
DN Cycle Mapping Relationship Maintenance
After obtaining the mapping relationship x->y from the upstream egress to the downstream ingress and the mapping relationship y->z from the downstream ingress to the egress, the mapping relationship x->z can be easily obtained. Since there are multiple circular queues, assuming that the number of circular queues of template T is N, the final cycle mapping relationship can be further obtained:
(x+i) MOD N -> (z+i) MOD N
For example, the UN and DN concurrently maintain a eight-cycle-queue template. Through learning, the cycle mapping relationship between the upstream egress and the downstream egress port is 4->6, then the mapping relationship maintained at the DN is:
4->6, 5->7, 6->0, 7->1,0->2,1->3,2->4,3->5.
Considerations for Cycle Mapping Operations
Cycle Learning Based on DetNet Flows
Among the three cycle self-learning messages mentioned in Section 2, both 3.1 and 3.3 can accurately identify the virtual cycle boundary where the message is located at the downstream node. When learning based on flow packets proposed in 3.2, since the arrival frequency and time of flow packets are random in cycle slot, flow packets cannot be directly used to accurately identify cycle boundaries.
Considering the scenario shown in the figure 4 upper half, since all packets of cycle 1 arrive at the downstream node, they all fall in the cycle 3, so the mapping relationship of UN egress and DN ingress, 1->3 is formed. In Figure 4 lower half, when the number of service packets increases, some packets fall into cycle 4. According to Section 4.2, it must be considered that the packet receiving cycle is 4, and the mapping relationship is adjusted to 1->4.
Ambiguous Ingress Cycle Example

       |  cycle 1        |                 |
UN     +-----------------+-----------------+
        \ \ \ \
         \ \ \ \
          \ \ \ \
           \ \ \ \
            \ \ \ \
   |         V V V V |                 |                 |
DN +-----------------+-----------------+-----------------+
            3                4                  5

       |  cycle x        |                 |
UN     +-----------------+-----------------+
        \ \ \ \\  \ \ \
         \ \ \ \\  \ \ \
          \ \ \ \\  \ \ \
           \ \ \ \\  \ \ \
            \ \ \ \\  \ \ \
   |         V V V VV| V V V           |                 |
DN +-----------------+-----------------+-----------------+
            3                4                  5

Which add a T to the single-hop jitter after adjustment in figure 4 . From the perspective of the entire service flow, the end-to-end jitter requirement of 2T cannot be met.
Idea 1 . When learning the mapping relationship for the first time, if all the UN egress cycle packets fall into the same DN ingress cycle, for example, fall into cycle y, then take the automatic adjustment and supposed it to fall into cycle y+1. At this time, by introducing a redundancy delay of T, the uncertainty introduced by flow packets can be avoided . The end-to-end delay increases by N*T, where N is the number of hops, and the max end-to-end jitter is still 2T.
Idea 2. Reshape the business message, and if there is more than flow packets in the cycle, since the queue is a FIFO, there will always be a flow packets to be sent at the beginning of the cycle, so only one message needs to be reserved and be used to mark the end of the cycle There is always a message at every moment. This method does not affect packet forwarding characteristics and bandwidth occupation.
Idea 3. The network construction is divided into two phases: testing and operation. When the flow packets requests access, it is required to send test flow packets at random or full flow for a predefined period. After the period, actual DetNet flow packets are sent according to the requirements of the traffic characteristics.
Monitoring of UN and DN Mapping Relationships
After the cycle mapping relationship is established, flow packets can directly search the mapping table to obtain the egress cycle label, and then deterministic forwarding can be obtained. However, when some problems occur, for example, the fiber switching occurs , forwarding node restarts, link attenuation occurs, or the learning process is re-triggered by the control plane or user, the mapping relationship of the upstream and downstream nodes may change. If such anomalies are not detected in time, errors will occur in the cycle mapping process, and DetNet flow determinism cannot be guaranteed.
In order to detect changes in time and update the mapping relationship,for each receiving flow packets, the DN still needs to detect ingress cycle, and compare it with the cycle mapping relationship maintained in Section 4.4. If it is inconsistent, it indicates that cycle mapping relationship has changed. For example, if the mapping relationship 2->5 is maintained in 4.4, when a UN flow packet is received and the ingress cycle of the DN is 4 or 5, it is considered correct, and in other cases it is considered problems has take place, it is necessary to re-execute the cycle mapping relationship calculation process described in Sections 4.
Cycle Mapping Relationship Storage
Consider the situation where multiple upstream nodes establish a cycle mapping to a single downstream node . When the downstream node maintains the mapping relationship, it needs to ensure the uniqueness of the table key value.
i.For point-to-point links, you can simply use {DN ingress port, DN egress port} as the key value;
ii.In the case of LAN (many-to-many or many-to-one) , multiple UN may reach Dn through the same ingress port, as shown in the figure 5, at this time, the downstream node entry is the same for multiple upstream nodes and cannot be used as a key value. Considering that in routing and forwarding, the MAC address of the upstream node will be carried in the message, so the MAC address can uniquely identify the upstream node. At this time, the UN egress source mac can be added to the key value:{mac, DN ingress port , DN egress port}.
UN and DN Connected by LAN

/-----\
| UN1 |             ________
+--+--+   _____    /        \
   |     /     \__/          \
   |    /                     \_____
   |   /                            \_____
   +-->|                                  \
       |                                   |
   +-->|                 LAN                |
   |    \                                   |      /-----\
   |     \                                  |------>| DN1 |
   |      \                                 |      +--+--+
/-----\    \                                /
| UN2 |     \___                          /
+--+--+         \                       /
                 \_____/  \___________/

UN and DN Concurrently Support Multiple Cycle Templates
If the upstream and downstream nodes support multiple cycle templates at the same time, for each cycle template, a mapping relationship can be formed through cycle self-learning process described in this draft, and the process of cycle mapping self -learning is exactly the same as that of a single cycle template.
In order to identify the periodic template at the downstream node, in addition to the time slot number, the periodic template T needs to be encapsulated in the learning message . The downstream node first extracts T and then performs learning based on the corresponding template. In addition, after the periodic self-learning calculation is completed, the entry key needs to be added with a template when storing, that is, {T, mac, DN ingress port , DN egress port} .
Security Considerations
Security considerations for DetNet are covered in the DetNet Architecture [RFC8655] and DetNet data plane [RFC8938], [RFC8939], [RFC8964] and DetNet security considerations [RFC9055]. The security considerations specified in [I-D.ietf-detnet-scaling-requirements] are also applicable to the procedures defined in this document.
IANA Considerations
This document makes no requests for IANA action.
Acknowledgements
TBA
Normative References
[RFC2119]

Key words for use in RFCs to Indicate Requirement Levels
S. Bradner
March 1997
BCP 14
RFC 2119
DOI 10.17487/RFC2119
[RFC8174]

Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words
B. Leiba
May 2017
BCP 14
RFC 8174
DOI 10.17487/RFC8174
[RFC8655]

Deterministic Networking Architecture
N. Finn
P. Thubert
B. Varga
J. Farkas
October 2019
RFC 8655
DOI 10.17487/RFC8655
[Deadline]

Deadline Based Deterministic Forwarding
Shaofu Peng
ZTE Corporation
Bin Tan
ZTE Corporation
Peng Liu
China Mobile
1 March 2022
Internet-Draft draft-peng-detnet-deadline-based-forwarding-05
[TCQF]

Deterministic Networking (DetNet) Data Plane - Tagged Cyclic Queuing and Forwarding (TCQF) for bounded latency with low jitter in large scale DetNets
Toerless Eckert
Futurewei Technologies USA
Stewart Bryant
University of Surrey ICS
Andrew G. Malis
Malis Consulting
Guangpeng Li
Huawei Network Technology Laboratory
6 November 2022
Internet-Draft draft-eckert-detnet-tcqf-02
[IEEE802.1Qch]

IEEE Standard for Local and metropolitan area networks -- Bridges and Bridged Networks - Amendment 29: Cyclic Queuing and Forwarding
IEEE
28 June 2017
IEEE 802.1Qch-2017
DOI 10.1109/IEEESTD.2017.7961303
Informative References
[I-D.ietf-detnet-scaling-requirements]

Requirements for Scaling Deterministic Networks
Peng Liu
China Mobile
Yizhou Li
Huawei
Toerless Eckert
Futurewei Technologies USA
Quan Xiong
ZTE Corporation
Jeong-dong Ryoo
ETRI
zhushiyin
New H3C Technologies
Xuesong Geng
Huawei
1 March 2023
Internet-Draft draft-ietf-detnet-scaling-requirements-01
[DIP]

Large-Scale Deterministic IP Network
Li Qiang
Huawei
Xuesong Geng
Huawei
Bingyang Liu
Huawei
Toerless Eckert
Huawei
Liang Geng
China Mobile
Guangpeng Li
2 September 2019
Internet-Draft draft-qiang-detnet-large-scale-detnet-05
[SR-TSN]

Segment Routed Time Sensitive Networking
Yaakov (J) Stein
RAD
29 August 2021
Internet-Draft draft-stein-srtsn-01
[RFC9320]

Deterministic Networking (DetNet) Bounded Latency
N. Finn
J.-Y. Le Boudec
E. Mohammadpour
J. Zhang
B. Varga
November 2022
RFC 9320
DOI 10.17487/RFC9320