Subject: Fw: New Version Notification for draft-miao-rtgwg-hpccplus-00.txt

From: "Miao, Rui" <[email-masked]>
Date: Wed, 16 Mar 2022 02:41:47 +0800

Hello All,

We have posted an updated version of HPCC++ at https://datatracker.ietf.org/doc/draft-miao-rtgwg-hpccplus/ .

There are two major changes since the last presentation. First, since HPCC++ is orthogonal to different in-band telemetry formats and transport protocols, we introduce a few reference implementations over those protocols for people to quickly pick up. 
Second, we advocate a better routing scheme for path selection and traffic admission, building on top of HPCC++'s precise link load information.

Please provide any comments you may have on the draft or its extensions.

Thanks,
Rui

------------------------------------------------------------------
From:internet-drafts <[email-masked]>
Sent At:2022 Mar. 7 (Mon.) 14:35
To:Harry <[email-masked]>; Barak Gafni <[email-masked]>; Changhoon Kim <[email-masked]>; Jeff Tantsura <[email-masked]>; Jeongkeun Lee <[email-masked]>; Rong Pan <[email-masked]>; Rui <[email-masked]>; Yuval Shpigelman <[email-masked]>
Subject:New Version Notification for draft-miao-rtgwg-hpccplus-00.txt

A new version of I-D, draft-miao-rtgwg-hpccplus-00.txt
has been successfully submitted by Rui Miao and posted to the
IETF repository.

Name:  draft-miao-rtgwg-hpccplus
Revision: 00
Title:  HPCC++: Enhanced High Precision Congestion Control
Document date: 2022-03-07
Group:  Individual Submission
Pages:  20
URL:            https://www.ietf.org/archive/id/draft-miao-rtgwg-hpccplus-00.txt
Status:         https://datatracker.ietf.org/doc/draft-miao-rtgwg-hpccplus/
Htmlized:       https://datatracker.ietf.org/doc/html/draft-miao-rtgwg-hpccplus

Abstract:
   Congestion control (CC) is the key to achieving ultra-low latency,
   high bandwidth and network stability in high-speed networks.
   However, the existing high-speed CC schemes have inherent limitations
   for reaching these goals.

   In this document, we describe HPCC++ (High Precision Congestion
   Control), a new high-speed CC mechanism which achieves the three
   goals simultaneously.  HPCC++ leverages inband telemetry to obtain
   precise link load information and controls traffic precisely.  By
   addressing challenges such as delayed signaling during congestion and
   overreaction to the congestion signaling using inband and granular
   telemetry, HPCC++ can quickly converge to utilize all the available
   bandwidth while avoiding congestion, and can maintain near-zero in-
   network queues for ultra-low latency.  HPCC++ is also fair and easy
   to deploy in hardware, implementable with commodity NICs and
   switches.

The IETF Secretariat

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Tue, 15 Mar 2022 19:43:34 +0000

Why is this draft in the Routing WG? This work is more applicable to the Transport or Internet Area.

Acee

From: rtgwg <[email-masked]> on behalf of "Miao, Rui" <miao.rui=[email-masked]>
Reply-To: "Miao, Rui" <[email-masked]>
Date: Tuesday, March 15, 2022 at 2:42 PM
To: Routing WG <[email-masked]>
Subject: Fw: New Version Notification for draft-miao-rtgwg-hpccplus-00.txt

Hello All,

Thanks,
Rui

Abstract:

The IETF Secretariat

---

From: "Miao, Rui" <[email-masked]>
Date: Wed, 16 Mar 2022 05:17:48 +0800

Hi Acee, 

Thanks for the comment. 

HPCC++ proposes an approach to leverage the in-band telemetry for traffic admission and path selection, which we think is more relevant to routing decisions. Besides, HPCC++ is orthogonal to any transports and thus we have several example implementations in the draft. 

Thanks,
Rui

------------------------------------------------------------------
From:Acee Lindem (acee) <[email-masked]>
Sent At:2022 Mar. 15 (Tue.) 12:43
To:Rui <[email-masked]>; rtgwg <[email-masked]>
Subject:Re: New Version Notification for draft-miao-rtgwg-hpccplus-00.txt

  

Thanks, 
Rui 

   

 Abstract:

 The IETF Secretariat

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Tue, 15 Mar 2022 21:47:52 +0000

Hi Rui,

Ok – I don’t know that the fact the telemetry which is a byproduct of the congestion control can be used as TE path selection criteria makes this applicable to the Routing Area. That is a real stretch…. Independent of specific transports, congestion control is a transport problem.

Thanks,
Acee

From: "Miao, Rui" <[email-masked]>
Reply-To: "Miao, Rui" <[email-masked]>
Date: Tuesday, March 15, 2022 at 5:18 PM
To: Acee Lindem <[email-masked]>, Routing WG <[email-masked]>
Subject: Re: New Version Notification for draft-miao-rtgwg-hpccplus-00.txt

Hi Acee,

Thanks for the comment.

Thanks,
Rui

Acee

Hello All,

Thanks,
Rui

Abstract:

The IETF Secretariat

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 15 Mar 2022 16:23:40 -0700

Hi Rui,
it seems that the proposed congestion control mechanism is dependent on
"inband telemetry". If my understanding is correct, I have several
questions:

   - I see that OAM methods listed in Section 6.1 can be classified, based
   on RFC 7799, as hybrid measurement methods. It is not clear why they are
   referred to as "inband". Could you please clarify the interpretation of
   "inband telemetry" in the draft.
   - Furthemore, do you see the case of using "out-of-band telemetry" as
   the basis for the proposed congestion control mechanism?
   - And lastly, in your opinion, can active OAM methods be used to provide
   the information necessary for the HPCC++?

Regards,
Greg

On Tue, Mar 15, 2022 at 2:18 PM Miao, Rui <miao.rui=
[email-masked]> wrote:

> Hi Acee,
>
> Thanks for the comment.
>
> HPCC++ proposes an approach to leverage the in-band telemetry for traffic
> admission and path selection, which we think is more relevant to routing
> decisions. Besides, HPCC++ is orthogonal to any transports and thus we have
> several example implementations in the draft.
>
> Thanks,
> Rui
>
> *From: *rtgwg <[email-masked]> on behalf of "Miao, Rui" <miao.rui=
> [email-masked]>
> *Reply-To: *"Miao, Rui" <[email-masked]>
> *Date: *Tuesday, March 15, 2022 at 2:42 PM
> *To: *Routing WG <[email-masked]>
> *Subject: *Fw: New Version Notification for
> draft-miao-rtgwg-hpccplus-00.txt
>
>
>
> Hello All,
>
>
>
> We have posted an updated version of HPCC++ at
> https://datatracker.ietf.org/doc/draft-miao-rtgwg-hpccplus/ .
>
>
>
> There are two major changes since the last presentation. First, since
> HPCC++ is orthogonal to different in-band telemetry formats and transport
> protocols, we introduce a few reference implementations over those
> protocols for people to quickly pick up.
>
> Second, we advocate a better routing scheme for path selection and traffic
> admission, building on top of HPCC++'s precise link load information.
>
>
>
> To:Harry <[email-masked]>; Barak Gafni <[email-masked]>;
> Changhoon Kim <[email-masked]>; Jeff Tantsura <
> [email-masked]>; Jeongkeun Lee <[email-masked]>; Rong Pan <
> [email-masked]>; Rui <[email-masked]>; Yuval Shpigelman <
> [email-masked]>
>
> URL:
> https://www.ietf.org/archive/id/draft-miao-rtgwg-hpccplus-00.txt
> Status:
> https://datatracker.ietf.org/doc/draft-miao-rtgwg-hpccplus/
> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-miao-rtgwg-hpccplus
>
>
> Abstract:
> _______________________________________________
> rtgwg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtgwg
>

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 15 Mar 2022 16:26:33 -0700

Thinking out loud
If the target applications for HPCC++ are in the URLLC class, perhaps the
DetNet WG might be interested in this presentation.

Regards,
Greg

On Tue, Mar 15, 2022 at 2:48 PM Acee Lindem (acee) <acee=
[email-masked]> wrote:

> Hi Rui,
>
>
>
> Ok – I don’t know that the fact the telemetry which is a byproduct of the
> congestion control can be used as TE path selection criteria makes this
> applicable to the Routing Area. That is a real stretch…. Independent of
> specific transports, congestion control is a transport problem.
>
>
>
> Thanks,
>
> Acee
>
>
>
> *From: *"Miao, Rui" <[email-masked]>
> *Date: *Tuesday, March 15, 2022 at 5:18 PM
> *To: *Acee Lindem <[email-masked]>, Routing WG <[email-masked]>
> *Subject: *Re: New Version Notification for

---

From: "Miao, Rui" <[email-masked]>
Date: Wed, 16 Mar 2022 08:47:07 +0800

Hi Acee,

The way we pointed out that the host is able to fetch precise ``load'' from the network via inband telemetry for congestion control can be integrated into different transports. The independence of specific transports stems from the fact it actually doesn't have to follow the unnecessary DATA/ACK self-clocking, but instead can use special-generated request-response packets.

Thanks,
Rui

Sent At:2022 Mar. 15 (Tue.) 14:47

Hi Rui, 
  
Thanks for the comment.  

Thanks, 
Rui 

 
Acee

Hello All, 

  
  
Thanks, 
Rui 

  

 Abstract:

 The IETF Secretariat

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 16 Mar 2022 00:53:50 +0000

What is your point? The fact that your draft is orthogonal to a specific transport doesn’t mean it isn’t applicable to the transport layer and its attendant mechanisms. It obviously belongs there and not in routing.
Thanks,
Acee

Date: Tuesday, March 15, 2022 at 8:47 PM

Hi Acee,

Thanks,
Rui

Hi Rui,

Thanks,
Acee

Hi Acee,

Thanks for the comment.

Thanks,
Rui

Acee

Hello All,

Thanks,
Rui

Abstract:

The IETF Secretariat

---

From: "Miao, Rui" <[email-masked]>
Date: Wed, 16 Mar 2022 09:40:04 +0800

Thanks Greg for the comments.

1. ``inband'' here means the host queries the network telemetry information along the path where its traffic traverses through in real time.
2.  Although the "out-of-band telemetry" may provide feedback-based control via management plane, the target scenario achieves congestion control for low latency (~10 us round trip time) where inband fits better.
3. I would think so. In fact, IOAM (draft-ietf-ippm-ioam-data-17) is part of our implementation examples. 

-Rui

------------------------------------------------------------------
From:Greg Mirsky <[email-masked]>
Sent At:2022 Mar. 15 (Tue.) 16:23
To:Rui <[email-masked]>
Cc:Acee Lindem (acee) <[email-masked]>; rtgwg <[email-masked]>
Subject:Re: New Version Notification for draft-miao-rtgwg-hpccplus-00.txt

Hi Rui,
it seems that the proposed congestion control mechanism is dependent on "inband telemetry". If my understanding is correct, I have several questions:

I see that OAM methods listed in Section 6.1 can be classified, based on RFC 7799, as hybrid measurement methods. It is not clear why they are referred to as "inband". Could you please clarify the interpretation of "inband telemetry" in the draft.
Furthemore, do you see the case of using "out-of-band telemetry" as the basis for the proposed congestion control mechanism?
And lastly, in your opinion, can active OAM methods be used to provide the information necessary for the HPCC++?
Regards,
Greg
On Tue, Mar 15, 2022 at 2:18 PM Miao, Rui <miao.rui=[email-masked]> wrote:
Hi Acee, 

Thanks for the comment. 

Thanks,
Rui

Thanks,
Rui

 Abstract:

 The IETF Secretariat

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 15 Mar 2022 19:25:59 -0700

Hi Rui,
thank you for the expedient responses to my questions. I think that there
are some aspects of how to obtain the information used to detect the
congestion that I'd like to clarify with your help:

   - From its description, HPCC++ depends on the high-precision information
   collected along the path, including timestamps. All methods listed in the
   Section 6.1 collect information into the trigger packet. But such a
   collection method negatively affects the accuracy of a timestamp because
   the time value cannot be taken in the course of physical transmission of
   the packet. What methods and techniques could be used to improve the
   accuracy of time values recorded?
   - As I understand it, HPCC++ relies on the collection of telemetry data
   on all nodes traversed by the packet. If that is true, then that would
   require that all nodes have their clocks synchronized. And to provide the
   high precision time values, the accuracy of clock synchronization in the
   domain must be at least 10 times higher then the required accuracy of time
   measurements. Hence my next question, What should be the accuracy of clock
   synchronization for HPCC++?
   - If I understand your answer #3 correctly, you consider IOAM as an
   active OAM method. According to draft-ietf-ippm-data and RFC 7799, IOAM is
   an example of hybrid type 1 measurement method. And since a useful active
   OAM protocol is always in-band with the monitored flow, I think that the
   list of protocols in Section 6.1 can be significantly expanded to include
   other known measurement protocols. What are your thoughts?

Regards,
Greg

On Tue, Mar 15, 2022 at 6:40 PM Miao, Rui <[email-masked]> wrote:

> 1. ``inband'' here means the host queries the network telemetry
> information along the path where its traffic traverses through in real time.
> 2.  Although the "out-of-band telemetry" may provide feedback-based
> control via management plane, the target scenario achieves congestion
> control for low latency (~10 us round trip time) where inband fits better.
>    - I see that OAM methods listed in Section 6.1 can be classified,
>    based on RFC 7799, as hybrid measurement methods. It is not clear why they
>    are referred to as "inband". Could you please clarify the interpretation of
>    - And lastly, in your opinion, can active OAM methods be used to
>    provide the information necessary for the HPCC++?
>
> Regards,
> Greg
>

---

From: Greg Mirsky <[email-masked]>
Date: Tue, 15 Mar 2022 19:27:38 -0700

I agree with Acee. The topic might be interesting but the Transport Area
seems more appropriate for this work. Perhaps something ADs and WG Chairs
can decide.

Regards,
Greg

On Tue, Mar 15, 2022 at 5:54 PM Acee Lindem (acee) <acee=
[email-masked]> wrote:

> What is your point? The fact that your draft is orthogonal to a specific
> transport doesn’t mean it isn’t applicable to the transport layer and its
> attendant mechanisms. It obviously belongs there and not in routing.
>
> Thanks,
>
> Acee
>
>
>
> *Date: *Tuesday, March 15, 2022 at 8:47 PM
> The way we pointed out that the host is able to fetch precise ``load''
> from the network via inband telemetry for congestion control can be
> integrated into different transports. The independence of specific
> transports stems from the fact it actually doesn't have to follow the
> unnecessary DATA/ACK self-clocking, but instead can use special-generated
> request-response packets.
>
>
>
> Thanks,
>
> Rui
>
>
>

---

From: Greg Mirsky <[email-masked]>
Date: Mon, 21 Mar 2022 08:15:43 -0700

Dear Authors,
thank you for the presentation at RTGWG meeting. I have several further
comments for your consideration:

   - I agree with Dean's comment that waiting for the congestion to occur
   and acting on the notification with unbounded latency will likely cause
   further degradation of the network condition because of the reactive
   mechanism. Polling, proposed by Dean, is one way to use telemetry data in a
   proactive manner. I would refer to the work on Precision Availability
   Metrics in Multi-SLO services
   <https://datatracker.ietf.org/doc/draft-mhmcsfh-ippm-pam/> we've
   presented earlier at the IPPM WG meeting. Monitoring the network and
   reporting crossed SLO thresholds may be the path to proactively detecting
   degradation in the network and avoiding congestion altogether.
   - Overall, it appears that there are two drafts in one - one describing
   HPPCC++ itself, and another on what telemetry data are required for
   HPPCC++. It seems like separating these might help the discussion and
   finding proper places to continue the work.
   - The draft and presentation emphasize the use of "in-band" OAM methods.
   Is the intention to collect telemetry information on each data packet? It
   could be helpful to generalize document and define not how information is
   obtained (hybrid, active, passive methods) but which information is
   required to enable HPPCC++.

Regards,
Greg

On Tue, Mar 15, 2022 at 7:25 PM Greg Mirsky <[email-masked]> wrote:

>    - From its description, HPCC++ depends on the high-precision
>    information collected along the path, including timestamps. All methods
>    listed in the Section 6.1 collect information into the trigger packet. But
>    such a collection method negatively affects the accuracy of a timestamp
>    because the time value cannot be taken in the course of
>    physical transmission of the packet. What methods and techniques could be
>    used to improve the accuracy of time values recorded?
>    - As I understand it, HPCC++ relies on the collection of telemetry
>    data on all nodes traversed by the packet. If that is true, then that would
>> Second, we advocate a better routing scheme for path selection and
>> traffic admission, building on top of HPCC++'s precise link load
>> information.
>>
>>
>>
>> To:Harry <[email-masked]>; Barak Gafni <
>> [email-masked]>; Changhoon Kim <[email-masked]>; Jeff Tantsura
>> <[email-masked]>; Jeongkeun Lee <[email-masked]>; Rong Pan
>> <[email-masked]>; Rui <[email-masked]>; Yuval Shpigelman <
>> [email-masked]>
>>

---

From: "Miao, Rui" <[email-masked]>
Date: Wed, 23 Mar 2022 05:21:59 +0800

Thanks Greg and Acee for the comments. 

We will iterate your concerns and address them in our updates. 

Thanks,
Rui

Sent At:2022 Mar. 21 (Mon.) 08:16

Dear Authors,
thank you for the presentation at RTGWG meeting. I have several further comments for your consideration:

I agree with Dean's comment that waiting for the congestion to occur and acting on the notification with unbounded latency will likely cause further degradation of the network condition because of the reactive mechanism. Polling, proposed by Dean, is one way to use telemetry data in a proactive manner. I would refer to the work on Precision Availability Metrics in Multi-SLO services we've presented earlier at the IPPM WG meeting. Monitoring the network and reporting crossed SLO thresholds may be the path to proactively detecting degradation in the network and avoiding congestion altogether.
Overall, it appears that there are two drafts in one - one describing HPPCC++ itself, and another on what telemetry data are required for HPPCC++. It seems like separating these might help the discussion and finding proper places to continue the work.
The draft and presentation emphasize the use of "in-band" OAM methods. Is the intention to collect telemetry information on each data packet? It could be helpful to generalize document and define not how information is obtained (hybrid, active, passive methods) but which information is required to enable HPPCC++.
Regards,
Greg
On Tue, Mar 15, 2022 at 7:25 PM Greg Mirsky <[email-masked]> wrote:

Hi Rui,
thank you for the expedient responses to my questions. I think that there are some aspects of how to obtain the information used to detect the congestion that I'd like to clarify with your help:

From its description, HPCC++ depends on the high-precision information collected along the path, including timestamps. All methods listed in the Section 6.1 collect information into the trigger packet. But such a collection method negatively affects the accuracy of a timestamp because the time value cannot be taken in the course of physical transmission of the packet. What methods and techniques could be used to improve the accuracy of time values recorded?
As I understand it, HPCC++ relies on the collection of telemetry data on all nodes traversed by the packet. If that is true, then that would require that all nodes have their clocks synchronized. And to provide the high precision time values, the accuracy of clock synchronization in the domain must be at least 10 times higher then the required accuracy of time measurements. Hence my next question, What should be the accuracy of clock synchronization for HPCC++?
If I understand your answer #3 correctly, you consider IOAM as an active OAM method. According to draft-ietf-ippm-data and RFC 7799, IOAM is an example of hybrid type 1 measurement method. And since a useful active OAM protocol is always in-band with the monitored flow, I think that the list of protocols in Section 6.1 can be significantly expanded to include other known measurement protocols. What are your thoughts?
Regards,
Greg
On Tue, Mar 15, 2022 at 6:40 PM Miao, Rui <[email-masked]> wrote:
Thanks Greg for the comments.

-Rui

Hi Rui,

Thanks for the comment. 

Thanks,
Rui

Thanks,
Rui

 Abstract:

 The IETF Secretariat