Subject: Re: [Lsr] LSR WG Adoption Poll for "Flexible Algorithms: Bandwidth, Delay, Metrics and Constraints" - draft-hegde-lsr-flex-algo-bw-con-02

From: <[email-masked]>
Date: Tue, 25 May 2021 10:32:40 +0800 (CST)

Dear All,

thank you for the discussion of my question on the unit of the Maximum Link Delay parameter.

Firstly, I am not suggesting it be changed to a nanosecond, but, perhaps, 10 nanoseconds or 100 nanoseconds.

To Tony's question, the delay is usually calculated from the timestamps collected at measurement points (MP). Several formats of a timestamp, but most protocols I'm familiar with, use 64 bit-long, e.g., NTP or PTP, where 32 bits represent seconds and 32 bits - a fraction of a second. As you can see, the nanosecond-level resolution is well within the capability of protocols like OWAMP/TWAMP/STAMP. As for use cases that may benefit from higher resolution of the packet delay metric, I can think of URLLC in the MEC environment. I was told that some applications have an RTT budget of in the tens microseconds range.

Shraddha, you've said

"The measurement mechanisms and advertisements in ISIS support micro-second granularity (RFC 8570)."

Could you direct me to the text in RFC 8570 that defines the measurement method, protocol that limits the resolution to a microsecond?

To Acee, I think that

"Any measurement of delay would include the both components of delay"

it depends on where the MP is located (yes, it is another "It depends" situation). 

I agree with Anoop that it could be beneficial to have a text in the draft that explains three types of delays a packet experiences and how the location of an MP affects the accuracy of the measurement and the metric.

Best regards,

Greg Mirsky

Sr. Standardization Expert
预研标准部/有线研究院/有线产品经营部 Standard Preresearch Dept./Wireline Product R&D Institute/Wireline Product Operation Division

E: [email-masked] 
www.zte.com.cn

---

From: Anoop Ghanwani <[email-masked]>
Date: Tue, 25 May 2021 00:11:01 -0700

Greg,

One thing to keep in mind is that even though we can measure latency at a
precision of 10's or 100's of nanoseconds, does it hurt to round the link
delay up to the nearest microsecond?  One way to look at this is that by
doing such rounding up, we add at most 1 usec worth of additional delay per
hop.  That was my rationale for thinking it's probably OK to leave the
resolution at 1 usec.

I don't have a strong opinion either way.

Anoop

On Mon, May 24, 2021 at 7:32 PM <[email-masked]> wrote:

> To Tony's question, the delay is usually calculated from the timestamps
> collected at measurement points (MP). Several formats of a timestamp, but
> most protocols I'm familiar with, use 64 bit-long, e.g., NTP or PTP, where
> 32 bits represent seconds and 32 bits - a fraction of a second. As you can
> see, the nanosecond-level resolution is well within the capability of
> protocols like OWAMP/TWAMP/STAMP. As for use cases that may benefit from
> higher resolution of the packet delay metric, I can think of URLLC in the
> MEC environment. I was told that some applications have an RTT budget of in
> the tens microseconds range.
>
>
> Shraddha, you've said
>
> Could you direct me to the text in RFC 8570 that defines the measurement
> method, protocol that limits the resolution to a microsecond?
>
>
> To Acee, I think that
>
> I agree with Anoop that it could be beneficial to have a text in the draft
> that explains three types of delays a packet experiences and how the
> location of an MP affects the accuracy of the measurement and the metric.
>
>
> Best regards,
>
> Greg Mirsky
>
>
> 预研标准部/有线研究院/有线产品经营部 Standard Preresearch Dept./Wireline Product R&D
> Institute/Wireline Product Operation Division
>
>
>
> _______________________________________________
> Lsr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lsr
>

---

From: Shraddha Hegde <[email-masked]>
Date: Tue, 25 May 2021 09:45:56 +0000

Snipped …

>Could you direct me to the text in RFC 8570 that defines the measurement method, protocol that limits the >resolution to a microsecond?

Pls refer RFC 8570 sec 4.1. The link delay is encoded in microseconds

“   Delay:  This 24-bit field carries the average link delay over a
      configurable interval in microseconds, encoded as an integer
      value.  When set to the maximum value 16,777,215
      (16.777215 seconds), then the delay is at least that value and may
      be larger.
“

Exact same text can be found in OSPF RFC 7471 sec 4.1.5 as well.

Rgds

Shraddha

Juniper Business Use Only
From: Lsr <[email-masked]> On Behalf Of [email-masked]
Sent: Tuesday, May 25, 2021 8:03 AM
To: [email-masked]
Subject: Re: [Lsr] LSR WG Adoption Poll for "Flexible Algorithms: Bandwidth, Delay, Metrics and Constraints" - draft-hegde-lsr-flex-algo-bw-con-02

[External Email. Be cautious of content]

Dear All,

Shraddha, you've said

To Acee, I think that

Best regards,

Greg Mirsky

E: [email-masked]<mailto:[email-masked]>
www.zte.com.cn<https://urldefense.com/v3/__http:/www.zte.com.cn/__;!!NEt6yMaO-gk!VRzSJtXYzBGvfAj_K6DQpR2shaOjIkVhd7dPvtosaNV9DVJCJkam3Ub0C07UEhVy$>

---

From: Tony Li <[email-masked]>
Date: Tue, 25 May 2021 09:52:08 -0700

Hi Greg,
> Firstly, I am not suggesting it be changed to a nanosecond, but, perhaps, 10 nanoseconds or 100 nanoseconds.

Ok.  The specific precision isn’t particularly relevant to me.  The real questions are whether microseconds are the right base or not, and whether we should shift to floating point for additional range or add more bits.

> To Tony's question, the delay is usually calculated from the timestamps collected at measurement points (MP). Several formats of a timestamp, but most protocols I'm familiar with, use 64 bit-long,…

It’s very true that folks have carried around nanosecond timestamps for a long time now.  No question there. My question is whether it is actually useful. While NTP has that precision in its timestamps, the actual precision  of NTP’s synchronization algorithms aren’t quite that strong.  In effect, many of those low order bits are wasted.

That’s not a big deal, but when we make the base more precise, we lose range.  If we go with 32 bits of nanoseconds, we limit ourselves to a link delay of ~4 seconds. Tolerable, but it will certainly disappoint Vint and his inter-planetary Internet. :-)

We could go with 64 bits of nanoseconds, but then we’ll probably only rarely use the high order bits, so that seems wasteful of bandwidth.

Or we can go to floating point. This will greatly increase the range, at the expense of having fewer significant bits in the mantissa.

Personally, I would prefer to stay with 32 bits, but I’m flexible after that.

Tony

---

From: <[email-masked]>
Date: Wed, 26 May 2021 04:32:32 +0800 (CST)

Hi Shraddha,

thank you for pointing out the text. Though it mentions that the value is the average delay calculated over configured, i.e., pre-defined interval, that seems it leaves out some important aspects of the measurement method, e.g., number of measurements in the set over which the average is calculated. Also, I'd note that the average is not the most informative and robust metric as it is more sensitive to extremes (spikes and drops) than, for example, median and percentile (the most often used are 95, 99, and 99.9).

Regards,

Greg Mirsky

Original Mail

Sender: ShraddhaHegde
To: gregory mirsky10211915;[email-masked];
Date: 2021/05/25 02:46
Subject: RE: [Lsr] LSR WG Adoption Poll for "Flexible Algorithms: Bandwidth, Delay, Metrics and Constraints" - draft-hegde-lsr-flex-algo-bw-con-02

Snipped …

      be larger.

“

Rgds

Shraddha

 

 

 

 

Dear All,

Shraddha, you've said

To Acee, I think that

Best regards,

Greg Mirsky

---

From: <[email-masked]>
Date: Wed, 26 May 2021 04:47:16 +0800 (CST)

Hi Anoop,

thank you for sharing your thoughts. I agree, it is very likely that in most cases, the potential inaccuracy of 1 usec per link would not affect the construction of a route. But for cases that require very low bounded latency, e.g., DetNet, such a level of uncertainty about the actual value of the delay might fail to find a route altogether. Consider a case where the measured e2e delay is N usec is well within the required level. But the sum of the advertised Maximum Link Delays is above that threshold.

I think that using 100 nsec as the unit of the Maximum Link Delay is a reasonable compromise between that scale and accuracy.

Regards,

Greg Mirsky

Original Mail

Sender: AnoopGhanwani
To: gregory mirsky10211915;
CC: [email-masked];
Date: 2021/05/25 00:11
Subject: Re: [Lsr] LSR WG Adoption Poll for "Flexible Algorithms: Bandwidth, Delay, Metrics and Constraints" - draft-hegde-lsr-flex-algo-bw-con-02

Greg,
One thing to keep in mind is that even though we can measure latency at a precision of 10's or 100's of nanoseconds, does it hurt to round the link delay up to the nearest microsecond?  One way to look at this is that by doing such rounding up, we add at most 1 usec worth of additional delay per hop.  That was my rationale for thinking it's probably OK to leave the resolution at 1 usec.  

Anoop 

Dear All,

Shraddha, you've said

To Acee, I think that

Best regards,

Greg Mirsky

---

From: <[email-masked]>
Date: Wed, 26 May 2021 05:02:33 +0800 (CST)

Hi Tony,

thank you for clarifying your view on this. Please find my notes in-line below under the GIM>> tag.

Regards,

Greg Mirsky

Original Mail

Sender: TonyLi
CC: lsr;
Date: 2021/05/25 09:52

Hi Greg,

GIM>> What I see from deployment of active measurement protocols, e.g., TWAMP and STAMP, is the strong interest in using PTP, i.e. IEEE 1588v2, timestamp format in the data plane. And the requirement (actually, there are different profiles) for the quality of clock synchronization for 5G is, as I understand it, achievable with PTP. I have no information if that is the case with NTP.

GIM>> Agree. I would propose to consider 100 usec as a unit, which gets close to 7 minutes.

GIM>> I think that we can stay with a 32 bit-long field and get better resolution at the same time.

Tony

---

From: Anoop Ghanwani <[email-masked]>
Date: Tue, 25 May 2021 15:07:30 -0700

That’s not a big deal, but when we make the base more precise, we lose
range.  If we go with 32 bits of nanoseconds, we limit ourselves to a link
delay of ~4 seconds. Tolerable, but it will certainly disappoint Vint and
his inter-planetary Internet. :-)
The current 24 bits of usec delay isn't that much better at ~16 sec.
(Unless there is a proposal to change that to 32 bits?)

On Tue, May 25, 2021 at 9:53 AM Tony Li <[email-masked]> wrote:

> Ok.  The specific precision isn’t particularly relevant to me.  The real
> questions are whether microseconds are the right base or not, and whether
> we should shift to floating point for additional range or add more bits.
>
> It’s very true that folks have carried around nanosecond timestamps for a
> long time now.  No question there. My question is whether it is actually
> useful. While NTP has that precision in its timestamps, the actual
> precision  of NTP’s synchronization algorithms aren’t quite that strong.
> In effect, many of those low order bits are wasted.
>
> That’s not a big deal, but when we make the base more precise, we lose
> range.  If we go with 32 bits of nanoseconds, we limit ourselves to a link
> delay of ~4 seconds. Tolerable, but it will certainly disappoint Vint and
> his inter-planetary Internet. :-)
>
> We could go with 64 bits of nanoseconds, but then we’ll probably only
> rarely use the high order bits, so that seems wasteful of bandwidth.
>
> Or we can go to floating point. This will greatly increase the range, at
> the expense of having fewer significant bits in the mantissa.
>

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Tue, 25 May 2021 23:00:15 +0000

Speaking as a WG member:

I think the argument for delays < 1 usec is very weak and haven’t heard any compelling arguments.

Thanks,
Acee

From: Lsr <[email-masked]> on behalf of Anoop Ghanwani <[email-masked]>
Date: Tuesday, May 25, 2021 at 6:08 PM
To: Tony Li <[email-masked]>
Cc: lsr <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Lsr] LSR WG Adoption Poll for "Flexible Algorithms: Bandwidth, Delay, Metrics and Constraints" - draft-hegde-lsr-flex-algo-bw-con-02

The current 24 bits of usec delay isn't that much better at ~16 sec.  (Unless there is a proposal to change that to 32 bits?)

On Tue, May 25, 2021 at 9:53 AM Tony Li <[email-masked]<mailto:[email-masked]>> wrote:

Hi Greg,

Tony

_______________________________________________
Lsr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/lsr

---

From: tom petch <[email-masked]>
Date: Fri, 28 May 2021 15:58:34 +0000

From: Lsr <[email-masked]> on behalf of [email-masked] <[email-masked]>
Sent: 25 May 2021 22:02

Inline under <tp>
Tom Petch

Hi Tony,

Regards,

Greg Mirsky

E: [email-masked]<mailto:[email-masked]>
www.zte.com.cn<http://www.zte.com.cn/>
Original Mail
Sender: TonyLi
Date: 2021/05/25 09:52

Hi Greg,

<tp>

A related item.  I was looking at draft-ietf-detnet-yang which uses uint32 nanoseconds for latency so that is another where nanoseconds can be found.

Tom Petch

Tony