Subject: [TICTOC] draft-ietf-mpls-loss-delay Timestamp

From: Stewart Bryant <[email-masked]>
Date: Thu, 03 Mar 2011 16:06:36 +0000

We have received a LC comment on draft-ietf-mpls-loss-delay concerning 
the default timestamp.

In the first version of the draft we proposed NTP, but following initial 
comments from the MPLS-TP community we changed to IEEE1588. This 
requirement for IEEE1588 can be traced back to the choice of using 
IEEE1588 in Y.1731.

We have now received a LC request to change the default back to NTP.

NTP is the "natural" choice for an IETF protocol. NTP is specified in 
IETF and is used in other MPLS protocols such as LSP ping. It is 
implemented on almost every host and every router. However in general 
the implementations provides a relatively low precision timestamp, and 
the NTP time distribution infrastructure operates on a best effort 
basis. Thus even a good client implementation would normally have a 
relatively low quality path to the server, which would result in a low 
quality of timestamp. NTP could be made to work to higher accuracy, but 
defacto upgrading NTP and providing high quality NTP paths to the time 
servers is not getting much attention. Computing timestamp differences 
is easier with NTP.

On the other hand IEEE1588 has defacto become the two way time tranfer 
protocol for precision applications. IEEE1588 only provides high quality 
time in well engineered networks with some form of hop by hop 
assistance. It is not widely implemented other than in equipment 
targeted to specific markets, although one of those markets is in mobile 
backhaul applications  where we expect to see significant initial 
deployment of MPLS-TP. Computing timestamp differences is harder with 
IEEE1588

The loss-delay work is targeting to be able to do one way packet delay 
measurement, so it does need a higher quality of timestamp than is 
required in general purpose network instrumentation.

Converting from IEEE1588 to NTP is not trivial, since they use different 
epochs, and different representations of sub-second time. In a 
"traditional" NTP implementation the time error due to on the fly 
conversion is likely to be small compared to the time error in the time 
synchronization system, but an IEEE1588 system would need hardware 
conversion to maintain the accuracy for an NTP timestamp.

So the question arises, should we make IEEE1588 or NTP the default for 
draft-ietf-mpls-loss-delay. Much as I would like to suggest that we 
should go back to NTP for consistency with other IETF protocols, I have 
difficulty reconciling this with the situation in network deployments 
and thus suggest that we continue to use IEEE1588.

What is the opinion of the working group?

Stewart (speaking as a draft editor)

---

From: "Shahram Davari" <[email-masked]>
Date: Thu, 3 Mar 2011 10:03:54 -0800

Hi Stewart,

Accurate delay measurement requires the Timestamp to be done in HW. It is true that most routers support NTP today, but the majority of those are really maintained in Software and AFAIK most HW based timestamps implementations are 1588 based and really the industry is shifting toward 1588 and SyncE for network synchronization. So I support 1588 as being the default.

Thanks,
Shahram

From: [email-masked] [mailto:[email-masked]] On Behalf Of Stewart Bryant
Sent: Thursday, March 03, 2011 8:07 AM
To: [email-masked]
Cc: [email-masked]
Subject: [TICTOC] draft-ietf-mpls-loss-delay Timestamp

We have received a LC comment on draft-ietf-mpls-loss-delay concerning the default timestamp.

In the first version of the draft we proposed NTP, but following initial comments from the MPLS-TP community we changed to IEEE1588. This requirement for IEEE1588 can be traced back to the choice of using IEEE1588 in Y.1731.

NTP is the "natural" choice for an IETF protocol. NTP is specified in IETF and is used in other MPLS protocols such as LSP ping. It is implemented on almost every host and every router. However in general the implementations provides a relatively low precision timestamp, and the NTP time distribution infrastructure operates on a best effort basis. Thus even a good client implementation would normally have a relatively low quality path to the server, which would result in a low quality of timestamp. NTP could be made to work to higher accuracy, but defacto upgrading NTP and providing high quality NTP paths to the time servers is not getting much attention. Computing timestamp differences is easier with NTP.

On the other hand IEEE1588 has defacto become the two way time tranfer protocol for precision applications. IEEE1588 only provides high quality time in well engineered networks with some form of hop by hop assistance. It is not widely implemented other than in equipment targeted to specific markets, although one of those markets is in mobile backhaul applications  where we expect to see significant initial deployment of MPLS-TP. Computing timestamp differences is harder with IEEE1588

The loss-delay work is targeting to be able to do one way packet delay measurement, so it does need a higher quality of timestamp than is required in general purpose network instrumentation.

Converting from IEEE1588 to NTP is not trivial, since they use different epochs, and different representations of sub-second time. In a "traditional" NTP implementation the time error due to on the fly conversion is likely to be small compared to the time error in the time synchronization system, but an IEEE1588 system would need hardware conversion to maintain the accuracy for an NTP timestamp.

So the question arises, should we make IEEE1588 or NTP the default for draft-ietf-mpls-loss-delay. Much as I would like to suggest that we should go back to NTP for consistency with other IETF protocols, I have difficulty reconciling this with the situation in network deployments and thus suggest that we continue to use IEEE1588.

---

From: [email-masked]
Date: Thu, 3 Mar 2011 10:26:00 -0800

My feeling is 1588 is the right answer in the long run since it is already
coupled to the hardware and widely deployed in networks using the "packet
clock derived time" in precise time and frequency delivery.

Pat

             "Shahram Davari"                                              
             <davari@broadcom.                                             
             com>                                                       To 
             Sent by:                  "[email-masked]"                
             tictoc-bounces@ie         <[email-masked]>,               
             tf.org                    "[email-masked]" <[email-masked]>     
                                                                        cc 
                                       "[email-masked]" <[email-masked]> 
             03/03/2011 10:05                                      Subject 
             AM                        Re: [TICTOC]                        
                                       draft-ietf-mpls-loss-delay          
                                       Timestamp                           

                                                                           

Hi Stewart,

Accurate delay measurement requires the Timestamp to be done in HW. It is
true that most routers support NTP today, but the majority of those are
really maintained in Software and AFAIK most HW based timestamps
implementations are 1588 based and really the industry is shifting toward
1588 and SyncE for network synchronization. So I support 1588 as being the
default.

Thanks,
Shahram

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Stewart Bryant
Sent: Thursday, March 03, 2011 8:07 AM
To: [email-masked]
Cc: [email-masked]
Subject: [TICTOC] draft-ietf-mpls-loss-delay Timestamp

We have received a LC comment on draft-ietf-mpls-loss-delay concerning the
default timestamp.

In the first version of the draft we proposed NTP, but following initial
comments from the MPLS-TP community we changed to IEEE1588. This
requirement for IEEE1588 can be traced back to the choice of using IEEE1588
in Y.1731.

NTP is the "natural" choice for an IETF protocol. NTP is specified in IETF
and is used in other MPLS protocols such as LSP ping. It is implemented on
almost every host and every router. However in general the implementations
provides a relatively low precision timestamp, and the NTP time
distribution infrastructure operates on a best effort basis. Thus even a
good client implementation would normally have a relatively low quality
path to the server, which would result in a low quality of timestamp. NTP
could be made to work to higher accuracy, but defacto upgrading NTP and
providing high quality NTP paths to the time servers is not getting much
attention. Computing timestamp differences is easier with NTP.

On the other hand IEEE1588 has defacto become the two way time tranfer
protocol for precision applications. IEEE1588 only provides high quality
time in well engineered networks with some form of hop by hop assistance.
It is not widely implemented other than in equipment targeted to specific
markets, although one of those markets is in mobile backhaul applications
where we expect to see significant initial deployment of MPLS-TP. Computing
timestamp differences is harder with IEEE1588

The loss-delay work is targeting to be able to do one way packet delay
measurement, so it does need a higher quality of timestamp than is required
in general purpose network instrumentation.

So the question arises, should we make IEEE1588 or NTP the default for
draft-ietf-mpls-loss-delay. Much as I would like to suggest that we should
go back to NTP for consistency with other IETF protocols, I have difficulty
reconciling this with the situation in network deployments and thus suggest
that we continue to use IEEE1588.

Stewart (speaking as a draft editor)
_______________________________________________
TICTOC mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/tictoc

---

From: Ron Cohen <[email-masked]>
Date: Thu, 3 Mar 2011 21:21:40 +0200

Hi,

I apologize in advance if my comments below where already been discussed.

The draft specify using PTPv1 timestamp format and not PTPv2 timestamp
format. The seconds part of the PTPv2 timestamp is 48bits, and not 32bits as
written in the draft. NTP timestamps will roll over in 2036. PTPv2
timestamps do not rollover.

I don't think its advisable to include timestamps that rolls over ~20 years
from now.

In addition, NTP timestamp, unlike PTP timestamp, 'jumps' in leap seconds
events. This makes calculation of time differences hard, and would render
some measurements ambiguous. It is much easier to use continuous timescale
like PTP.

In my opinion changing the timestamp format to PTPv2 timestamp format (i.e.
48bit/32bit sec/ns) should be the right way to go. I would also remove the
NTP timestamp format option to avoid leap seconds confusion and 2036
rollover events.

Best,
Ron

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 3 Mar 2011 21:35:42 +0200

Ron,
Lots of thanks for the important input.

I am somewhat less worried about the forthcoming rollover of NTP timestamps in 25 years from now. (Reminds me of Bug 2000:-). But leap seconds represent a serious technical point that cannot be ignored IMHO.

My 2c,
     Sasha

________________________________
From: [email-masked] [[email-masked]] On Behalf Of Ron Cohen [[email-masked]]
Sent: Thursday, March 03, 2011 9:21 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [TICTOC] draft-ietf-mpls-loss-delay Timestamp

Hi,

The draft specify using PTPv1 timestamp format and not PTPv2 timestamp format. The seconds part of the PTPv2 timestamp is 48bits, and not 32bits as written in the draft. NTP timestamps will roll over in 2036. PTPv2 timestamps do not rollover.

I don't think its advisable to include timestamps that rolls over ~20 years from now.

In addition, NTP timestamp, unlike PTP timestamp, 'jumps' in leap seconds events. This makes calculation of time differences hard, and would render some measurements ambiguous. It is much easier to use continuous timescale like PTP.

In my opinion changing the timestamp format to PTPv2 timestamp format (i.e. 48bit/32bit sec/ns) should be the right way to go. I would also remove the NTP timestamp format option to avoid leap seconds confusion and 2036 rollover events.

Best,
Ron

On Thu, Mar 3, 2011 at 8:26 PM, <[email-masked]<mailto:[email-masked]>> wrote:
My feeling is 1588 is the right answer in the long run since it is already
coupled to the hardware and widely deployed in networks using the "packet
clock derived time" in precise time and frequency delivery.

Pat

            "Shahram Davari"
            <davari@broadcom.
            com>                                                       To
            Sent by:                  "[email-masked]<mailto:[email-masked]>"
            tictoc-bounces@ie         <[email-masked]<mailto:[email-masked]>>,
            tf.org<http://tf.org>                    "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
                                                                       cc
                                      "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
            03/03/2011 10:05                                      Subject
            AM                        Re: [TICTOC]
                                      draft-ietf-mpls-loss-delay
                                      Timestamp

Hi Stewart,

Thanks,
Shahram

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of
Stewart Bryant
Sent: Thursday, March 03, 2011 8:07 AM
To: [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: [TICTOC] draft-ietf-mpls-loss-delay Timestamp

Stewart (speaking as a draft editor)
_______________________________________________
TICTOC mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tictoc

_______________________________________________
TICTOC mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tictoc

---

From: "Shahram Davari" <[email-masked]>
Date: Thu, 3 Mar 2011 11:45:03 -0800

Hi Ron,

Although the PTPv2 is 80 bytes, but I don't see a need to use all 80 bytes for delay measurement. The upper 2^32 seconds covers almost 136 years, which should be good enough to measure delay. Also lets' be compatible with Y.1731 format since that HW already exist.

Thx
Shahram

From: Ron Cohen [mailto:[email-masked]]
Sent: Thursday, March 03, 2011 11:22 AM
To: [email-masked]
Cc: Shahram Davari; [email-masked]; [email-masked]; [email-masked]
Subject: Re: [TICTOC] draft-ietf-mpls-loss-delay Timestamp

Hi,

Best,
Ron

Pat

            "Shahram Davari"
            <davari@broadcom.
            com>                                                       To

Hi Stewart,

Thanks,
Shahram

---

From: "Shahram Davari" <[email-masked]>
Date: Thu, 3 Mar 2011 11:54:57 -0800

Ron,

Sorry. I meant 80 bits (not bytes).

Thx
Shahram

From: [email-masked] [mailto:[email-masked]] On Behalf Of Shahram Davari
Sent: Thursday, March 03, 2011 11:45 AM
To: Ron Cohen; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [mpls] [TICTOC] draft-ietf-mpls-loss-delay Timestamp

Hi Ron,

Thx
Shahram

Hi,

Best,
Ron

Pat

            "Shahram Davari"
            <davari@broadcom.
            com>                                                       To

Hi Stewart,

Thanks,
Shahram

---

From: Ron Cohen <[email-masked]>
Date: Thu, 3 Mar 2011 23:35:11 +0200

Hi Shahram, I guess adding a requirement to take timestamp rollover into
account would do. If there is a strong reason to also leave NTP timestamp as
an option, a note explaining the ramification of leap seconds should be
added. I suggest that a continuous timescale (i.e. PTP) would be selected in
the draft as the default/must option. Best, Ron

On Thu, Mar 3, 2011 at 9:45 PM, Shahram Davari <[email-masked]> wrote:

> Hi Ron,
>
>
>
> Although the PTPv2 is 80 bytes, but I don’t see a need to use all 80 bytes
> for delay measurement. The upper 2^32 seconds covers almost 136 years, which
> should be good enough to measure delay. Also lets’ be compatible with Y.1731
> format since that HW already exist.
>
>
>
> Thx
>
> Shahram
>
>
>
> *From:* Ron Cohen [mailto:[email-masked]]
> *Sent:* Thursday, March 03, 2011 11:22 AM
> *To:* [email-masked]
> *Cc:* Shahram Davari; [email-masked]; [email-masked];
> [email-masked]
> *Subject:* Re: [TICTOC] draft-ietf-mpls-loss-delay Timestamp
>
>
>
> Hi,
>

---

From: "Bhatia, Manav (Manav)" <[email-masked]>
Date: Fri, 4 Mar 2011 05:48:41 +0530

I agree with this. NTP is mostly done in SW as against 1588 which is already supported in HW in most vendor platforms. Going by this, i would like to see 1588 as the default.

Cheers, Manav

________________________________
From: [email-masked] [mailto:[email-masked]] On Behalf Of Shahram Davari
Sent: Thursday, March 03, 2011 11.34 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [TICTOC] draft-ietf-mpls-loss-delay Timestamp

Hi Stewart,

Thanks,
Shahram

---

From: Benjamin Niven-Jenkins <[email-masked]>
Date: Thu, 3 Mar 2011 21:27:02 +0000

Stewart,

IMO a default serves two purposes:

1) A baseline everyone must implement
2) A reduction in the amount of additional configuration to get something working.

NTP arguably covers (1) better because as the draft notes it has had wide deployment on the Internet. 

However I would think (and I'm no expert so may be wrong) that NTP has not generally been used for the sort of use cases that draft-ietf-mpls-loss-delay attempts to cover and I do not think it would provide the level of accuracy expected by Transport Network Operators.

If NTP were the default then I would expect that in the majority of cases Transport Network Operators would immediately change from the NTP default to 1588. The one case I would think where maybe operators would stick with NTP is if their equipment cannot for some reason support 1588 but does support NTP. Even in those cases I expect the operator may well just choose to not use draft-ietf-mpls-loss-delay at all as measurements that aren't sufficiently accurate for a transport network operator could be considered worse than no measurements at all. So IMO 1588 covers (2) better and it is not a new protocol so 1588 has also seen quite a bit of deployment AFAIK.

Therefore the only case where making NTP the default makes sense IMO is where it is felt that implementing 1588 is so burdensome that a significant proportion of equipment will exist that will only ever be capable of supporting NTP timestamps and that deployers of draft-ietf-mpls-loss-delay would accept the level of accuracy provided by NTP. I am not the right person to judge whether that would be the case but my gut tells me that it won't.

So I'd vote for 1588 to be the default.

Ben

On 3 Mar 2011, at 16:06, Stewart Bryant wrote:

> mpls mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mpls

---

From: "Sprecher, Nurit (NSN - IL/Hod HaSharon)" <[email-masked]>
Date: Fri, 4 Mar 2011 11:32:07 +0100

Stewart hi,

The accuracy targets as described below are approaching a level where
on-path support is needed. 

In this case the clear choice is IEEE 1588 because it specifies on-path
support whereas NTP does not. 

Even without on-path support IEEE 1588 could be better if better
accuracy is pursued by sending multiple timing packets per second per
slave/client, because standard NTP clients send messages at a very low
rate for saving server resources.

I support the default of IEEE1588!

Best regards,

Nurit

From: [email-masked] [mailto:[email-masked]] On Behalf Of
ext Stewart Bryant
Sent: Thursday, March 03, 2011 6:07 PM
To: [email-masked]
Cc: [email-masked]
Subject: [mpls] draft-ietf-mpls-loss-delay Timestamp

---

From: "Mallette, Edwin" <[email-masked]>
Date: Mon, 7 Mar 2011 09:44:57 -0500

I concur with the points made by Nurit and Ben.  In addition there is the asymmetric delay problems (for a given link) that lots of work has gone into to solve with 1588 - 1588 Transparent Clock / 1588 residency time compensation.  To the best of my knowledge this is not being addressed for NTP.

Thus +1 for the default of IEEE1588.

Ed

From: "Sprecher, Nurit (NSN - IL/Hod HaSharon)" <[email-masked]<mailto:[email-masked]>>
Date: Fri, 4 Mar 2011 05:32:07 -0500
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Cc: "Zhao, Peng (NSN - CN/Shanghai)" <[email-masked]<mailto:[email-masked]>>, "Pietilainen, Antti (NSN - FI/Espoo)" <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [mpls] draft-ietf-mpls-loss-delay Timestamp

Stewart hi,
The accuracy targets as described below are approaching a level where on-path support is needed.
In this case the clear choice is IEEE 1588 because it specifies on-path support whereas NTP does not.
Even without on-path support IEEE 1588 could be better if better accuracy is pursued by sending multiple timing packets per second per slave/client, because standard NTP clients send messages at a very low rate for saving server resources.
I support the default of IEEE1588!
Best regards,
Nurit

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of ext Stewart Bryant

________________________________
CONFIDENTIALITY NOTICE: This e-mail may contain information that is privileged, confidential or otherwise protected from disclosure. If you are not the intended recipient of this e-mail, please notify the sender immediately by return e-mail, purge it and do not disseminate or copy it.

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 07 Mar 2011 18:19:33 -0500

In message <[email-masked]>
"Mallette, Edwin" writes:
>  
> I concur with the points made by Nurit and Ben.  In addition there is
> the asymmetric delay problems (for a given link) that lots of work has
> gone into to solve with 1588 - 1588 Transparent Clock / 1588 residency
> time compensation.  To the best of my knowledge this is not being
> addressed for NTP.
>  

NTP has always, probably from day one but at least going back to the
early 1990s (AFAIK), compensated for asymetric delay.

With hardware assist, NTP deployed over a link is every bit as
accurate as PTP deployed over a link with hardware assist.  It is
probably more accurate due to the jitter compensation used by NTP and
omited by PTP.  Some hardware doesn't quite get all the jitter out.

Admitedly, NTP is more complex due to the compensation that is
included in NTP and not in PTP.

OTOH if there is something that favors PTP, then we should approach
the TICTOC WG and pass along that requirements.  For example, a two
step transmit, allowing the transmit timestamp on a given packet to be
sent with the next packet, may simplify hardware.  If so, we ask
TICTOC to support that in NTP.

I support making the default NTP.  This is the IETF and given a very
nearly equivalent IETF solution and an IEEE solution we should require
the IETF solution, make the IEEE solution optional, and if there are
any shortcomings to the IETF solution, address that in the appropriate
IETF WG.

Curtis

> From: "Sprecher, Nurit (NSN - IL/Hod HaSharon)"
>        <[email-masked]<mailto:[email-masked]>>
> To: "[email-masked]<mailto:[email-masked]>"
>        <[email-masked]<mailto:[email-masked]>>,
>        "[email-masked]<mailto:[email-masked]>"
>        <[email-masked]<mailto:[email-masked]>>
> Cc: "Zhao, Peng (NSN - CN/Shanghai)"
>        <[email-masked]<mailto:[email-masked]>>, "Pietilainen,
>        Antti (NSN - FI/Espoo)"
>        <[email-masked]<mailto:[email-masked]>>,
>        "[email-masked]<mailto:[email-masked]>"
>        <[email-masked]<mailto:[email-masked]>>
> From: [email-masked]<mailto:[email-masked]>
    [mailto:[email-masked]] On Behalf Of ext Stewart Bryant
> In the first version of the draft we proposed NTP, but following
> initial comments from the MPLS-TP community we changed to
> IEEE1588. This requirement for IEEE1588 can be traced back to the
> choice of using IEEE1588 in Y.1731.
>  
> quality of timestamp. NTP could be made to work to higher accuracy,
> but defacto upgrading NTP and providing high quality NTP paths to the
> time servers is not getting much attention. Computing timestamp
> differences is easier with NTP.
>  
> protocol for precision applications. IEEE1588 only provides high
> quality time in well engineered networks with some form of hop by hop
> targeted to specific markets, although one of those markets is in
> mobile backhaul applications where we expect to see significant
> initial deployment of MPLS-TP. Computing timestamp differences is
> harder with IEEE1588
>  
> Converting from IEEE1588 to NTP is not trivial, since they use
> different epochs, and different representations of sub-second time. In
> a "traditional" NTP implementation the time error due to on the fly
> conversion is likely to be small compared to the time error in the
> time synchronization system, but an IEEE1588 system would need
> hardware conversion to maintain the accuracy for an NTP timestamp.
>  
> should go back to NTP for consistency with other IETF protocols, I
> have difficulty reconciling this with the situation in network
> deployments and thus suggest that we continue to use IEEE1588.
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 8 Mar 2011 13:17:03 +0200

Curtis, and all,
To the best of my understanding, this is not about accuracy of synchronization using NTP or PTP.
It is about the default timestamp format in the Delay Measurement (DM) messages for MPLS.

Whatever accuracy can be achieved with one  of these, the timestamp format is probably the least significant factor affecting the result.

I have already stated on this list that, IMHO, the question itself is moot, because the DM can operate with disparate timestamp formats at two ends: actual format is specified in the message, and there is no need to do conversion in HW or in hard real time. What's more, 2-way DM does not even require conversion from one timestamp format to another, just ability to compute differences between two timestamps in the same format both for NTP and PTP formats.

In addition, the points in the data path where the timestamps are captured by a HW assist for the needs of a given protocol do not necessarily match the points where these timestamps should be taken for the purpose of DM. While it is probably possible to live with DM that ignores the time a packet is queued for transmission in the originating LSR, the results could look fishy, especially if the objective is to measure PDV and not delay per se. In other words, a timestamping HW assist that is optimal from the point of view of PTP "as close to actual ingress/egress as possible"), may be not very effective for DM (and, of course, vice versa). 

My 2c,
     Sasha

> -----Original Message-----
> Curtis Villamizar
> Sent: Tuesday, March 08, 2011 1:20 AM
> To: Mallette, Edwin
> Cc: [email-masked]; Zhao, Peng (NSN - CN/Shanghai); Pietilainen, Antti
> (NSN - FI/Espoo); [email-masked]; [email-masked]

---

From: Alexander Vainshtein <[email-masked]>
Date: Tue, 8 Mar 2011 22:13:06 +0200

Curtis,
Lots of thanks for a prompt and very detailed response.

1. I have proposed the "send what you can, accept both" approach in my earlier email to the TICTOC list
(http://www.ietf.org/mail-archive/web/tictoc/current/msg00877.html). So it seems that we are on the same wavelength.

2. The "timestamp capture point in the data path" was not about the timestamp location in the message.
It is about the stage of the data processing when the timestamp can be captured. E.g., some designs use commercial HW that captures the timestamps between PHY and MAC. This is OK for PTP and NTP, but excludes the time spent in the egress queue of the node from delay measurement.

3. You have said that 20 microseconds look to you as very good accuracy for delay measurements. This probably depends on the application. E.g., if we must synchronize two base stations within 2-3 microseconds of relative ToD error, this accuracy will hardly help you to understand why your synchronization mechanisms do not work:-).
I am somewhat suspicious about a measurement procedure that does not specify accuracy explicitly. But I understand the position taken by the draft authors.

Regards,
     Sasha

> From: [email-masked] [mailto:[email-masked]]
> Sent: Tuesday, March 08, 2011 9:37 PM
> To: Alexander Vainshtein
> Cc: [email-masked]; [email-masked]; Zhao, Peng (NSN - CN/Shanghai);
> Pietilainen, Antti (NSN - FI/Espoo); [email-masked];
> [email-masked]; Mallette, Edwin
> In message
> <[email-masked]>
> Alexander Vainshtein writes:
> >
> > Curtis, and all,
> >
> > To the best of my understanding, this is not about accuracy of
> > synchronization using NTP or PTP.  It is about the default timestamp
> > format in the Delay Measurement (DM) messages for MPLS.
> >
> > Whatever accuracy can be achieved with one of these, the timestamp
> > format is probably the least significant factor affecting the result.
>
> For very high timestamp accuracy, the timestamp must be generated in
> hardware.  Perhaps the answer is send either format (maximize the
> supported hardware) and accept both.
>
> On the receive side, the locally generated time stamp can be converted
> or the sent timestamp can be converted to achieve a common format.
>
> It is the transmit timestamp that is hard to get into a format that
> the hardware does not support.
>
> > I have already stated on this list that, IMHO, the question itself is
> > moot, because the DM can operate with disparate timestamp formats at
> > two ends: actual format is specified in the message, and there is no
> > need to do conversion in HW or in hard real time. What's more, 2-way
> > DM does not even require conversion from one timestamp format to
> > another, just ability to compute differences between two timestamps
> in
> > the same format both for NTP and PTP formats.
>
> It seems that the timestamp fields are nearly identical.
> IEEE-1588-2008 section 5,2,3 contains:
>
>   struct Timestamp {
>       UInteger48 secondsField;
>       UInteger32 nanosecondsFeild;
>   };
>
> Hopefully IEEE will see this excerpt as an acceptable "fair use".
>
> This is very similar to the unix/bsd/linux timespec.
>
>   struct timespec {
>       time_t  tv_sec;    /* seconds */
>       long    tv_nsec;   /* and nanoseconds */
>   };
>
> The NTP 64 bit timestamp is also quite similar (RFC5905).
>
>        0                   1                   2                   3
>        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |                            Seconds                            |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |                            Fraction                           |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
>                              NTP Timestamp Format
>
> Briefly (from RFC5905) "It includes a 32-bit unsigned seconds field
> spanning 136 years and a 32-bit fraction field resolving 232
> picoseconds."  The resolution of NTP 64 bit timestamp is slightly
> better by about a factor of four.
>
> The draft-ietf-mpls-loss-delay-01 draft chops off the top 16 bits and
> therefore is really using unix/bsd/linux timespec format.
>
> The conversion for PTP to timespec is tacking on or removing 16 bits
> of leading zeros.  That conversion will work for the next 90+ years,
> which gives us a lot of time to decide how to extend time_t and
> timespec and all the things that go with it, however NTP has this
> covered in the 128 bit format when we go beyond era 0.
>
> Is there an endian-ness issue that I'm missing?  In any case, this is
> acheivable as long as we specify "send either" and "accept both".
>
> Perhaps the expected lifetime of the solar system would be an adequate
> target, but 2106 may be OK for now so NTP 64 bit format is OK.  :-)
>
> OTOH, NTP also supports a 32 bit stamp:
>
>        0                   1                   2                   3
>        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>       |          Seconds              |           Fraction            |
>       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
>
> Again from RFC5905 "The 32-bit short format is used in delay and
> dispersion header fields where the full resolution and range of the
> other formats are not justified.  It includes a 16-bit unsigned
> seconds field and a 16-bit fraction field."
>
> 1 second / 65536 is less than 20 usec.  This should be fine in delay
> measurements.  It is less than 4km worth of fiber if I've done the
> math right.  Otherwise we stick with the 64 bit format only.
>
> I should also point out that 20 usec accuracy may be within the range
> acheivable in software, or at least packet processor microcode.
>
> As spcified in draft-ietf-mpls-loss-delay-01 the 32 bit timestamp
> would be padded out to 64 bits, so there is no motivation to use it.
>
> > In addition, the points in the data path where the timestamps are
> > captured by a HW assist for the needs of a given protocol do not
> > necessarily match the points where these timestamps should be taken
> > for the purpose of DM. While it is probably possible to live with DM
> > that ignores the time a packet is queued for transmission in the
> > originating LSR, the results could look fishy, especially if the
> > objective is to measure PDV and not delay per se. In other words, a
> > timestamping HW assist that is optimal from the point of view of PTP
> > "as close to actual ingress/egress as possible"), may be not very
> > effective for DM (and, of course, vice versa).
>
> We have been specifying hardware that allows the PTP and NTP 64 bit
> and 32 bit time stamp formats to be placed at an arbitrary byte
> location.  Packets from CPU to hardware have an internal header that
> gets stripped before going out and one thing that can go in that
> header is an indication of time stamp format and where to put it (byte
> offset).  Final modifications for NTP, PTP, LM, DM, happen there.
>
> I am aware that some "merchant silicon" has some built in assumptions
> about how these timestamps will be used and may only be able to
> support hardware asssisted time synchronization using PTP over
> Ethernet.  That does not mean we should limit our specs in IETF to use
> PTP formats only.
>
> > My 2c,
> >      Sasha
>
> Therefore I support "send any", and "accept all" as the MUST.  This
> guarentees interoperability and removes any need to negociate formats.
> As long as hardware can record stamps, software can convert them and
> produce a delay value.
>
> Thanks,
>
> Curtis
>
> > > > the asymmetric delay problems (for a given link) that lots of
> work
> > > has
> > > > slave/client, because standard NTP clients send messages at a
> very
> > > low
> > > > the implementations provides a relatively low precision
> timestamp,
> > > and
> > > > relatively low quality path to the server, which would result in
> a
> > > low
> > > > different epochs, and different representations of sub-second
> time.
> > > In
> > > > So the question arises, should we make IEEE1588 or NTP the
> default
> > > for

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 08 Mar 2011 14:37:17 -0500

In message <[email-masked]>

> another, just ability to compute differences between two timestamps in

  struct Timestamp {
      UInteger48 secondsField;

  struct timespec {

       0                   1                   2                   3

                             NTP Timestamp Format

       0                   1                   2                   3

Thanks,

Curtis

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 08 Mar 2011 19:06:08 -0500

In message <[email-masked]>
Alexander Vainshtein writes:
> (http://www.ietf.org/mail-archive/web/tictoc/current/msg00877.html). So
> it seems that we are on the same wavelength.

Agree.

> 2. The "timestamp capture point in the data path" was not about the
> timestamp location in the message.  It is about the stage of the data
> processing when the timestamp can be captured. E.g., some designs use
> commercial HW that captures the timestamps between PHY and MAC. This
> is OK for PTP and NTP, but excludes the time spent in the egress queue
> of the node from delay measurement.

That is why in NTP, PTP, and draft-ietf-mpls-loss-delay there are four
timestamps.  The total round trip delay subtracts the third and second
to eliminate queueing and turn around.

> 3. You have said that 20 microseconds look to you as very good
> accuracy for delay measurements. This probably depends on the
> application. E.g., if we must synchronize two base stations within 2-3
> microseconds of relative ToD error, this accuracy will hardly help you
> to understand why your synchronization mechanisms do not work:-).  I
> am somewhat suspicious about a measurement procedure that does not
> specify accuracy explicitly. But I understand the position taken by
> the draft authors.

20 usec is OK for draft-ietf-mpls-loss-delay type measurements.

draft-ietf-mpls-loss-delay is not used to synchronize clocks.

For that NTP is used and that uses the 64 fit format for that
purpose.  The 64 bit time stamp is relative to the NTP era.  We are in
era zero until the year 2106.

You seem to misinterpret what I said 20 usec is OK for.

As I mentioned, NTP is good to just under 1 nsec if the hardware is
dead accurate in producing both transmit and receive time stamps.  If
there is some inaccuracy in the timestamping, then NTP is better
equiped to remove that inaccuracy (using multiple samples) than PTP
is.

BTW - If all that is needed is an accurate oscillator (for example for
CE or mobile backhaul), then a slightly assymetric path is OK.

Curtis

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 9 Mar 2011 15:03:12 +0200

Curtis,
Lots of thanks for a prompt and delayed response.

Please see some comments inline below. I've shipped the next that is not related to these comments.

Regards,
     Sasha

> Sent: Wednesday, March 09, 2011 2:06 AM
> To: Alexander Vainshtein
--- snipped ---
> to eliminate queuing and turn around.
[[[Sasha]]] Please consider the following use case:
- I am going to use a certain LSP as a tunnel for TDM PWs
- I would like to measure packet delay *variation* (PDV) introduced by this LSP in order to configure the jitter buffer of these TDM PWs. I expect to do that by measuring delay for multiple samples and computing the PDV as (Max. measured delay - Min. measured delay)
- If queuing delay at the LSP head- and tail-nodes is not accounted in my delay measurements, my PDV estimation will not account for that also (because queuing only increases Max. measured delay)
- As a consequence, my PDV estimation may be not suitable for configuring the jitter buffer correctly. As a consequence, my TDM customers will see errors in their traffic when TDM PW packets that arrive too late to be accommodated into the jitter buffer are discarded and their payload replaced with "all ones".
Do I miss something here?

> 20 usec is OK for draft-ietf-mpls-loss-delay type measurements.
[[[Sasha]]] I would be happy to accept that. But the draft does not yield this (or any other) number.
--- snipped to the end ---