Subject: Re: [Ntp] Drafts on interleaved modes and correction field

From: Tal Mizrahi <[email-masked]>
Date: Wed, 1 Nov 2017 11:41:26 +0200

Small correction:
Slow-to-fast cut-through is *not* possible.
Fast-to-slow cut-through *is* possible.

I had it reversed in my previous mail. :)

Regards,
Tal.

On Wed, Nov 1, 2017 at 8:48 AM, Tal Mizrahi <[email-masked]>
wrote:

> Hi Miroslav,
>
> >> Moreover, we never know which nodes along the path use
> store-and-forward,
> >> and which nodes use cut-through. In cut-through forwarding the packet
> >> transmission may start before the last bit is received, so it certainly
> >> makes more sense to measure the first-bit-in-to-first-bit-out.
> >
> >That's a good point. With cut-through switching the correction would
> >be negative and generally couldn't be determined in time to be put in
> >the packet as the length is unknown. Are there any switches that can
> >cut-through from a faster link to slower link?
>
> To the best of my knowledge cut-through from a fast link to a slow link is
> not possible. Cut-through from a slow link to a fast link is certainly
> possible, and of course between ports that have the same link speed.
> In the two latter cases the transmission may start before the end of the
> reception.
>
> Regards,
> Tal.
>
>
>
> On Tue, Oct 31, 2017 at 5:57 PM, Miroslav Lichvar <[email-masked]>
> wrote:
>
>> On Tue, Oct 31, 2017 at 04:59:52PM +0200, Tal Mizrahi wrote:
>> > >With NTP there is generally no HW support, so clients and servers
>> > >should use trailer RX timestamps and preamble TX timestamps to avoid
>> > >an asymmetric delay with different link speeds. The following document
>> > >explains the issue nicely.
>> > >
>> > >https://www.eecis.udel.edu/~mills/stamp.html
>> > >
>> > >Do you think it will be difficult to implement? If the link speed and
>> > >length of the packet is known, the timestamp can be transposed.
>> >
>> > The link speed is not necessarily known to the end points.
>>
>> I meant that the devices which are supposed to modify the correction
>> field know the link speed and can transpose preamble timestamps if
>> necessary.
>>
>> The end points don't need to know the link speed on the path. Just the
>> speed of their own link if they need to transpose their timestamps.
>>
>> > Furthermore, the
>> > link speed may change several times along the path.
>>
>> In such case we want the correction to still be symmetric, right?
>>
>> > Maybe I am missing something, but it appears that when measuring
>> > last-bit-in-to-first-bit-out we are ignoring the packet reception time;
>> it
>> > seems to make sense to include the packet reception time in the
>> residence
>> > time.
>>
>> For PTP yes, but not for NTP. The reception time is included in the
>> network delay as measured by the client, which will be symmetric as
>> long as no device on the path includes in the correction the
>> transmission time or reception time.
>>
>> > Moreover, we never know which nodes along the path use
>> store-and-forward,
>> > and which nodes use cut-through. In cut-through forwarding the packet
>> > transmission may start before the last bit is received, so it certainly
>> > makes more sense to measure the first-bit-in-to-first-bit-out.
>>
>> That's a good point. With cut-through switching the correction would
>> be negative and generally couldn't be determined in time to be put in
>> the packet as the length is unknown. Are there any switches that can
>> cut-through from a faster link to slower link?
>>
>> Maybe it would make sense to allow corrections from preamble RX
>> timestamps when the link speeds are equal.
>>
>> > >The reason for a separate transmit correction is to support the
>> > >interleaved modes. The receive correction might not be necessary. If
>> > >you have a better use for the 2 octets, I think it could be dropped
>> > >and included in the origin correction instead.
>> >
>> > So it looks like two correction fields would suffice: one for the
>> forward
>> > direction, and one for the reverse direction. Right?
>>
>> I think it's three fields if we want both support the interleaved
>> modes and allow the client to check the delay correction in both
>> directions. Where would be the transmit correction included if there
>> were only two fields?
>>
>> --
>> Miroslav Lichvar
>>
>
>

---

From: Miroslav Lichvar <[email-masked]>
Date: Mon, 6 Nov 2017 13:36:44 +0100

> Fast-to-slow cut-through *is* possible.

Do you know if there are any switches that actually support it? I
suspect the buffering would be a major complication for a relatively
small improvement in the overall latency.

If a switch cut through from a faster link to a slower link, it would
know the length of the packet and could calculate the (negative)
correction before the correction field was transmitted, right? I
assume that the field is in the second half of the packet and the
ratio between the speeds is at least 2:1.

I think the draft can specify that the correction needs to be
calculated from the end of the reception and beginning of the
transmission if the link speeds are different. If the speeds are
equal, it can be any combination of beginning/end of the
reception/transmission, because a symmetric correction will be made in
the opposite direction. The delay will be different, but there won't
be any asymmetry.

Does that make sense?

-- 
Miroslav Lichvar

---

From: Tal Mizrahi <[email-masked]>
Date: Wed, 8 Nov 2017 19:32:07 +0200

Hi Miroslav,

> Do you know if there are any switches that actually support it?

Yes, some switches actually support it fast-to-slow cut-through.

>correction before the correction field was transmitted, right?

Hmmm... So the switch would have to evaluate the packet length based on the
Length field in the UDP header, and use that in the correction field
computation.
It would be significantly easier to implement a correction field
computation based on first-bit-in-to-first-bit-out.

I may be missing something here, but I would argue that if we consistently
compute the correction based on first-bit-in-to-first-bit-out, the packet
transmission times on both directions would cancel out, and would not
affect the symmetry of the computation.
Can you provide an example in which this hypothesis is wrong?

Thanks,
Tal.

On Mon, Nov 6, 2017 at 2:36 PM, Miroslav Lichvar <[email-masked]>
wrote:

> > > On Tue, Oct 31, 2017 at 5:57 PM, Miroslav Lichvar <[email-masked]
> >
> > > wrote:
> > >

---

From: Miroslav Lichvar <[email-masked]>
Date: Thu, 9 Nov 2017 10:38:51 +0100

> Length field in the UDP header, and use that in the correction field

Good point. Using the UDP length field should be simpler and give the
switch more time to calculate the correction than waiting for the
end of the received frame.

> computation based on first-bit-in-to-first-bit-out.

I understand that. The question is if it's worth the incompatibility
that it would create.

> affect the symmetry of the computation.

Yes, but only if the client, server and all switches with different
link speeds between them used the beginning of the reception too.

The clients and servers would have to switch between two modes of
timestamping. One for networks where the correction fields is not
supported and one for networks where all switches (with different link
speeds) support it. It's all or nothing. 

On the other hand, a correction field based on the end of the
reception is compatible with timestamping used by existing clients and
servers, and a gradual replacement of switches in the network does not
create an asymmetry.

> Can you provide an example in which this hypothesis is wrong?

Ok, let's say there is a server on 100 Mbit link, a client on a 1000
Mbit link, and a single switch (store-and-forward) between them. The
propagation delay is 100 nanoseconds on both links and the switch has
a constant asymmetric delay of 500/1000 nanoseconds. The client sends
a 1000 bit frame at time X and the server responds with a frame of the
same length at Y. All timestamping is perfectly accurate. 

The intervals of the receptions and transmissions are:

client TX: [X, X+1000]
switch RX: [X+100, X+1100]
switch TX: [X+1600, X+11600]
server RX: [X+1700, X+11700]
server TX: [Y, Y+10000]
switch RX: [Y+100, Y+10100]
switch TX: [Y+11100, Y+12100]
client RX: [Y+11200, Y+12200]

If the correction field was ignored, the delays of the request and
response would be 11700 and 12200 nanoseconds respectively. The
calculated offset would have an error of 250 nanoseconds.

If the switch calculated the corrections from the beginning of the
receptions, the delays would be 10200 and 1200 nanoseconds. The error
in the offset would be 4500 nanoseconds.

If the switch calculated the corrections from the end of the
reception, the delays would be 11200 and 11200 nanoseconds. The offset
would have no error.

This shows that the switch has to timestamp the same point of the
frame as the server and client. Otherwise, the correction field may
actually increase the error in the measured offset.

-- 
Miroslav Lichvar

---

From: Tal Mizrahi <[email-masked]>
Date: Thu, 9 Nov 2017 15:11:46 +0200

Hi Miroslav,

Very nice example.
It clarifies your point.

I believe the inconsistency you raised here could be resolved if both the
client and server would measure all timestamps as the time-of-the-first-bit
(both reception and transmission).
If that would be the case, then (using your example):

Client TX timestamp = X
Switch (client->server) correction field = 1500
Server RX timestamp = X+1700

Server TX timestamp = Y
Switch (server->client) correction field = 11000
Client RX timestamp = Y+11200

In this case the client would compute the delay on both directions to be
200 nanoseconds, as desired.
Again this all works only if the client / server use the first bit as the
reference for both TX and RX timestamps.

Cheers,
Tal.

---

From: Greg Dowd <[email-masked]>
Date: Thu, 9 Nov 2017 16:18:12 +0000

As I recall, this all works if every device in the network are timestamp correction aware.  Tal, I think you have probably been involved in many discussions of this type, .  I tend to believe that OSI model implies that we should not be too focused on specifying which bit of which symbol is selected.

My recommendation is that we call out something like 802.3bf (?) and provide examples of the asymmetry/implementation issues associated with selections.  The reality is that timestamps are struck and are “fixed up” to be SOH or ETB in hardware anyway.  The current NTP protocol has already defined the “default” as beginning of tx/end of rx historically.

What do you think?

Thanks…Greg

From: ntp [mailto:[email-masked]] On Behalf Of Tal Mizrahi
Sent: Thursday, November 9, 2017 5:12 AM
To: Miroslav Lichvar <[email-masked]>
Cc: Aanchal Malhotra <[email-masked]>; [email-masked]
Subject: Re: [Ntp] Drafts on interleaved modes and correction field

EXTERNAL EMAIL
Hi Miroslav,

Very nice example.
It clarifies your point.

I believe the inconsistency you raised here could be resolved if both the client and server would measure all timestamps as the time-of-the-first-bit (both reception and transmission).
If that would be the case, then (using your example):

Client TX timestamp = X

Server TX timestamp = Y

In this case the client would compute the delay on both directions to be 200 nanoseconds, as desired.
Again this all works only if the client / server use the first bit as the reference for both TX and RX timestamps.

Cheers,
Tal.

On Thu, Nov 9, 2017 at 11:38 AM, Miroslav Lichvar <[email-masked]<mailto:[email-masked]>> wrote:

client TX: [X, X+1000]

--
Miroslav Lichvar