Subject: Re: [TICTOC] Enterprise profile update

From: Richard Cochran <[email-masked]>
Date: Sat, 5 Apr 2014 13:20:35 +0200

Doug,

On Wed, Mar 12, 2014 at 12:41:33PM -0700, Douglas Arnold wrote:

> 1. Elimination of the Unicast only and Multicast only modes of operation.
>  The only mode of operation is the hybrid multicast/unicast mode.  In this
> mode the master sends sync and announce as multicast, and it responds to
> delay request messages "in kind".  In other words a delay response is
> unicast if the delay requests is unicast, and multicast if the delay
> requests is multicast.

Nice. This idea makes way more sense than the unicast option in 1588,
IMHO.

> Please take a look and tell me what you think?

Typo:

3.  Technical Terms

    ...

    Unicast Discovery: A mechanism for PTP slaves to establish a
    unicast communication with PTP masters using a configures table of
    master IP addresses and Unicast Message Negotiation.

Did you mean "configuration"?

Thanks
Richard

---

From: Richard Cochran <[email-masked]>
Date: Sat, 5 Apr 2014 15:27:51 +0200

3.  Technical Terms

     ...

     PTP port: An interface of a PTP clock with the network.  Note that
     there may be multiple PTP ports running on one physical interface,
     for example a unicast slave which talks to several Grandmaster
     clocks in parallel.

Really? I reviewed all of the 1588 unicast options, and I could not
find any text that implied using unicast results in a new port. Can
you explain what you mean by "multiple ports"?

Thanks,
Richard

---

From: Richard Cochran <[email-masked]>
Date: Sat, 5 Apr 2014 15:37:48 +0200

> Nice. This idea makes way more sense than the unicast option in 1588,

On second thought, why doesn't this profile simply say that slaves may
use 1588 section 16.1 to negotiate unicast Delay_Resp messages?

As is, this profile mandates behavior WRT unicast that isn't really
allowed by 1588, unless you say that it falls under the very vague and
open ended 7.3.1 clause.

I would prefer the common sense unicast handling in your profile, but
unfortunately 1588 already specifies something else.

Thanks
Richard

---

From: Kevin Gross <[email-masked]>
Date: Sun, 6 Apr 2014 23:59:21 -0600

I can't remember if this was discussed in TICTOCK yet. Apologies if I'm
bringing up a settled point.

I appreciate the scalability of unicast delay request/response but I want
to make sure everyone is aware of a hazard with hybrid mode. On some
networks, unicasts and multicasts are delivered through separate network
paths. On these networks, half the round trip delay of a delay
request/response unicast exchange will not be a good estimate of sync
multicast propagation time.

It may be best to leave a multicast delay request/response option for
devices that need higher accuracy or want to have the option to compare
unicast and multicast delay request/response performance to determine
whether hybrid mode is viable on the network they are connected through.

Kevin Gross - AVA Networks

On Sat, Apr 5, 2014 at 7:37 AM, Richard Cochran <[email-masked]>wrote:

> _______________________________________________
> TICTOC mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tictoc
>

---

From: Douglas Arnold <[email-masked]>
Date: Mon, 7 Apr 2014 22:48:35 +0200

Hi Kevin,

This is a good point and ma require some warning text be added to the draft

Doug

-- 
Doug

---

From: "Laurent Montini (lmontini)" <[email-masked]>
Date: Tue, 8 Apr 2014 12:40:51 +0000

Hi Kevin,

Assuming that the IP mcast tree is still built on RPF check, in what scenarios are you considering the ucast path differing from mcast path?

I don’t say this can not happen (e.g. ECMP or LAG) but I’d like to get more precise use cases.

Moreover based on this RPF check, the mcast return path for delay_Req can still be different.
Delay_Resp should take the same path than Sync, Announce.

Thanks,
Laurent

From: Kevin Gross <[email-masked]<mailto:[email-masked]>>
Date: Monday 7 April 2014 07:59
To: Richard Cochran <[email-masked]<mailto:[email-masked]>>
Cc: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [TICTOC] Enterprise profile update

I can't remember if this was discussed in TICTOCK yet. Apologies if I'm bringing up a settled point.

I appreciate the scalability of unicast delay request/response but I want to make sure everyone is aware of a hazard with hybrid mode. On some networks, unicasts and multicasts are delivered through separate network paths. On these networks, half the round trip delay of a delay request/response unicast exchange will not be a good estimate of sync multicast propagation time.

It may be best to leave a multicast delay request/response option for devices that need higher accuracy or want to have the option to compare unicast and multicast delay request/response performance to determine whether hybrid mode is viable on the network they are connected through.

On Sat, Apr 5, 2014 at 7:37 AM, Richard Cochran <[email-masked]<mailto:[email-masked]>> wrote:

Thanks
Richard

_______________________________________________
TICTOC mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tictoc

---

From: Wojciech Owczarek <[email-masked]>
Date: Tue, 8 Apr 2014 15:31:58 +0200

I would say that with the latest generations of network kit, the unicast vs. multicast latency differences start to dither and those asymmetries will eventually be a thing of the past - yet perhaps it might be very worthwhile to add some advisory wording.

As a very general comment I will add that you cannot substitute standards for good network design and architecture practices - and I think that all those issues such as asymmetric paths, routing layout etc., are a matter of network design more than anything else. When you deploy the enterprise profile, it's under the assumption that you know how your network is designed and you are aware of the caveats resulting from it. To me, multicast distribution and routing is easy to manage, but I work in a multicast-centric environment.

Nevertheless, I do support the call for allowing the mixed unicast/multicast operation, but not exclusively - also allowing to use pure multicast operation. From the implementation perspective this is simple: if the PTP_UNICAST flag of the incoming delayReq was true, you reply to the source. If not, you reply to the common multicast group. We have been doing that in ptpd for quite a while now.

Regards,
Wojciech

-
Wojciech Owczarek

On 8 Apr 2014, at 14:40, "Laurent Montini (lmontini)" <[email-masked]> wrote:

> From: Kevin Gross <[email-masked]>
> To: Richard Cochran <[email-masked]>
> Cc: "[email-masked]" <[email-masked]>
> On Sat, Apr 5, 2014 at 7:37 AM, Richard Cochran <[email-masked]> wrote:

---

From: Richard Cochran <[email-masked]>
Date: Tue, 8 Apr 2014 16:45:27 +0200

On Tue, Apr 08, 2014 at 03:31:58PM +0200, Wojciech Owczarek wrote:

> Nevertheless, I do support the call for allowing the mixed
> unicast/multicast operation, but not exclusively - also allowing to
> use pure multicast operation. From the implementation perspective
> this is simple: if the PTP_UNICAST flag of the incoming delayReq was
> true, you reply to the source. If not, you reply to the common
> multicast group. We have been doing that in ptpd for quite a while
> now.

But why not simply do what it says to do in 1588-2008 Section 16.1 ?

[ This question is directed at both ptpd and the new profile. ]

If the profile and the implementations follow the standard, then it
makes it far easier for different products to inter-operate.

I am willing to bet that some stacks already implement 16.1, and
following the standard in the enterprise profile will make it simple
for such stacks to support the new profile. In contrast, mandating new
behavior makes more work for everyone.

Sometimes you cannot avoid having special behavior in a profile, but
in this case I don't think the departure from 1588 brings any tangible
benefit.

Thanks,
Richard

---

From: Kevin Gross <[email-masked]>
Date: Tue, 8 Apr 2014 13:09:58 -0600

The common asymmetric scenario is with PIM-SM. Under PIM-SM, multicasts are
at least initially routed through a rendezvous point which would likely be
a detour compared to a unicast path. My understanding is that most
implementations then go on to build a more reasonable distribution tree for
multicasts but there is nothing in the PIM RFCs that require or even
encourage this.

The objection I've heard to 16.1 is that it imposes arguably unnecessary
overhead in setting up unicast delay request/response. It is simpler for a
master just to know (based on profile) that when it receives a unicast
delay request, it should simply generate a unicast response to the sender.
If the enterprise profile is defined to work in this way, a slave can
switch mode on the fly and quickly compare results between unicast and
multicast and determine the best mode of operation.

---

From: Richard Cochran <[email-masked]>
Date: Tue, 8 Apr 2014 22:08:30 +0200

> multicast and determine the best mode of operation.

Yes, I agree that 16.1 is a real PITA to implement. However, that is
what is in the standard. It seems strange to me that profiles should
try and second guess aspects that are already standardized. The more
profiles do that, the harder it gets for humble implementations to
support the many variations.

Is there a feeling that 16.1 was a mistake, is not implemented in real
life, and is disappearing in the next 1588 edition?

If so, I'll surely *not* bother to implement it in linuxptp.

Thanks,
Richard

---

From: "Dowd, Greg" <[email-masked]>
Date: Tue, 8 Apr 2014 20:49:28 +0000

I don't think unicast negotiation will go away.  16.1 works quite well for managed networks.  An original premise here was that we were trying to move to support an "Internet" profile.  From that perspective, the ntp protocol has had the same requirements and handled them pretty well, although we all debate certain aspects.  One big problem is scaleability.  It would be ideal for ptp servers to be stateless.  Default profile is fine for that.  But the overhead of the multicast delay request-response system is scales poorly.  So, the hybrid profile.  This allows the slaves to be bootstrapped without configuration.  They register the listen, similar to ntp multicast, they discover ptp service stream and then act appropriately.  They could just jam the offset occasionally, they could have a short "ranging" session for path delay (again, like ntp multicast) or they could mobilize an association and perform ranging functions at a rate similar to the server transmission rate.  One other note about having a profile is to correct a profile nit which I see almost as a bug.  Currently, the default profiles mandates that event messages originate from 319 while data messages can originate from any source port.  This limitation provides little value in my opinion and breaks NAT.  Obviously, a non-PTP aware gateway device doing NAT opens a whole other can of worms but protocol, or data flow, is one layer of operation and performance is another.

-----Original Message-----
From: TICTOC [mailto:[email-masked]] On Behalf Of Richard Cochran
Sent: Tuesday, April 08, 2014 1:09 PM
To: Kevin Gross
Cc: [email-masked]
Subject: Re: [TICTOC] Enterprise profile update

On Tue, Apr 08, 2014 at 01:09:58PM -0600, Kevin Gross wrote:
> 
> The objection I've heard to 16.1 is that it imposes arguably 
> unnecessary overhead in setting up unicast delay request/response. It 
> is simpler for a master just to know (based on profile) that when it 
> receives a unicast delay request, it should simply generate a unicast response to the sender.

Yes, I agree that 16.1 is a real PITA to implement. However, that is what is in the standard. It seems strange to me that profiles should try and second guess aspects that are already standardized. The more profiles do that, the harder it gets for humble implementations to support the many variations.

Is there a feeling that 16.1 was a mistake, is not implemented in real life, and is disappearing in the next 1588 edition?

Thanks,
Richard

---

From: Wojciech Owczarek <[email-masked]>
Date: Tue, 8 Apr 2014 22:44:12 +0100

> But why not simply do what it says to do in 1588-2008 Section 16.1 ?
Because doing this will require the GM to start keeping track of all the
grants and their expiry and maintain a table of slaves, and will require
the GM and the slave to implement the grants, refusals, cancellations and
expiry. What the profile suggests hugely simplifies this. Out of the
implementations supporting this behaviour today (many more than ptpd), most
do it without unicast negotiation. This obviously doesn't mean anything,
but shows that the industry already has a preference.

On the master side, this behaviour actually is not a departure from the
standard - it's explicitly permitted:

9.5.12 states:

Delay_Resp messages shall be transmitted as multicast if the associated
Delay_Req message was sent as
multicast.
Delay_Resp messages shall be transmitted as unicast if the associated
Delay_Req message was sent as
unicast.

- no mention of clause 16 here.

9.5.11.1 (slave side - delay req) states:

Delay_Req messages shall be transmitted as multicast except if:
a) The optional unicast provisions of Clause 16 are used

Which indeed essentially already defines the mixed mode in combination with
16.1, but this is a combination of a "shall" with an optional feature, so
this is inconsistent - therefore I think that the standard needs amended
anyway, because currently both you and I are 50% right.

Regards
Wojciech

-- 
-

Wojciech Owczarek

---

From: Wojciech Owczarek <[email-masked]>
Date: Tue, 8 Apr 2014 23:19:02 +0100

On 8 April 2014 20:09, Kevin Gross <[email-masked]> wrote:

> The common asymmetric scenario is with PIM-SM. Under PIM-SM, multicasts
> are at least initially routed through a rendezvous point which would likely
> be a detour compared to a unicast path. My understanding is that most

There's the RFC and there's best practice. Sane implementations from major
vendors allow you to control the RPT to SPT cutover limits (in bytes,
bandwidth or time), per group or source/group pair, and also on most
enterprise kit (with well developed multicast support), by default, the SPT
cutover happens instantly, as soon as the last hop router receives the
first multicast packet. So if you're the first multicast group joiner, you
probably wait for an announce. Even if the announce is the first packet you
receive, by the time you send your delay request, PIM has already switched
from the shared tree to shortest path tree and multicast is already
following the unicast path.

Again, placing your PIM RPs is a matter of design, a well engineered
network will mitigate the asymmetry on PIM side. If the GM is sufficiently
far away for there to be asymmetries, the traffic is probably going through
your core - and that's where the RP should be. Also you normally do
multiple "instances" of the PIM RP (PIM anycast RPs) in combination with
MSDP so that there is always one close to you.

The case of ECMP remains, but if you want determinism, you design for
determinism.

Where I definitely agree that multicast vs. unicast asymmetry exists, is in
chassis based devices where unicast path in worst case is ASIC-fabric-ASIC,
but multicast is replicated between line cards as a binary tree so latency
can be from unicast latency  to n * unicast latency.

To me, this profile initiative (or standard initiative) is targeted towards
those who specifically need it and a lot of whom already do it.

Regards
Wojciech

-- 
-

Wojciech Owczarek

---

From: Richard Cochran <[email-masked]>
Date: Wed, 9 Apr 2014 07:11:16 +0200

> but shows that the industry already has a preference.

Well, if it is a "de facto" standard, then it does mean something. It
means that 16.1 is not really used or even implemented, in which case,
it should be removed from the next edition of 1588.

Is there anyone on the committee who can speak to this?

> - no mention of clause 16 here.

None the less, there is no provision in 1588 for transmitting unicast
Delay_Req *except* by using 16.1.
 
> anyway, because currently both you and I are 50% right.

I think the wording of 1588 is consistent. You must use 16.1 if you
want to send unicast Delay_Req messages. If the authors of the
standard meant to allow slaves to send unicast any time they feel like
it, then there would have been no need for 16.1 in the first place.

>From my point of view, I see the unfortunate trend of PTP profiles
becoming mini standards all on there own (gPTP is an extreme case of
this), each one adding changes in incompatible ways. This is a
nightmare for anyone who wants to try and implement this stuff.

Thanks,
Richard

---

From: Kevin Gross <[email-masked]>
Date: Wed, 9 Apr 2014 00:04:02 -0600

I've not been able to keep up with 1588v3 to know whether this has been
discussed there.

There is a 1588 profile under development by SMPTE for professional video
synchronization. Unicast delay request without 16.1 negotiation is
encouraged in the SMPTE profile as currently drafted. If we put this into
the enterprise profile, perhaps it is overstepping but we will not be alone.

---

From: Richard Cochran <[email-masked]>
Date: Wed, 9 Apr 2014 08:42:10 +0200

On Tue, Apr 08, 2014 at 08:49:28PM +0000, Dowd, Greg wrote:

> It would be ideal for ptp servers to be stateless.  Default profile
> is fine for that.  But the overhead of the multicast delay
> request-response system is scales poorly.  So, the hybrid profile.

You can still implement 16.1 without using any state. Consider the
following options.

1. Requests for unicast Sync are always denied.
2. Requests for unicast Delay_Resp are always granted.
3. Incoming unicast Delay_Req messages receive a unicast response.

If the enterprise profile would recommend this, then you could have
the hybrid behavior using standard 1588 options.

In my opinion, profiles should ideally be a menu of 1588 options to be
used together. If it can be avoided, then profiles should not mandate
new behaviors.

It appears to me that the use case addressed in the enterprise profile
is already covered by 16.1 without needing anything more.

Thanks,
Richard

---

From: "Meyer, Peter" <[email-masked]>
Date: Wed, 9 Apr 2014 07:29:15 +0000

Hi Richard,

Relating to your comment

> It means that 16.1 is not really used or even implemented, in which case, it should be removed from the next edition of 1588.

Please note ITU-T has standardized Telecom Profile for Frequency using unicast negotiation.  It is also very likely the Telecom Profile for Partial On-Path Phase (under development) will also use unicast negotiation.   I would think it very unlikely / not conceivable that unicast negotiation will be removed in future editions of IEEE1588.

Regards,

Peter Meyer
Timing & Synchronization
Microsemi Corporation

---

From: "Laurent Montini (lmontini)" <[email-masked]>
Date: Wed, 9 Apr 2014 07:31:23 +0000

Richard,

> it should be removed from the next edition of 1588.

16.1 is utilized by ITU-T G.8265.1.
A telecom slave can set ptp communication path and can communicate with
multiple GMs particularly for backup reason.
A slave can negotiate the message interval (e.g. related to network PDV
expectation) and the GM can refuse a grant (e.g., when running out of
resource.
Utilization of negotiated IP unicast avoids having all PTP comm path to be
constantly up & running (as would be the case with static config).

I don¹t think IEEE 1588-2008 prevents setting an IP unicast path
statically if preferred.
Note negotiation could allow a slave to request low message interval for
fast stabilization (burst mode) then ask to lower rate when in steady
state.

Thanks,
Laurent

---

From: John Fletcher <[email-masked]>
Date: Wed, 9 Apr 2014 10:00:25 +0000

Wojciech,

In your email below, you omitted the option b)

9.5.11.1 states:

Delay_Req messages shall be transmitted as multicast except if:
a) The optional unicast provisions of Clause 16 are used
b) Specified otherwise by a profile

In the case of the Enterprise and SMPTE Profiles, option b) is in effect, yes?

Regards,
John

From: TICTOC [mailto:[email-masked]] On Behalf Of Wojciech Owczarek
Sent: 08 April 2014 22:44
To: Richard Cochran
Cc: [email-masked]
Subject: Re: [TICTOC] Enterprise profile update

<snip>

Which indeed essentially already defines the mixed mode in combination with 16.1, but this is a combination of a "shall" with an optional feature, so this is inconsistent - therefore I think that the standard needs amended anyway, because currently both you and I are 50% right.

Regards
Wojciech

--

----------------------------

http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.

---------------------

---

From: Richard Cochran <[email-masked]>
Date: Wed, 9 Apr 2014 15:39:22 +0200

> Please note ITU-T has standardized Telecom Profile for Frequency using unicast negotiation.  It is also very likely the Telecom Profile for Partial On-Path Phase (under development) will also use u…

So if 16.1 is in current use by other profiles, then the enterprise
profile should also make use of it, unless there is a compelling
reason not to.

Right now, I don't see any reason why the enterprise profile should
not also use 16.1.

Thanks,
Richard

---

From: Richard Cochran <[email-masked]>
Date: Wed, 9 Apr 2014 15:54:19 +0200

> In the case of the Enterprise and SMPTE Profiles, option b) is in effect, yes?

Well, in 1588, there is more to it than that.

19.2.2:

  A node that uses a transport protocol for which the mapping is
  defined in an annex of this standard shall conform to that annex.

D.3:

  PTP messages shall use the multicast message specified in Table D.1.

The enterprise profile uses the IPv4 transport, and it therefore nodes
must conform to Annex D, which in turn specifies using multicast. The
option of unicast is available via 16.1, and so the enterprise profile
should mandate that option.

At least, that is how I understand the text.

Another interpretation would be, "we can do anything we want regarding
unicast," justified by the vague wording of 7.3.1.

Thanks,
Richard

---

From: "Dowd, Greg" <[email-masked]>
Date: Wed, 9 Apr 2014 15:39:00 +0000

How would that be using 16.1?  It provides no utility and just wastes bandwidth.  What would we achieve by doing this?  It is an optional element to the protocol so I don't understand the necessity.  I agree trying to work within the bounds of the protocol as defined is a good goal but if you read the protocol specification, profiles were indeed meant for exactly this purpose.  A subset of functions defined by the protocol agreed upon for a particular application, interoperable amongst a set of members.

-----Original Message-----
From: Richard Cochran [mailto:[email-masked]] 
Sent: Tuesday, April 08, 2014 11:42 PM
To: Dowd, Greg
Cc: Kevin Gross; [email-masked]
Subject: Re: [TICTOC] Enterprise profile update

On Tue, Apr 08, 2014 at 08:49:28PM +0000, Dowd, Greg wrote:

> It would be ideal for ptp servers to be stateless.  Default profile is 
> fine for that.  But the overhead of the multicast delay 

You can still implement 16.1 without using any state. Consider the following options.

If the enterprise profile would recommend this, then you could have the hybrid behavior using standard 1588 options.

In my opinion, profiles should ideally be a menu of 1588 options to be used together. If it can be avoided, then profiles should not mandate new behaviors.

It appears to me that the use case addressed in the enterprise profile is already covered by 16.1 without needing anything more.

Thanks,
Richard

---

From: Richard Cochran <[email-masked]>
Date: Wed, 9 Apr 2014 18:02:12 +0200

On Wed, Apr 09, 2014 at 03:39:00PM +0000, Dowd, Greg wrote:

> How would that be using 16.1?  It provides no utility and just

If it provides no utility, then why is it part of the standard, and
why is it used by other profiles?

> wastes bandwidth.  What would we achieve by doing this?

Interoperability. As I said before,

and,

> It is an optional element to the protocol so I don't understand the
> necessity.  I agree trying to work within the bounds of the protocol
> as defined is a good goal but if you read the protocol
> specification, profiles were indeed meant for exactly this purpose.
> A subset of functions defined by the protocol agreed upon for a
> particular application, interoperable amongst a set of members.

Yes, and one of those functions defined in 1588 is unicast messaging,
in section 16.1.

Thanks,
Richard