Subject: Re: [Netconf] Comments on draft-ietf-netconf-yang-push-18

From: Robert Wilton <[email-masked]>
Date: Tue, 4 Sep 2018 16:22:49 +0100

Hi Tim,

Please see comments (as a non author) inline ...

On 29/08/2018 16:28, Yutianpeng (Tim) wrote:
>
> Hi,
>
> Some comments as below:
>
> ------------------------------
>
> There is one point missing in the draft so far I think: what if the 
> subscriber is the initiator of the configuration changes?
>
> In this case, the subscriber is aware of configuration changes 
> initiated by itself without relying on the push mechanism. Introducing 
> push mechanism will dramatically impact the performance of the system 
> in this case.
>
> My suggestion is:
>
> 3.3. On-Change Considerations
>
>    5.  If the resulting patch record is non-empty, and it contains the 
> records not initiated by the subscriber, select these records and send 
> to corresponding subscriber.
>
> We can have further discussion on this to make it optional instead of 
> mandatory.
>
I'm not keen on this change.  This does not seem like a good separation 
of concerns.  Besides, I'm not convinced why this should really be a 
performance issue given that config changes are relatively rare compared 
to all the operational state in the system that may be changing.

> -----------------------------
>
> Both “subscriber” and “receiver” are used, bit confusing sometimes. 
> Shall we only use “subscriber”
>
They are different entities, the definitions are in 
draft-ietf-netconf-subscribed-notifications:

    Receiver: A target to which a publisher pushes subscribed event
    records.  For dynamic subscriptions, the receiver and subscriber are
    the same entity.

    Subscriber: A client able to request and negotiate a contract for the
    generation and push of event records from a publisher.  For dynamic
    subscriptions, the receiver and subscriber are the same entity.

> ------------------------
>
> 3.11.1. Robustness and reliability
>
>    Note: It is perfectly acceptable to have a series of "push-change-
>
>    update" notifications (and even "push update" notifications) serially
>
>    queued at the transport layer awaiting transmission.  It is not
>
>    required to merge pending update records.  I.e., the dampening period
>
>    applies to update record creation, not transmission.
>
> My suggestion is:
>
> The publisher should increase dampening period automatically in case 
> the message queue reaches a limit.
>
> This is to avoid notifications lost due to queue limit in case 
> subscriber is busy or temporarily unreachable.
>
I don't think that this is how the dampening that is currently defined 
should work, since it is expected to provide a strong guarantee that it 
will either deliver the notifications to the receiver, or otherwise 
notify to the receiver that some updates [may] have been lost.

I could imagine that another, softer type of dampening, could 
potentially be defined in future as a separate extension/augmentation, 
but I don't think that should go into this document, because it would 
massively slow it down, and the industry already needs this standard now.

> -----------------------------
>
> It would be appreciated if suggestions can be given on how push shall 
> be used on running, startup and candidate datastores
>
It should work the same way as YANG push on any other datastore. The 
mechanism described in the draft seems to be entirely datastore 
agnostic.  In what way do you think it may behave differently?

Although I could imagine that implementations may not support 
subscriptions on startup or candidate, in which case I would expect that 
the subscription would be rejected.

Thanks,
Rob

> -------------------------------
>
> Regards,
>
> Tim
>
>
>
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Tue, 4 Sep 2018 08:27:31 -0700

On Tue, Sep 4, 2018 at 8:22 AM, Robert Wilton <
rwilton=[email-masked]> wrote:

> In this case, the subscriber is aware of configuration changes initiated
> by itself without relying on the push mechanism. Introducing push mechanism
> will dramatically impact the performance of the system in this case.
>
> My suggestion is:
>
> records not initiated by the subscriber, select these records and send to
> corresponding subscriber.
>
> I'm not keen on this change.  This does not seem like a good separation of
> concerns.  Besides, I'm not convinced why this should really be a
> performance issue given that config changes are relatively rare compared to
> all the operational state in the system that may be changing.
>
>
+1 -- this change should not be made.
This is not a useful optimization.

> Both “subscriber” and “receiver” are used, bit confusing sometimes. Shall
> we only use “subscriber”
>
> The publisher should increase dampening period automatically in case the
> message queue reaches a limit.
>
> This is to avoid notifications lost due to queue limit in case subscriber
> is busy or temporarily unreachable.
>
> will either deliver the notifications to the receiver, or otherwise notify
> to the receiver that some updates [may] have been lost.
>
> I could imagine that another, softer type of dampening, could potentially
> be defined in future as a separate extension/augmentation, but I don't
> think that should go into this document, because it would massively slow it
> down, and the industry already needs this standard now.
>
>
+1 -- the server does what it is told and does not decide to change the
subscription on its own

> It would be appreciated if suggestions can be given on how push shall be
> used on running, startup and candidate datastores
>
> mechanism described in the draft seems to be entirely datastore agnostic.
> In what way do you think it may behave differently?
>

Andy

> Netconf mailing [email-masked]://www.ietf.org/mailman/listinfo/netconf
>
>
>

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 04 Sep 2018 17:33:43 +0200 (CEST)

> This is not a useful optimization.

+1

> +1 -- the server does what it is told and does not decide to change the

+1

/martin

---

From: "Yutianpeng (Tim)" <[email-masked]>
Date: Tue, 4 Sep 2018 16:20:15 +0000

Hi,
Thanks for reply.
Some explanations as below, please check.
Tim

-----Original Message-----
From: Netconf [mailto:[email-masked]] On Behalf Of Martin Bjorklund
Sent: 04 September 2018 16:34
To: [email-masked]
Cc: [email-masked]; rwilton=[email-masked]
Subject: Re: [Netconf] Comments on draft-ietf-netconf-yang-push-18

Andy Bierman <[email-masked]> wrote:
> > initiated by itself without relying on the push mechanism. 
> > Introducing push mechanism will dramatically impact the performance of the system in this case.
> >
> > My suggestion is:
> >
> >    5.  If the resulting patch record is non-empty, and it contains 
> > the records not initiated by the subscriber, select these records 
> > and send to corresponding subscriber.
> >
> > We can have further discussion on this to make it optional instead 
> > of mandatory.
> >
> > I'm not keen on this change.  This does not seem like a good 
> > separation of concerns.  Besides, I'm not convinced why this should 
> > really be a performance issue given that config changes are 
> > relatively rare compared to all the operational state in the system that may be changing.
> >
> >

+1

<Tim>
I might haven't explain the main concerns on this clearly.
When I see Yang-push, I find it very useful as it really solves problems all NMS/controllers are facing. But I do have a concern on the point I mentioned. 
Let me give an example that Yang-Push can be used:
Before Yang-push:
NMS is using Netconf yang to manage a router
NMS is using polling mechanism to sync config
Anyone make changes not through NMS, e.g. CLI, NMS is not aware of the change in time. 

After Yang-push:
NMS is using Netconf yang to manage a router
NMS is using Yang-push mechanism to sync configuration. In order to make sure all database in sync state, <running> database is subscribed  --- I believe this could be a widely use case
In case anyone make changes not through the NMS, changes will be notified to NMS via push mechanism.
With yang push solution, it solves restriction of polling mechanism pretty well.
But as I explained, yang push mechanism has an obvious drawback, each configuration from NMS will have a push message back to NMS, which is not needed in many use scenarios. This behavior actually impact performance a lot.
If this msg can be filtered directly without going through all Netconf +SSH/TLS + TCP process, the performance of the subscriber will improve a lot I believe.
Based on my understanding yang-push is to solve polling latency, but it should not have significant performance impact on existing applications. If NMS configure a device with 1000 changes and the device push 1000 changes back to NMS, I don't think this is an elegant implementation when Netconf has already provide enough transaction mechanism itself. The impact of push mechanism is it impact performance of 99% of scenario (normal NMS->device configure process) to solve 1% of use case (configuration not sync in time). 
I understand the difficulty of introducing big changes at this stage, but I really wish we can have an elegant push mechanism. Appreciate any comments on this.
</Tim>

> > I don't think that this is how the dampening that is currently 
> > defined should work, since it is expected to provide a strong 
> > guarantee that it will either deliver the notifications to the 
> > receiver, or otherwise notify to the receiver that some updates [may] have been lost.
> >
> > potentially be defined in future as a separate 
> > extension/augmentation, but I don't think that should go into this 
> > document, because it would massively slow it down, and the industry already needs this standard now.
> >
> >
> +1 -- the server does what it is told and does not decide to change 
> +the
> subscription on its own

+1

/martin

> > It would be appreciated if suggestions can be given on how push 
> > shall be used on running, startup and candidate datastores
> >
> > It should work the same way as YANG push on any other datastore.  
> > The mechanism described in the draft seems to be entirely datastore agnostic.
> > subscriptions on startup or candidate, in which case I would expect 
> > that the subscription would be rejected.
> >
<Tim>
Thanks Rob.
I think you understand my concerns. The whole yang push mechanism is datastore agnostic. 
But my position is: if we follow this draft, I am pretty sure implementation of yang push later will be different across all vendors. Someone will accept subscription on startup but others will reject. Some will support pushing changes on candidate but others not.
I believe the original intention of Netconf to provide a standardized mechanism and we are all working to scandalized as much as possible to allow different vendors to have interoperability with each other. 
If IETF working group can give instructions on how push mechanism can be adopted to different datastores instead of let vendors make decisions themselves, I believe we have a solution easier get interoperated. 
</Tim>

> > Netconf mailing 
> > [email-masked]://www.ietf.org/mailman/listinfo/netconf
> >
> >
> >

---

From: Andy Bierman <[email-masked]>
Date: Tue, 4 Sep 2018 09:42:32 -0700

On Tue, Sep 4, 2018 at 9:20 AM, Yutianpeng (Tim) <[email-masked]>
wrote:

> When I see Yang-push, I find it very useful as it really solves problems
> all NMS/controllers are facing. But I do have a concern on the point I
> mentioned.
> NMS is using Yang-push mechanism to sync configuration. In order to make
> sure all database in sync state, <running> database is subscribed  --- I
> believe this could be a widely use case
> But as I explained, yang push mechanism has an obvious drawback, each
> configuration from NMS will have a push message back to NMS, which is not
> needed in many use scenarios. This behavior actually impact performance a
> lot.
> If this msg can be filtered directly without going through all Netconf
> +SSH/TLS + TCP process, the performance of the subscriber will improve a
> lot I believe.
> Based on my understanding yang-push is to solve polling latency, but it
> should not have significant performance impact on existing applications. If
> NMS configure a device with 1000 changes and the device push 1000 changes
> back to NMS, I don't think this is an elegant implementation when Netconf
> has already provide enough transaction mechanism itself. The impact of push
> mechanism is it impact performance of 99% of scenario (normal NMS->device
> configure process) to solve 1% of use case (configuration not sync in
> time).
> I understand the difficulty of introducing big changes at this stage, but
> I really wish we can have an elegant push mechanism. Appreciate any
> comments on this.
> </Tim>
>
>

You are assuming the code that changed the configuration is also the code
that implements
the YP subscriber function, and there is some internal optimization to
update the data maintained
by the subscriber.

The standard imposes no such requirement -- the subscriber code can be
completely independent
of all other network management -- this goes for server-side as well.

Andy

> But my position is: if we follow this draft, I am pretty sure
> implementation of yang push later will be different across all vendors.
> Someone will accept subscription on startup but others will reject. Some
> will support pushing changes on candidate but others not.
> I believe the original intention of Netconf to provide a standardized
> mechanism and we are all working to scandalized as much as possible to
> allow different vendors to have interoperability with each other.
> If IETF working group can give instructions on how push mechanism can be
> adopted to different datastores instead of let vendors make decisions
> themselves, I believe we have a solution easier get interoperated.
> </Tim>
>
>
> > > Thanks,
> > > Rob
> > >
> > >
> > >
> >
> > Andy
> >
> >
> > >
> > >

---

From: Robert Wilton <[email-masked]>
Date: Tue, 4 Sep 2018 17:53:45 +0100

Hi Tim,

Please see inline ...

> I understand the difficulty of introducing big changes at this stage, but I really wish we can have an elegant push mechanism. Appreciate any comments on this.

(1) I'm not convinced that this will be a performance problem, 
particularly if dampening is applied to the subscription, or a periodic 
subscription is used instead.  I.e. I see that YANG push performance 
should be no worse than polling, and I anticipate that it will be 
significantly better.
(2) The YANG push draft is already 4 years old. I think that we must 
just finish what we are working on now, then we can discuss 
extensions/future work. If we keep churning the requirements and 
solution then we just end up with no solution, and the market looks 
elsewhere (as is already happening)

So, my pragmatic suggestion is:
  - we collectively try not to get side-tracked discussing this in 
detail now,
  - implement it as it is defined in the draft,
  - benchmark it to see how it performs (perhaps also compare to 
performance if JSON, or CBOR is used as an encoding as well),
  - if it really performs poorly (that is not due to implementation), 
then write up an ID describing the problem (including why 
dampening/periodic subscriptions don't help), with figures to back it 
up, etc,
  - the WG can then evaluate whether this is really a problem that needs 
to be solved, and if it does then someone can write a draft proposing an 
appropriate solution.

Does that seem reasonable?

Thanks,
Rob

---

From: "Yutianpeng (Tim)" <[email-masked]>
Date: Tue, 4 Sep 2018 17:22:50 +0000

-----Original Message-----
From: Robert Wilton [mailto:[email-masked]] 
Sent: 04 September 2018 17:54
To: Yutianpeng (Tim) <[email-masked]>; [email-masked]
Cc: Martin Bjorklund <[email-masked]>; [email-masked]
Subject: Re: [Netconf] Comments on draft-ietf-netconf-yang-push-18

Hi Tim,

Please see inline ...

On 04/09/2018 17:20, Yutianpeng (Tim) wrote:
> NMS is using Netconf yang to manage a router NMS is using polling 
> mechanism to sync config Anyone make changes not through NMS, e.g. 
> CLI, NMS is not aware of the change in time.
>
> After Yang-push:
> NMS is using Netconf yang to manage a router NMS is using Yang-push 
> mechanism to sync configuration. In order to make sure all database in 
> sync state, <running> database is subscribed  --- I believe this could be a widely use case In case anyone make changes not through the NMS, changes will be notified to NMS via push mechanism.

(1) I'm not convinced that this will be a performance problem, particularly if dampening is applied to the subscription, or a periodic subscription is used instead.  I.e. I see that YANG push performance should be no worse than polling, and I anticipate that it will be significantly better.
(2) The YANG push draft is already 4 years old. I think that we must just finish what we are working on now, then we can discuss extensions/future work. If we keep churning the requirements and solution then we just end up with no solution, and the market looks elsewhere (as is already happening)

So, my pragmatic suggestion is:
  - we collectively try not to get side-tracked discussing this in detail now,
  - implement it as it is defined in the draft,
  - benchmark it to see how it performs (perhaps also compare to performance if JSON, or CBOR is used as an encoding as well),
  - if it really performs poorly (that is not due to implementation), then write up an ID describing the problem (including why dampening/periodic subscriptions don't help), with figures to back it up, etc,
  - the WG can then evaluate whether this is really a problem that needs to be solved, and if it does then someone can write a draft proposing an appropriate solution.

Thanks,
Rob

////Yeah, let's do in this way.  
Thanks, 
Tim

---

From: Alexander Clemm <[email-masked]>
Date: Wed, 5 Sep 2018 01:33:49 +0000

+1 on all accounts.  

Tim, as explained, let us not introduce this change.  It makes implementation more complex and may even introduce other issues.  Eliminating "double-reporting" of changes to a client application that is making changes to datastore nodes to whose updates it has also subscribed is a minor optimization.  If you really have a need for this feature, it can be augmented into the model and be described in a separate draft.  

Thanks
--- Alex 

> From: Netconf [mailto:[email-masked]] On Behalf Of Robert Wilton
> Sent: Tuesday, September 04, 2018 9:54 AM
> > When I see Yang-push, I find it very useful as it really solves problems all
> NMS/controllers are facing. But I do have a concern on the point I mentioned.
> > sync state, <running> database is subscribed  --- I believe this could be a widely
> use case In case anyone make changes not through the NMS, changes will be
> notified to NMS via push mechanism.
> needed in many use scenarios. This behavior actually impact performance a lot.
> > If this msg can be filtered directly without going through all Netconf +SSH/TLS
> + TCP process, the performance of the subscriber will improve a lot I believe.
> > Based on my understanding yang-push is to solve polling latency, but it should
> not have significant performance impact on existing applications. If NMS
> configure a device with 1000 changes and the device push 1000 changes back to
> NMS, I don't think this is an elegant implementation when Netconf has already
> provide enough transaction mechanism itself. The impact of push mechanism is
> it impact performance of 99% of scenario (normal NMS->device configure
> process) to solve 1% of use case (configuration not sync in time).
> > I understand the difficulty of introducing big changes at this stage, but I really
> wish we can have an elegant push mechanism. Appreciate any comments on
> this.
> 
> (1) I'm not convinced that this will be a performance problem, particularly if
> dampening is applied to the subscription, or a periodic subscription is used
> instead.  I.e. I see that YANG push performance should be no worse than polling,
> and I anticipate that it will be significantly better.
> (2) The YANG push draft is already 4 years old. I think that we must just finish
> what we are working on now, then we can discuss extensions/future work. If we
> keep churning the requirements and solution then we just end up with no
> solution, and the market looks elsewhere (as is already happening)
> 
>   - benchmark it to see how it performs (perhaps also compare to performance if
> JSON, or CBOR is used as an encoding as well),
>   - if it really performs poorly (that is not due to implementation), then write up
> an ID describing the problem (including why dampening/periodic subscriptions
> don't help), with figures to back it up, etc,
>   - the WG can then evaluate whether this is really a problem that needs to be
> solved, and if it does then someone can write a draft proposing an appropriate
> solution.
> 
> > But my position is: if we follow this draft, I am pretty sure implementation of
> yang push later will be different across all vendors. Someone will accept
> subscription on startup but others will reject. Some will support pushing changes
> on candidate but others not.
> mechanism and we are all working to scandalized as much as possible to allow
> different vendors to have interoperability with each other.

---

From: "Yutianpeng (Tim)" <[email-masked]>
Date: Wed, 5 Sep 2018 08:45:24 +0000

Thanks Alex
Agree, it could be minor. Let's keep current push unchanged.
Thanks
Tim

-----Original Message-----
From: Alexander Clemm 
Sent: 05 September 2018 02:34
To: Robert Wilton <rwilton=[email-masked]>; Yutianpeng (Tim) <[email-masked]>; [email-masked]
Subject: RE: [Netconf] Comments on draft-ietf-netconf-yang-push-18

+1 on all accounts.  

Thanks
--- Alex 

> >>> There is one point missing in the draft so far I think: what if 
> >>> the subscriber is the initiator of the configuration changes?
> >>>
> >>> Introducing push mechanism will dramatically impact the 
> >>> performance of
> the system in this case.
> >>>
> >>> My suggestion is:
> >>>
> >>>     5.  If the resulting patch record is non-empty, and it 
> >>> contains the records not initiated by the subscriber, select these 
> >>> records and send to corresponding subscriber.
> >>>
> >>> separation of concerns.  Besides, I'm not convinced why this 
> >>> should really be a performance issue given that config changes are 
> >>> relatively rare compared to all the operational state in the 
> >>> system that may
> be changing.
> >>>
> >>>
> > mechanism to sync configuration. In order to make sure all database 
> > in sync state, <running> database is subscribed  --- I believe this 
> > could be a widely
> use case In case anyone make changes not through the NMS, changes will 
> be notified to NMS via push mechanism.
> configuration from NMS will have a push message back to NMS, which is 
> not needed in many use scenarios. This behavior actually impact performance a lot.
> not have significant performance impact on existing applications. If 
> NMS configure a device with 1000 changes and the device push 1000 
> changes back to NMS, I don't think this is an elegant implementation 
> when Netconf has already provide enough transaction mechanism itself. 
> The impact of push mechanism is it impact performance of 99% of 
> scenario (normal NMS->device configure
> particularly if dampening is applied to the subscription, or a 
> periodic subscription is used instead.  I.e. I see that YANG push 
> performance should be no worse than polling, and I anticipate that it will be significantly better.
>   - the WG can then evaluate whether this is really a problem that 
> needs to be solved, and if it does then someone can write a draft 
> proposing an appropriate solution.
> 
> >>> The publisher should increase dampening period automatically in 
> >>> case the message queue reaches a limit.
> >>>
> >>> receiver, or otherwise notify to the receiver that some updates 
> >>> [may] have
> been lost.
> >>>
> >>> document, because it would massively slow it down, and the 
> >>> industry
> already needs this standard now.
> >>>
> >>>
> >>> The mechanism described in the draft seems to be entirely 
> >>> datastore
> agnostic.
> >>> subscriptions on startup or candidate, in which case I would 
> >>> expect that the subscription would be rejected.
> >>>
> > <Tim>
> > Thanks Rob.
> > implementation of
> yang push later will be different across all vendors. Someone will 
> accept subscription on startup but others will reject. Some will 
> support pushing changes on candidate but others not.