Subject: [Netconf] Comments on draft-lhotka-netconf-restconf-transactions-00

From: Robert Wilton <[email-masked]>
Date: Wed, 11 Jul 2018 17:36:47 +0100

Hi Lada,

I've had a read of this draft, and have provided some comments below.

So, my top level comment is that I don't know whether or not RESTCONF 
needs this functionality or not.  I've heard some operators state that 
they think that clients can just construct an "atomic" change, and hence 
don't have the need for a server side staging area.  Perhaps a good 
question to ask in Montreal?

The rest of my comments below, apply to the proposed technical solution, 
and obviously only apply if this is a needed enhancement. :-)

1) Generally, I definitely prefer the idea of per session staging areas 
(aka private candidates) described in this draft over a shared lockable 
candidate datastore.  This follows my belief that loosely coupled 
concurrent systems are more robust than tightly coupled ones (e.g. with 
shared locking).

2) I don't think that this draft needs to mention <intended> at all.  
Instead, everywhere you mention <intended> then you should be saying 
<running>.  I.e. your staging datastores should update <running> on a 
commit operation, just like a commit of <candidate> updates <running>.  
<intended> is always just updated as a side effect of a write to 
<running>, and as such is a tangential consideration.

3) Rather than having clients interact via {+restconf}/data, I think 
that it would be much better to require NMDA and then have clients 
interact via {+restconf}/ds/ietf-restconf-transactions:staging, as per 
draft-ietf-netconf-nmda-restconf-04 section 3.1.  The new staging 
datastore identity should also be defined in your module to inherit from 
ietf-datastores:datastore identity.  I think that this probably also 
more closely aligns to restful principals.

4) So, I think that the <staging> datastore itself only contains the 
proposed changes (additions, modifications, and deletes) to <running> 
when they are committed.  I think that clients may also want to see the 
combined configuration of the current contents of <running> with the 
delta held in <staging> applied.  This could be exposed either as (i) a 
new RPC, (ii) as an extra query parameter or (iii) As another read-only 
datastore.  A new RPC has the disadvantage that it probably wouldn't 
support all the query parameters, so my instinctive preference would be 
to one of the other two latter options.

5) If private candidate datastores are being added to RESTCONF, then 
should they also be added to NETCONF?  If they are added to both then I 
think that they should be added in the same way, as much as possible, 
perhaps both could be updated in a single draft to save repetitive 
text?  In general, I like (Kent's?) idea of NETCONF WG writing a RFC 
that describes all the common parts of NETCONF and RESTCONF that the 
individual protocol docs can then reference rather than writing similar 
or equivalent text in two places.

But otherwise, I think that it is an interesting idea, and certainly 
warrants some WG discussion.

Thanks,
Rob

---

From: Mahesh Jethanandani <[email-masked]>
Date: Wed, 11 Jul 2018 11:41:26 -0700

WG,

This draft now is on the agenda for the meeting on Monday, afternoon session #I from 13:30-15:30.

Lada, you do have your 15 min.

Mahesh & Kent.

> On Jul 11, 2018, at 9:36 AM, Robert Wilton <rwilton=[email-masked]> wrote:
> 
> Hi Lada,
> 
> So, my top level comment is that I don't know whether or not RESTCONF needs this functionality or not.  I've heard some operators state that they think that clients can just construct an "atomic" change, and hence don't have the need for a server side staging area.  Perhaps a good question to ask in Montreal?
> 
> The rest of my comments below, apply to the proposed technical solution, and obviously only apply if this is a needed enhancement. :-)
> 
> 1) Generally, I definitely prefer the idea of per session staging areas (aka private candidates) described in this draft over a shared lockable candidate datastore.  This follows my belief that loosely coupled concurrent systems are more robust than tightly coupled ones (e.g. with shared locking).
> 
> 2) I don't think that this draft needs to mention <intended> at all.  Instead, everywhere you mention <intended> then you should be saying <running>.  I.e. your staging datastores should update <running> on a commit operation, just like a commit of <candidate> updates <running>.  <intended> is always just updated as a side effect of a write to <running>, and as such is a tangential consideration.
> 
> 3) Rather than having clients interact via {+restconf}/data, I think that it would be much better to require NMDA and then have clients interact via {+restconf}/ds/ietf-restconf-transactions:staging, as per draft-ietf-netconf-nmda-restconf-04 section 3.1.  The new staging datastore identity should also be defined in your module to inherit from ietf-datastores:datastore identity.  I think that this probably also more closely aligns to restful principals.
> 
> 4) So, I think that the <staging> datastore itself only contains the proposed changes (additions, modifications, and deletes) to <running> when they are committed.  I think that clients may also want to see the combined configuration of the current contents of <running> with the delta held in <staging> applied.  This could be exposed either as (i) a new RPC, (ii) as an extra query parameter or (iii) As another read-only datastore.  A new RPC has the disadvantage that it probably wouldn't support all the query parameters, so my instinctive preference would be to one of the other two latter options.
> 
> 5) If private candidate datastores are being added to RESTCONF, then should they also be added to NETCONF?  If they are added to both then I think that they should be added in the same way, as much as possible, perhaps both could be updated in a single draft to save repetitive text?  In general, I like (Kent's?) idea of NETCONF WG writing a RFC that describes all the common parts of NETCONF and RESTCONF that the individual protocol docs can then reference rather than writing similar or equivalent text in two places.
> 
> But otherwise, I think that it is an interesting idea, and certainly warrants some WG discussion.
> 
> Thanks,
> Rob
> 
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

Mahesh Jethanandani
[email-masked]

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Jul 2018 19:22:02 +0200

Hi Rob,

thanks for your comments, please see inline.

> question to ask in Montreal?

I think what you mean is an analogy to git, where all changes are
applied on the client's side and then new commits are pushed to the
server. However, git was designed for this mode of operation - I think
with RESTCONF it wouldn't be so efficient. And also, the client
functionality would be probably difficult to implement in a plain
browser whereas browser-based clients can be easily used with RESTCONF
extended according to my draft.

> <running>, and as such is a tangential consideration.

The main reason for using <intended> is that the target datastore into
which staging datastores are merged has to be valid at all
times. <running> has somewhat fuzzy semantics both in NETCONF and under
NMDA. But yes, the text also says that essentially we have <running> and
<intended> being the same. NMDA explicitly permits this simplification.

> more closely aligns to restful principals.

Again, in RESTCONF it is unclear what the "unified" datastore really
is. We wanted to make the semantics clear and explicit and, in
particular, permit configuration edits only via the staging
datastore. With your suggestion, it is not clear to me whether the
client could also interact with {+restconf}/data.

> to one of the other two latter options.

Do you mean to be able to see the result of a "dry run" of a commit?
This would be certainly possible and, in fact, in our implementation it
is pretty trivial.

> or equivalent text in two places.

But private candidates are already an option in NETCONF, right? One
possibility would be to make it the ONLY option, because shared candidates
have known problems.

Thanks, Lada

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 12 Jul 2018 19:23:01 +0200

> Lada, you do have your 15 min.

Thanks a lot, Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 12 Jul 2018 23:42:52 +0200

> <intended> being the same. NMDA explicitly permits this simplification.

I think NMDA says that <running> and <intented> in general are _not_
the same but that for _some_ implementations they may be de factor the
same or indistinguishable. If your proposal assumes <running> equals
<intended>, then your proposal is only applicable to _some_
implementations.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Robert Wilton <[email-masked]>
Date: Fri, 13 Jul 2018 11:02:04 +0100

Hi Lada,

> extended according to my draft.
No, I wasn't thinking that they would be separate commits, but a single 
client commit.

I guess it may depend on whether it is a machine constructing the 
configuration change (in which case merging it into a single request 
should be plausibly straight forward), or human's doing the interaction, 
although even then I still wonder whether creating an edit buffer on the 
client side, and then pushing that to the server as a single update 
isn't a slightly cleaner paradigm.

Perhaps the draft could have a background that explains some of the 
expected usages of private candidate datastores.

> <intended> being the same. NMDA explicitly permits this simplification.

<running> has the configuration supplied by the user before any template 
expansion, or inactive config removal.
<intended> is the same configuration data, but after template expansion, 
inactive config removal, and any other random config manipulations that 
the server might do.

If the device doesn't do "template expansion, inactive config removal, 
and any other random config manipulations", then <intended> is trivially 
the same as <running>.

Whenever <running> is due to be changed, <intended> is also updated at 
the same time, and validated.

Hence <intended> is always valid, and by implication, so is <running>, 
since you cannot make a change to <running> without also updating, and 
validating <intended> at the exact same time.  I.e. they succeed or fail 
together.

I think that your <staging> datastore design works much better with NMDA 
if you update <running> instead of <intended>.

> client could also interact with {+restconf}/data.
The problem with {+restconf}/data is that is combines the *desired* 
configuration with the *actual* operational state.  This combination 
cannot always be done in a sane way if the system isn't in a steady state.

I think that we should be trying to deprecate {+restconf}/data, I think 
that cleaner/simpler semantics can be achieved by interacting via 
explicit datastores.

E.g.
(1) If a RESTCONF client wants to make an atomic update to the 
configuration, then it just writes to <running>.
(2) If a RESTCONF client wants private staged configuration then it does 
it via <staging> and a commit to <running>.  From a system perspective 
this is pretty much the same as (1) any way.
(3 ) If a shared candidate datastore is required, then a client writes 
to <candidate> and then commits configuration to <running>.
(4) If <running> can be locked, then attempts by other clients to commit 
to <running> when it is locked must fail.

> This would be certainly possible and, in fact, in our implementation it
let me ask two different question first:

(1) If I call GET on <staging> then do I see just what I have changed 
(and explicitly don't see anything that I haven't changed), or do I see 
all of the base configuration with my private changes merged in?

(2) If the answer to Q1 is you see the base configuration + private 
changes merged in, then is it the base configuration fixed from the 
point in time that <staging> was initialized? Or does it float, i.e. it 
always updates to the latest committed base configuration in running?

> possibility would be to make it the ONLY option, because shared candidates
How do you do private candidate in NETCONF?  I thought that it was only 
shared candidate that had been standardized.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Jul 2018 15:30:01 +0200

> <intended>, then your proposal is only applicable to _some_

It is indeed <intended> what is intended in my draft, I can easily avoid
mentioning <running>. The only minor deviation is that RFC 8432
says that <intended> doesn't survive reboots, which is not desirable in
this case.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 13 Jul 2018 15:43:51 +0200

> says that <intended> doesn't survive reboots, which is not desirable in

Then you are making a mistake and you are unnecessarily limiting the
applicability of your proposal.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 13 Jul 2018 16:19:10 +0200

> isn't a slightly cleaner paradigm.

I agree that a lot can be done on the client side, but eventually the
data has to be sent to the server, and it is possible that the target
config datastore has changed in the mean time by another client - this
is a conflict that has to be resolved somehow.

> expected usages of private candidate datastores.

The aim is to enable transactions and concurrent R/W access of multiple
clients. This drafts attempts to solve it on the server side, somebody
else may want to propose a client-side solution.

I think both may be potentially useful - one can have capable
servers and restricted clients, or vice versa.

> if you update <running> instead of <intended>.

Yes, but <running> can be writable or not, may be locked and may be
invalid.

If RESTCONF is the only protocol, then it is perhaps just a matter of
naming, but if NETCONF is used along with RESTCONF on the same device, I
want to avoid their interference as much as possible. My idea is that
contributions from NETCONF and RESTCONF only meet at <intended>.

> that cleaner/simpler semantics can be achieved by interacting via 

If this is done, then it would make sense to do what you suggest. For
the time being, the advantage is that clients only suporting RFC 8040
can be used with my enhancements - the commit and reset operations can
be added separately, e.g as simple curl scripts. 

> to <running> when it is locked must fail.

This is all very complicated, I don't want to force RESTCONF users into
learning NETCONF first. Keep it simple, stupid.

> all of the base configuration with my private changes merged in?

After you do commit or reset, your staging repository becomes
(conceptually) an exact, private and writable copy of <intended>. If you
do some changes, you see them along with the other config data (modulo
NACM).  However, you don't see any changes that have been done to
<intended> in the mean time.

> always updates to the latest committed base configuration in running?

In our implementation, it is the data from the point of time when
<staging> was last initialized (after commit or reset). I think it would
be possible to let <staging> track the changes in intended as long as
the user doesn't start editing it.

> shared candidate that had been standardized.

RFC 6241 says this in sec. 8.3.1:

   The candidate configuration can be shared among multiple sessions.
   Unless a client has specific information that the candidate
   configuration is not shared, it MUST assume that other sessions are
   able to modify the candidate configuration at the same time.

Lada

---

From: Robert Wilton <[email-masked]>
Date: Fri, 13 Jul 2018 16:02:52 +0100

> is a conflict that has to be resolved somehow.
This is resolved at the time that any config change is merged into 
<running>.  Either the config change can be merged without errors and 
validates successfully (via <intended>), or the merge fails, or 
validation fails.  If either the merge or validate fails then <running> 
is not changed, the config change is rejected, and the client notified.

> else may want to propose a client-side solution.
Clients can already do it today, as per my previous answer.  I don't 
think that there is anything to standardize here.

> Yes, but <running> can be writable or not, may be locked and may be
Yes, and that is all fine.

> want to avoid their interference as much as possible.
You can't.  Ultimately there are two mechanisms writing the same data, 
they need to be sympathetic to each other.

>   My idea is that
Alas. I don't think that fits well with the NMDA architecture at all.  
The NMDA architecture assumes that all conventional client configuration 
operations combine at <running> rather than <intended>.  This is also 
the merge point today when both NETCONF and RESTCONF are being used.

The purpose of <intended> is as a mechanism to handle template 
expansion, inactive config, and possibly other default server config.  
It isn't meant to be another configuration merge point.

> learning NETCONF first. Keep it simple, stupid.
It is not complicated, particularly if the server doesn't implement 
locking of shared candidate.

I prefer explicit behavior.

E.g. I don't think that RESTCONF auto-magically committing the contents 
of a shared <candidate> datastore makes the two protocols work together 
simpler.  More likely it was occasionally cause very surprising, and 
potentially very bad, things happening to a devices configuration (e.g. 
if the NETCONF client isn't employing locking).

> <intended> in the mean time.
OK, so I think that it is useful to be able to get/see the delta against 
the base copy, and perhaps an operation for it to sync and merge with 
the latest baseline version.  Obviously, there would need to be a 
mechanism to report merge conflicts.

>     able to modify the candidate configuration at the same time.
This implies to me that NETCONF's candidate datastore is generally 
regarded as being shared, not private.

Thanks,
Rob

---

From: Andy Bierman <[email-masked]>
Date: Fri, 13 Jul 2018 08:16:06 -0700

Hi,

I do not think this problem should be worked on for RESTCONF.
In the future, a protocol-independent solution for
concurrent edit operations might be interesting.

Very strongly disagree that the /restconf/data "unified" URI should be
deprecated
or that requiring multiple editing steps is REST-full.

Customers like the client-side simplicity of RESTCONF.
They can use simple curl commands (or library equivalent).
Operations are 1-shot and stateless.

RESTCONF has no sessions, so the NETCONF session locking described in RFC
6241
does not work for RESTCONF.

Andy

On Fri, Jul 13, 2018 at 8:02 AM, Robert Wilton <
rwilton=[email-masked]> wrote:

> validates successfully (via <intended>), or the merge fails, or validation
> fails.  If either the merge or validate fails then <running> is not
> changed, the config change is rejected, and the client notified.
>
>
> Clients can already do it today, as per my previous answer.  I don't think
> that there is anything to standardize here.
>
>
> Alas. I don't think that fits well with the NMDA architecture at all.  The
> NMDA architecture assumes that all conventional client configuration
> operations combine at <running> rather than <intended>.  This is also the
> merge point today when both NETCONF and RESTCONF are being used.
>
> The purpose of <intended> is as a mechanism to handle template expansion,
> inactive config, and possibly other default server config.  It isn't meant
> to be another configuration merge point.
>
>
> E.g. I don't think that RESTCONF auto-magically committing the contents of
> a shared <candidate> datastore makes the two protocols work together
> potentially very bad, things happening to a devices configuration (e.g. if
> the NETCONF client isn't employing locking).
>
> the base copy, and perhaps an operation for it to sync and merge with the
> latest baseline version.  Obviously, there would need to be a mechanism to
> report merge conflicts.
>
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 15 Jul 2018 14:53:31 +0200

Hi,

due to my stupidity, I won't be present at the NETCONF session
tomorrow. I wasn't aware that Canada also introduced that nice ETA
immigration form and couldn't get it until my plane departed. :-(

I was able to rebook my flight and will arrive (hopefully) Monday night.

With apologies,

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 15 Jul 2018 15:48:17 +0200

> concurrent edit operations might be interesting.

Sounds like a good plan for the next two decades. :-)

> or that requiring multiple editing steps is REST-full.

The editing steps are exactly the same for a given user, the only catch is that
nothing happens as a result of the editing. I don't see anything in RFC 8040
that prevents postponing the application of configuration changes.

BTW, I also don't like deprecating {+restconf}/data. 

> They can use simple curl commands (or library equivalent).

They would be able to do the same with just one extra step - the commit/reset
operation, which can be executed via curl easily, too.

Early revisions of draft-ietf-netconf-restconf contained statements like

   Applications that require more complex transaction capabilities might
   consider NETCONF instead of RESTCONF.

Is it what you still suggest? My motivation for writing the present draft was
exactly to enable some of these capabilities without resorting to NETCONF.

> RESTCONF has no sessions, so the NETCONF session locking described in RFC 6241

My draft does NOT introduce locks. Actually, after the comments by Rob and
Juergen, I am now inclined to use <running> rather than <intended> as the commit
target. Then, if <running> happens to be locked (from outside, e.g. NETCONF),
then the commit operation has to be denied, but it has nothing to do with
sessions.

Lada

> 
> Andy
> 
> 
> 
> 
> On Fri, Jul 13, 2018 at 8:02 AM, Robert Wilton
> <rwilton=[email-masked]> wrote:
> > 
> > fails.  If either the merge or validate fails then <running> is not changed,
> > the config change is rejected, and the client notified.
> > 
> >  You can't.  Ultimately there are two mechanisms writing the same data, they
> > need to be sympathetic to each other.
> > 
> > >   My idea is that
> >  It is not complicated, particularly if the server doesn't implement locking
> > of shared candidate.
> > 
> > E.g. I don't think that RESTCONF auto-magically committing the contents of a
> > shared <candidate> datastore makes the two protocols work together simpler. 
> > More likely it was occasionally cause very surprising, and potentially very
> > bad, things happening to a devices configuration (e.g. if the NETCONF client
> > isn't employing locking).
> > 
> >  This implies to me that NETCONF's candidate datastore is generally regarded
> > as being shared, not private.
> > 
> > Thanks,
> > Rob
> > 
> > 
> > > Lada
> > > 
> > > > Thanks,
> > > > Rob
> > > > 
> > > > 
> > > > > Thanks, Lada
> > > > >

---

From: Andy Bierman <[email-masked]>
Date: Sun, 15 Jul 2018 07:55:14 -0700

> Sounds like a good plan for the next two decades. :-)
Your draft ignores the concurrency problem. E.G:

At time T0, config is /foo
Client1 and Client2 both start editing

At T1, Client1 merges /foo/A (config is /foo/A)

At T2, Client2 replaces /foo with /foo/B

At this point all of edits from Client1 are lost.
Imagine if git worked this way :-(

The edit from Client2 has to be rejected instead of accepted.
A clever system will allow Client2 to resync to time T1 and then try the
commit again

> exactly to enable some of these capabilities without resorting to NETCONF.
yes because NETCONF has sessions and locking and lock management built-in to
session management

> then the commit operation has to be denied, but it has nothing to do with
Correct. Your draft introduces concurrent candidate datastores for RESTCONF
only.
Private staging areas do not work well if simplistic "last one wins"
commits are done.
I am not sure RESTCONF needs any scratchpad datastore. This was created
to assist human CLI datastore editing in NETCONF.  RESTCONF has YANG Patch
to easily collect edits to apply all at once.

RESTCONF for NMDA allows access to the real candidate datastore.
The /restconf/operations resource allows RESTCONF access to any
YANG-defined RPC
(which includes ietf-netconf RPCs).   I don't see any problem to solve in
order
to make RESTCONF be more like NETCONF.

Lada
Andy

> > > > > > > needs this functionality or not.  I've heard some operators
> state
> > > > > > > that
> > > > > > > they think that clients can just construct an "atomic" change,
> and
> > > > > > > hence
> validation
> changed,
> think
> > > > > > > 1) Generally, I definitely prefer the idea of per session
> staging
> > > > > > > areas
> > > > > > > concurrent systems are more robust than tightly coupled ones
> (e.g.
> > > > > > > with
> > > > > > > shared locking).
> > > > > > >
> > > > > > > <running>.  I.e. your staging datastores should update
> <running> on
> > > > > > > a
> > > > > >  The main reason for using <intended> is that the target
> datastore
> > > > > > into
> > > > > > NMDA. But yes, the text also says that essentially we have
> <running>
> > > > > > and
> the
> expansion,
> > > > > > > interact via {+restconf}/ds/ietf-restconf-transactions:staging,
> as
> > > > > > > per
> > > > > > > datastore identity should also be defined in your module to
> inherit
> > > > > > > from
> > > > > > > when they are committed.  I think that clients may also want
> to see
> > > > > > > the
> > > > > > > delta held in <staging> applied.  This could be exposed either
> as
> > > > > > > (i) a
> > > > > > > new RPC, (ii) as an extra query parameter or (iii) As another
> read-
> > > > > > > only
> > > > > > > support all the query parameters, so my instinctive preference
> would
> > > > > > > be
> > > > > > This would be certainly possible and, in fact, in our
> implementation
> > > > > > it
> > > > > > is pretty trivial.
> > > > > >
> the
> > > > > > > should they also be added to NETCONF?  If they are added to
> both
> > > > > > > then I

---

From: Robert Wilton <[email-masked]>
Date: Sun, 15 Jul 2018 16:29:36 -0400

Hi Lada,

> BTW, I also don't like deprecating {+restconf}/data.
My opposition to {+restconf}/data is that works nicely with operational 
state.  I.e. it has the same issues as NETCONF <get> operation, which is 
why <get-data> was introduced in the NETCONF NMDA draft.

Defining a version of {+restconf}/data that abstracts and hides 
<candidate>, commits, updating startup is fine, and we discussed this as 
option as part of NMDA.

However, I still regard automatically committing the contents of 
<candidate> from a RESTCONF operation as very risky behaviour.

> exactly to enable some of these capabilities without resorting to NETCONF.
I would like RESTCONF to be a fully viable alternative to NETCONF. 
Particularly because it have easily handle alternative encodings.

>> does not work for RESTCONF.
This point that Andy raised concerns me.  If RESTCONF has no sessions 
that how does a <staging> datastore work?

> then the commit operation has to be denied, but it has nothing to do with
Note that in my previous comments, I wasn't saying that RESTCONF has to 
support <candidate>, or <locks>, but I was trying to explain how private 
candidate datastores could inter-operate with them if they were supported.

Thanks,
Rob

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 16 Jul 2018 14:13:57 +0200

> why <get-data> was introduced in the NETCONF NMDA draft.

This is true, but I would change the semantics of this resource instead
of deprecating it. The shorter URIs are very convenient.

Perhaps this resource should just be a shortcut to <running>, or to
<staging> if my enhancement is supported.

> <candidate> from a RESTCONF operation as very risky behaviour.

Absolutely, if you mean the <candidate> and commit as in NETCONF. 

> Particularly because it have easily handle alternative encodings.

Yes, that's my motivation, too.

> that how does a <staging> datastore work?

RESTCONF runs on top of a TLS session that provides user identity, but
this is nothing new. Otherwise, I don't see any single bit of client's
state that the server has to take into account.

Each user has a private <staging> that is in a certain well-defined
state (on the server). It is analogical to webmail and similar
appllications and perfectly RESTful. 

> candidate datastores could inter-operate with them if they were

As I wrote, I agree with using <running> in place of <intended>, as you
suggested, but I believe <candidate> should not be accesible and
operable from RESTCONF.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 16 Jul 2018 19:08:19 -0700

> why <get-data> was introduced in the NETCONF NMDA draft.
It works for the functionality that was defined before NMDA existed.
The point of the unified /restconf/data URI is that it is the same on every
server.
There is no discovery phase and 1 of N editing models.  The server hides
those details to simplify client programming.

> <candidate>, commits, updating startup is fine, and we discussed this as
This is what the RFC 8040 version of /restconf/data does

Please learn to add new functionality in a way that does not break shipping
code.

Use some other URL besides /restconf/data for new functionality.

Andy

>>>> of a

---

From: Robert Wilton <[email-masked]>
Date: Tue, 17 Jul 2018 06:52:01 -0400

Hi Andy,

On 16/07/2018 22:08, Andy Bierman wrote:
>
>
> On Sun, Jul 15, 2018 at 1:29 PM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi Lada,
>
>
>     My opposition to {+restconf}/data is that works nicely with
>     operational state.  I.e. it has the same issues as NETCONF <get>
>     operation, which is why <get-data> was introduced in the NETCONF
>     NMDA draft.
>
>
> The point of the unified /restconf/data URI is that it is the same on 
> every server.
>     <candidate>, commits, updating startup is fine, and we discussed
>     this as option as part of NMDA.
>
>

Yes, the 8040 version does this, but it also co-mingles state which can 
be a problem (both for NMDA and pre-NMDA).

>
>
> Please learn to add new functionality in a way that does not break 
> shipping code.
I have not proposed breaking any shipping code.

I've suggested deprecating /data - this doesn't break existing clients, 
but warns them that the functionality will disappear in future.
I've suggested defining a version of /data that keeps the useful config 
abstraction but without the co-mingled state issues.  Yes, this would be 
on a new URL, or possibly defined as an abstract datastore.

> Use some other URL besides /restconf/data for new functionality.
Of course, I've never intentionally suggested otherwise.

Thanks,
Rob

>     This point that Andy raised concerns me.  If RESTCONF has no
>     sessions that how does a <staging> datastore work?
>
>     Note that in my previous comments, I wasn't saying that RESTCONF
>     has to support <candidate>, or <locks>, but I was trying to
>     explain how private candidate datastores could inter-operate with
>     them if they were supported.
>
>     Thanks,
>     Rob
>
>
>
>         Lada
>
>             Andy
>
>
>
>
>             <rwilton=[email-masked]
>             <mailto:[email-masked]>> wrote:
>
>                     Robert Wilton <[email-masked]
>                     <mailto:[email-masked]>> writes:
>
>                         Hi Lada,
>
>
>                             Robert Wilton <[email-masked]
>                             <mailto:[email-masked]>> writes:
>
>                                 Hi Lada,
>
>                                 I've had a read of this draft, and
>                                 have provided some comments
>                                 below.
>
>                                 So, my top level comment is that I
>                                 don't know whether or not
>                                 RESTCONF
>                                 needs this functionality or not.  I've
>                                 heard some operators state
>                                 that
>                                 they think that clients can just
>                                 construct an "atomic" change, and
>                                 hence
>                                 The rest of my comments below, apply
>                                 to the proposed technical
>                                 solution,
>                                 1) Generally, I definitely prefer the
>                                 idea of per session staging
>                                 areas
>                                 (aka private candidates) described in
>                                 this draft over a shared
>                                 lockable
>                                 concurrent systems are more robust
>                                 than tightly coupled ones (e.g.
>                                 with
>                                 shared locking).
>
>                                 <running>.  I.e. your staging
>                                 datastores should update <running> on
>                                 a
>                                 commit operation, just like a commit
>                                 of <candidate> updates
>                                 <running>.
>                               The main reason for using <intended> is
>                             that the target datastore
>                             into
>                             times. <running> has somewhat fuzzy
>                             semantics both in NETCONF and
>                             under
>                             NMDA. But yes, the text also says that
>                             essentially we have <running>
>                             and
>                             <intended> being the same. NMDA explicitly
>                             permits this
>                             simplification.
>
>                           <running> has the configuration supplied by
>                         the user before any
>                         template
>                                 interact via
>                                 {+restconf}/ds/ietf-restconf-transactions:staging,
>                                 as
>                                 per
>                                 datastore identity should also be
>                                 defined in your module to inherit
>                                 from
>                         cannot always be done in a sane way if the
>                         system isn't in a steady
>                         state.
>
>                                 proposed changes (additions,
>                                 modifications, and deletes) to
>                                 <running>
>                                 when they are committed.  I think that
>                                 clients may also want to see
>                                 the
>                                 delta held in <staging> applied.  This
>                                 could be exposed either as
>                                 (i) a
>                                 new RPC, (ii) as an extra query
>                                 parameter or (iii) As another read-
>                                 only
>                                 support all the query parameters, so
>                                 my instinctive preference would
>                                 be
>                             This would be certainly possible and, in
>                             fact, in our implementation
>                             it
>                             is pretty trivial.
>
>                                 should they also be added to NETCONF? 
>                                 If they are added to both
>                                 then I
>                                 think that they should be added in the
>                                 same way, as much as
>                                 possible,
>                                 individual protocol docs can then
>                                 reference rather than writing
>                                 similar
>                             possibility would be to make it the ONLY
>                             option, because shared
>                             candidates
>                             have known problems.
>
>                 [email-masked] <mailto:[email-masked]>
>                 <https://www.ietf.org/mailman/listinfo/netconf>
>
>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 17 Jul 2018 08:07:13 -0700

On Tue, Jul 17, 2018 at 3:52 AM, Robert Wilton <[email-masked]> wrote:

> Yes, the 8040 version does this, but it also co-mingles state which can be
> a problem (both for NMDA and pre-NMDA).
>
>
>

So far the only standard thing you can do with NMDA is report the
operational value
of configuration data nodes.   For devices that do not have noticeable
delays to apply configuration,
there won't be any difference between the config=true nodes in <running>
and <operational> .
For these devices, NMDA is not interesting and even irrelevant.

> but warns them that the functionality will disappear in future.

This certainly breaks existing clients.
There is no reason for the functionality to disappear in the future except
the NETCONF WG does not care about stability.

If you need the functionality of NMDA, then use NMDA for RESTCONF.

> abstraction but without the co-mingled state issues.  Yes, this would be on
> a new URL, or possibly defined as an abstract datastore.
>

Moving /restconf/data or changing it (as Lada suggests) is not backward
compatible

Andy

---

From: Robert Wilton <[email-masked]>
Date: Tue, 17 Jul 2018 11:23:30 -0400

On 17/07/2018 11:07, Andy Bierman wrote:
>
>
> On Tue, Jul 17, 2018 at 3:52 AM, Robert Wilton <[email-masked] 
>>         operational state.  I.e. it has the same issues as NETCONF
>>         <get> operation, which is why <get-data> was introduced in
>>         the NETCONF NMDA draft.
>>
>>
>>     The point of the unified /restconf/data URI is that it is the
>>     same on every server.
>>         Defining a version of {+restconf}/data that abstracts and
>>         hides <candidate>, commits, updating startup is fine, and we
>>         discussed this as option as part of NMDA.
>>
>>
>     Yes, the 8040 version does this, but it also co-mingles state
>     which can be a problem (both for NMDA and pre-NMDA).
>
>
>
>
>
> there won't be any difference between the config=true nodes in 
> <running> and <operational> .
Assuming that the devices are bug free and never fail to immediately 
apply the configuration then you are right.  Or equally, for some 
networks these could be considered rare corner case conditions when 
dropping back to manual intervention is fine.

>
>>
>>
>>     Please learn to add new functionality in a way that does not
>>     break shipping code.
>     I've suggested deprecating /data - this doesn't break existing
>     clients, but warns them that the functionality will disappear in
>     future.
>
>
>
This could be a RESTCONF 2.0.  Servers could support both versions and 
leave it to clients to decide which version to use.  I.e. leave it to 
the market to decide whether backwards compatibility is required.

> the NETCONF WG does not care about stability.
I disagree.  I think that the existing functionality only models simple 
devices under optimistic scenarios.  My belief as networks move towards 
more automation, accuracy of the operational data will become more 
important/significant.

Thanks,
Rob

>     I've suggested defining a version of /data that keeps the useful
>     config abstraction but without the co-mingled state issues. Yes,
>     this would be on a new URL, or possibly defined as an abstract
>     datastore.
>
>
>
> Moving /restconf/data or changing it (as Lada suggests) is not 
> backward compatible
>
>
>
>
>>
>>         However, I still regard automatically committing the contents
>>         of <candidate> from a RESTCONF operation as very risky behaviour.
>>
>>
>>         I would like RESTCONF to be a fully viable alternative to
>>         NETCONF. Particularly because it have easily handle
>>         alternative encodings.
>>
>>
>>         Note that in my previous comments, I wasn't saying that
>>         RESTCONF has to support <candidate>, or <locks>, but I was
>>         trying to explain how private candidate datastores could
>>         inter-operate with them if they were supported.
>>
>>         Thanks,
>>         Rob
>>
>>
>>
>>             Lada
>>
>>                 Andy
>>
>>
>>
>>
>>                                     I've had a read of this draft,
>>                                     and have provided some comments
>>                                     below.
>>
>>                                     So, my top level comment is that
>>                                     I don't know whether or not
>>                                     RESTCONF
>>                                     needs this functionality or not. 
>>                                     I've heard some operators state
>>                                     that
>>                                     The rest of my comments below,
>>                                     apply to the proposed technical
>>                                     solution,
>>                                     1) Generally, I definitely prefer
>>                                     the idea of per session staging
>>                                     areas
>>                                     (aka private candidates)
>>                                     described in this draft over a shared
>>                                     lockable
>>                                     candidate datastore.  This
>>                                     follows my belief that loosely
>>                                     coupled
>>                                     commit operation, just like a
>>                                     commit of <candidate> updates
>>                                     <running>.
>>                                   The main reason for using
>>                                 <intended> is that the target datastore
>>                                 into
>>                                 NMDA. But yes, the text also says
>>                                 that essentially we have <running>
>>                                 and
>>                                 <intended> being the same. NMDA
>>                                 explicitly permits this
>>                                 simplification.
>>
>>                               <running> has the configuration
>>                             supplied by the user before any
>>                             template
>>                             and any other random config
>>                             manipulations", then <intended> is trivially
>>                             the same as <running>.
>>
>>                                     3) Rather than having clients
>>                                     interact via {+restconf}/data, I
>>                                     think
>>                                     ietf-datastores:datastore
>>                                     identity.  I think that this
>>                                     probably also
>>                             cannot always be done in a sane way if
>>                             the system isn't in a steady
>>                             state.
>>
>>                                     when they are committed.  I think
>>                                     that clients may also want to see
>>                                     the
>>                                     combined configuration of the
>>                                     current contents of <running>
>>                                     with the
>>                                     delta held in <staging> applied. 
>>                                     This could be exposed either as
>>                                     (i) a
>>                                     datastore.  A new RPC has the
>>                                     disadvantage that it probably
>>                                     wouldn't
>>                                     support all the query parameters,
>>                                     so my instinctive preference would
>>                                     be
>>                                 This would be certainly possible and,
>>                                 in fact, in our implementation
>>                                 it
>>                                 is pretty trivial.
>>
>>                                     5) If private candidate
>>                                     datastores are being added to
>>                                     RESTCONF, then
>>                                     should they also be added to
>>                                     NETCONF?  If they are added to both
>>                                     then I
>>                                     think that they should be added
>>                                     in the same way, as much as
>>                                     possible,
>>                                     text?  In general, I like
>>                                     (Kent's?) idea of NETCONF WG
>>                                     writing a RFC
>>                                     that describes all the common
>>                                     parts of NETCONF and RESTCONF
>>                                     that the
>>                                 possibility would be to make it the
>>                                 ONLY option, because shared
>>                                 candidates
>>                                 have known problems.
>>

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 17 Jul 2018 11:31:58 -0400

On Tue, 2018-07-17 at 08:07 -0700, Andy Bierman wrote:
> > Yes, the 8040 version does this, but it also co-mingles state which can be a
> > problem (both for NMDA and pre-NMDA). 
> > 
> > 
> > 
> 
> 
> 
> So far the only standard thing you can do with NMDA is report the operational
> value
> of configuration data nodes.   For devices that do not have noticeable delays
> to apply configuration,
> there won't be any difference between the config=true nodes in <running> and
> <operational> .
> > I've suggested deprecating /data - this doesn't break existing clients, but
> > warns them that the functionality will disappear in future.
> > 
> 
> 

It depends on what you mean by compatibility. I believe it is necessary to
support NMDA and the new YANG Library in RESTCONF, and then by definition
operation state data has to be separated from (editable) configuration data.

It is IMO a good thing, and from the viewpoint of RESTCONF implementers it is a
considerable simplification. Finally the operators' requirement from the initial
2002 IAB workshop (documented in RFC 3535) becomes satisfied, namely

       It is necessary to make a clear distinction between configuration
       data, data that describes operational state and statistics.  Some
       devices make it very hard to determine which parameters were
       administratively configured and which were obtained via other
       mechanisms such as routing protocols.

Lada

> > > > > > > > > > > I've had a read of this draft, and have provided some
> > > > > > > > > > > comments
> > > > > > > > > > > below.
> > > > > > > > > > > 
> > > > > > > > > > > So, my top level comment is that I don't know whether or
> > > > > > > > > > > not
> > > > > > > > > > > RESTCONF
> > > > > > > > > > > needs this functionality or not.  I've heard some
> > > > > > > > > > > operators state
> > > > > > > > > > > that
> > > > > > > > > > > they think that clients can just construct an "atomic"
> > > > > > > > > > > change, and
> > > > > > > > > > > hence
> > > > > > > > > > > The rest of my comments below, apply to the proposed
> > > > > > > > > > > technical
> > > > > > > > > > > solution,
> > > > > > > > > > > (aka private candidates) described in this draft over a
> > > > > > > > > > > shared
> > > > > > > > > > > lockable
> > > > > > > > > > > concurrent systems are more robust than tightly coupled
> > > > > > > > > > > ones (e.g.
> > > > > > > > > > > with
> > > > > > > > > > > shared locking).
> > > > > > > > > > > 
> > > > > > > > > > > commit operation, just like a commit of <candidate>
> > > > > > > > > > > updates
> > > > > > > > > > > <running>.
> > > > > > > > > > times. <running> has somewhat fuzzy semantics both in
> > > > > > > > > > NETCONF and
> > > > > > > > > > under
> > > > > > > > >    <running> has the configuration supplied by the user before
> > > > > > > > > any
> > > > > > > > > template
> > > > > > > > > > > interact via {+restconf}/ds/ietf-restconf-
> > > > > > > > > > > transactions:staging, as
> > > > > > > > > > > per
> > > > > > > > > > > datastore identity should also be defined in your module
> > > > > > > > > > > to inherit
> > > > > > > > > > > from
> > > > > > > > > cannot always be done in a sane way if the system isn't in a
> > > > > > > > > steady
> > > > > > > > > state.
> > > > > > > > > 
> > > > > > > > > > > proposed changes (additions, modifications, and deletes)
> > > > > > > > > > > to
> > > > > > > > > > > <running>
> > > > > > > > > > > when they are committed.  I think that clients may also
> > > > > > > > > > > want to see
> > > > > > > > > > > the
> > > > > > > > > > > delta held in <staging> applied.  This could be exposed
> > > > > > > > > > > either as
> > > > > > > > > > > (i) a
> > > > > > > > > > > new RPC, (ii) as an extra query parameter or (iii) As
> > > > > > > > > > > another read-
> > > > > > > > > > > only
> > > > > > > > > > > support all the query parameters, so my instinctive
> > > > > > > > > > > preference would
> > > > > > > > > > > be
> > > > > > > > > > > should they also be added to NETCONF?  If they are added
> > > > > > > > > > > to both
> > > > > > > > > > > then I
> > > > > > > > > > > think that they should be added in the same way, as much
> > > > > > > > > > > as
> > > > > > > > > > > possible,
> > > > > > > > > > > individual protocol docs can then reference rather than
> > > > > > > > > > > writing
> > > > > > > > > > > similar
> > > > > > > > > > possibility would be to make it the ONLY option, because
> > > > > > > > > > shared
> > > > > > > > > > candidates
> > > > > > > > > > have known problems.
> > > > > > > > > > 
> > > > > > > > > >

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 17 Jul 2018 17:53:49 +0200

On Tue, Jul 17, 2018 at 08:07:13AM -0700, Andy Bierman wrote:
> operational value of configuration data nodes.  For devices that do
> not have noticeable delays to apply configuration, there won't be
> any difference between the config=true nodes in <running> and
> <operational> .  For these devices, NMDA is not interesting and even
> irrelevant.

It is not only delay that causes applied config to differ from
intended config but there is also plugable hardware, sometimes
intended config gets augmented or even overwritten by control plane
protocols, sometimes local software on the device bypasses the config
system, sometimes there are simply bugs.

The number of devices that pretend to not need NMDA may be larger than
the number of devices that really do not need NMDA.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 17 Jul 2018 09:08:56 -0700

> system, sometimes there are simply bugs.

Not all devices have line cards.
If you read RFC 8040, it says RESTCONF is not intended to be a replacement
for NETCONF
so simple devices are likely to never need to report differences between
intended and applied
configuration.

> the number of devices that really do not need NMDA.

Where are those NMDA devices?
Since this is so urgent I would expect to see deployments by now.
There is no point in working on 3rd-party client apps until that happens.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 17 Jul 2018 18:23:47 +0200

> Not all devices have line cards.

There are more things you can plug these days. But yes, there are
devices that only have a power plug. ;-)

> If you read RFC 8040, it says RESTCONF is not intended to be a
> replacement for NETCONF so simple devices are likely to never need
> to report differences between intended and applied configuration.

I doubt that both implied statements "RESTCONF is only for simple
devices" and "simple devices do not need NMDA" are true. Time will
tell.

> Since this is so urgent I would expect to see deployments by now.

RFC 8349 appeared in March, the NMDA extensions for NC and RC got
stuck in the publishing queue. Lets be fair and give it some time.

/js

---

From: Andy Bierman <[email-masked]>
Date: Tue, 17 Jul 2018 09:59:39 -0700

> devices that only have a power plug. ;-)
And hard-wired interfaces that are not FRUs.

> devices" and "simple devices do not need NMDA" are true. Time will

Let the market decide, not the IETF.

> stuck in the publishing queue. Lets be fair and give it some time.

It is not as if the ability to detect a line card not plugged in or buggy
software
did not exist before NMDA.  It just requires separate "oper" and "state"
data model design.
NMDA can eliminate that extra leaf. The main design assumption by NMDA is
that there are
so many of these corner-cases that this should be the one-size-fits-all.
(Time will tell.)

/js

Andy