Subject: [netconf] draft-lindblad-netconf-transaction-id

From: Benoit Claise <[email-masked]>
Date: Tue, 21 Sep 2021 13:59:58 +0200

Jan, all,

I would like to restart the discussion regarding the draft 
draft-lindblad-netconf-transaction-id.

For me, the primary use case is a configuration revision id in a router, 
which would increase when the config changes; this would solve the 
concrete operational problem to know when a server configuration changed.
Note: I understand that the draft doesn't mention the notion of sequence 
or increasing id, but that would be a nice addition IMO.

The mechanism in the draft solves the issue of two servers that keep 
modifying the same router based on their conflicting policies, not aware 
of each other. When logging the transaction Id + NETCONF client 
responsible for the transaction directly in the router, we can find back 
who is responsible for the latest change, along with the specific 
request details directly from the NETCONF client.
This is the typical situation of NMS1 configure X, and NMS2 remove X. 
But also, NETFCONF versus CLI configuration.
Note: I understand that the idea of tracking which NMS is the originator 
for some change in the trans-id is currently not part of the draft. I 
believe it should be added.

Finally, combining an on-change notification on the revision-id variable 
(assuming it's leaf in a YANG module on the server), the configuration 
change would become an event for whoever tracks the network state.  And 
if we design this mechanism correctly, I hope we could link the 
transaction ID with a orchestrator service request. So it could help 
logging network wide service requests history and help the closed loop 
automation in the end.

Feedback?

Regards, Benoit

---

From: "Jan Lindblad (jlindbla)" <[email-masked]>
Date: Tue, 21 Sep 2021 15:56:46 +0000

Benoit,

> I would like to restart the discussion regarding the draft draft-lindblad-netconf-transaction-id.

This is excellent, and I would very much like to work with you and the WG on refining this draft.

The draft can be found here, if anyone is interested to have a look again
https://www.ietf.org/archive/id/draft-lindblad-netconf-transaction-id-00.txt

> For me, the primary use case is a configuration revision id in a router, which would increase when the config changes; this would solve the concrete operational problem to know when a server configuration changed.

This is exactly where I started in my pondering this problem. A standardized mechanism for obtaining a transaction id from a NETCONF server would be a good thing. Right now there are a number of proprietary mechanisms in circulation, not ideal. After starting to write a draft along these lines I realized a few ways I could improve the proposal.

A single per server top-level transaction id is actually not very useful in a broad range of use cases where multiple clients are involved. In environments where people care about transaction ids, there are multiple clients managing the servers. In many such cases, each client is managing configuration elements in different subtrees on a given server, or at least different instances of some list (such as the interface list). It would be useful for them to know when they are working side by side, and when they start clobbering each other. A single top level transaction id will not help there. By introducing a kind of transaction id on each container and list element, they could do just that.

> Note: I understand that the draft doesn't mention the notion of sequence or increasing id, but that would be a nice addition IMO.

That was my initial thought as well, and we can certainly discuss this. When I thought about this I found it quite useful that the transaction id was assigned by the client, not the server. This way the client knows the transaction id in advance and doesn't need to wait for the server to respond with the allocated id before moving on. In many real cases, this makes the client implementation a lot more efficient, but perhaps more importantly, it allows the client to issue new configuration changes towards the server even before the last one has completed, since it knows the transaction ids already. "Pipelining". By allowing the client to specify the transaction id, the same id can also be used across many servers in a network wide transaction, which is also quite useful for the audit trail. The id could even contain customer service request ids and similar.

> The mechanism in the draft solves the issue of two servers that keep modifying the same router based on their conflicting policies, not aware of each other. When logging the transaction Id + NETCONF client responsible for the transaction directly in the router, we can find back who is responsible for the latest change, along with the specific request details directly from the NETCONF client.
> This is the typical situation of NMS1 configure X, and NMS2 remove X. But also, NETFCONF versus CLI configuration.
> Note: I understand that the idea of tracking which NMS is the originator for some change in the trans-id is currently not part of the draft. I believe it should be added.

Agreed, this is useful. It's true that this idea (to identify the client in the id) is not mentioned in the current draft, but it is already possible for a client to do this by inserting its own name in the transaction id. I'm certainly open to formalizing this into a requirement if you like. Adding it doesn't change the basic principles of the mechanism in any way.

> Finally, combining an on-change notification on the revision-id variable (assuming it's leaf in a YANG module on the server), the configuration change would become an event for whoever tracks the network state.

I'd be interested to hear your thoughts on notifications in this context. We have RFC 6470, and in my mind, much of the reason for my work here is to remove the need for config change notifications, as they can easily get quite costly, both to send and to process as a receiver. Plus they are always "arriving late", so a client can never rely on being up to date regardless. You may have use cases that warrant some new form of change notifications, however, so I'd be interested to hear more.

>   And if we design this mechanism correctly, I hope we could link the transaction ID with a orchestrator service request. So it could help logging network wide service requests history and help the closed loop automation in the end.

Yes, exactly. I find this very appealing.

Best Regards,
/jan

---

From: Kent Watsen <[email-masked]>
Date: Fri, 24 Sep 2021 20:17:17 +0000

I support this work, though I prefer to see it exactly align with RESTCONF’s ETag and Timestamp.  Coincidentally, I recently filed this NETCONF-Next issue: https://github.com/netconf-wg/netconf-next/issues/12 <https://github.com/netconf-wg/netconf-next/issues/12>.

K.  // contributor

> On Sep 21, 2021, at 7:59 AM, Benoit Claise <[email-masked]> wrote:
> 
> Jan, all,
> 
> Finally, combining an on-change notification on the revision-id variable (assuming it's leaf in a YANG module on the server), the configuration change would become an event for whoever tracks the network state.  And if we design this mechanism correctly, I hope we could link the transaction ID with a orchestrator service request. So it could help logging network wide service requests history and help the closed loop automation in the end.
> 
> Feedback?
> 
> Regards, Benoit
> 
> 
> _______________________________________________
> netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: "Jan Lindblad (jlindbla)" <[email-masked]>
Date: Mon, 27 Sep 2021 21:01:34 +0000

Kent,

I support this work, though I prefer to see it exactly align with RESTCONF’s ETag and Timestamp.  Coincidentally, I recently filed this NETCONF-Next issue: https://github.com/netconf-wg/netconf-next/issues/12.

Very glad to hear it, Kent!

I completely understand your preference for "exactly aligning with RESTCONF's ETag and Timestamp." Indeed, that is where I started myself. However, I did find a couple of places where there might not exist an exact 1:1 mapping from RESTCONF to NETCONF behavior. Let me explain.

RFC 8040 sec 3.4.1.2 says:

   The server MUST maintain an entity-tag for the top-level
   {+restconf}/data resource

A NETCONF translation of that would probably be an entity-tag for each config datastore (e.g. running and candidate).

RFC 8040 sec 3.5.2 says:

   For configuration data resources, the server SHOULD maintain a
   resource entity-tag for each resource and return the "ETag" header
   field when it is retrieved as the target resource with the GET or
   HEAD methods.

Now, if we want similar entity-tags for non-top level objects in NETCONF, we would need some NETCONF compatible way of encoding that. There are no RESTCONF resources in NETCONF. In the draft, I proposed to use XML attributes for communicating entity-tags both for datastore entity-tags as well as any other "resource like" object. I propose that containers, lists and list entries would be classified as "resource like" here, and able to carry an entity-tag.

Next we have the If-Match header (and few other similar ones).

RFC 8040 sec 3.4.1.2 says:

   The client MAY use
   an "If-Match" header in edit operation requests to cause the server
   to reject the request if the resource entity-tag does not match the
   specified value.

Since there are no REST headers in NETCONF, they have to go somewhere else. We could place them as XML elements in the <edit-config> and <get-config> request. But this is only convenient if we want apply the If-Match concept on datastore entity-tags. Not if we wanted to match against entity-tags deeper in the data tree, such as a list element somewhere. In the draft, I proposed to use XML attributes for this as well. That allows using the same entity-tags matching mechanism against any "resource like" object in the data tree.

Another question is how we respond to a <get-config> request when it comes to entity-tags. I see a few options here:
a) Entity-tag values are only sent for the top level node of any request (as in RESTCONF)
b) Entity-tag values are sent for some containers, lists and list elements (server implementor decides which ones)
c) Entity-tag values are sent for all (supported) containers, lists and list elements

Option (a) will be very coarse grained, where clients will only know the entity-tags for the root of subtrees they frequently request. Option (b) is chattier and more fine grained, where the server implementor decides what is a good trade off. Option (c) is chattier still, but gives the client full freedom to choose how to interact.

In order to measure how good this idea is, I asked some of my colleagues for a NETCONF trace from a NETCONF client managing a NETCONF server. The system is a commercial production application, but running in a development lab. It should be a reasonable approximation of how a real world system could behave, albeit only one data point.

In the trace there are 1946 NETCONF messages exchanged over 3.5 hours between one particular client and server. The client has to be prepared that operators or other client interfere with the device configuration, but throughout the execution of this trace, no actual interference occurred.

The first observation is that 194 (10.6%) of the messages are hello messages (counting both directions). 10133 kB are spent on the hellos, which is 91.0% of the total communications between client and server in this trace. Not ideal I guess. This server has a lot of YANG 1.0 modules, which makes the hello a bit long winded. Luckily, I believe generating and processing the hello messages at least isn't particularly heavy on either server or client side. For the rest of the discussion below, I completely ignore the hellos. The 91% are henceforth gone.

Another observation is that this particular client and server both implement a proprietary, basic transaction-id mechanism which gives the client a way to detect top-level datastore changes through a <get> operation.

The properties I am interested to measure are the number of messages, the number of roundtrips and the number of bytes exchanged. In principle I would also like to know the CPU load of the systems involved, but this trace gives me no way to find out. As approximation, I will use the number of bytes sent and received in get-config and edit-config. Due to pipelining, i.e. sending multiple messages at once without waiting for an answer, the number of roundtrips (the number of times the client has to wait for a reply) is lower than the number of messages.

The trace contains
93 <edit-config> messages with 157 kB
1116 pipelined (279 roundtrips) "transaction overhead" messages (discard-changes, ok, lock, ... commit, ok) with 246 kB
6 <get-config> request messages with 91 kB
6 <data> responses with 305 kB
191 <get> transaction-id messages with 91 kB
191 <data> transaction-id responses with 111 kB

Total: 569 roundtrips, 1002 kB

I will use this "real world" example as baseline when considering what gains we might get by implementing one entity-tag mechanism or another. Using the mechanism described in the draft, I estimate the following behavior:

93 <edit-config> messages with 204 kB (up 30%)
1116 pipelined (279 roundtrips) "transaction overhead" messages with 246 kB (no change)
6 <get-config> request messages with 91 kB bytes (no change)
6 <data> responses with 6 kB (down 100%)

Total: 378 roundtrips (down 34%), 547 kB bytes (down 45%)

If we decide to go with option (a) above instead, i.e. only communicate the entity-tags on the top level object (or any one object, however we decide to encode this operation), we will reduce the information sent on the wire. This is great if the possibility of a clash with an operator or other management system is very rare. Here's the data:

93 <edit-config> messages with 158 kB (up 0%)
1116 pipelined (279 roundtrips) "transaction overhead" messages with 246 kB bytes (no change)
6 <get-config> request messages with 91 kB bytes (no change)
6 <data> responses with 6 kB bytes (down 100%)

Total: 378 roundtrips (down 34%), 500 kB bytes (down 50%)

As soon as a another manager does anything in the same datastore, a clash would be declared, and we're back to square one with option (a). If two managers are expected to routinely work side by side in the same datastore, but without actually clobbering each other, options (b) or (c) are far superior, as this diagram is trying to convey.

The blue line in the diagram shows the proprietary transaction-id mechanism, which lacks any kind of If-Match kind of functionality. The orange line shows the number of <edit-config> bytes sent, which also serves as an approximation of the CPU load, for different probabilities that another manager is doing anything in the same datastore in between two consecutive <edit-config> operations. If the probability that anyone else is doing something is near 0%, sending as little entity-tag information as possible is best. The gray line shows option (c) where entity-tag data is sent for every container, list and list element. Option (b) would be a horizontal line somewhere in between the gray and blue line, depending on how coarse granularity the server implementor would choose to go with.

While the above discussion has been about the choices of mapping a RESTCONF concept to a NETCONF world, there is one aspect where the draft I wrote purposely deviates from the RESTCONF Etag mechanism. The draft proposes that it is the *client* that assigns the entity-tag values, rather than the server assigning some value. This has two major advantages in my opinion.

i) This allows a client to send multiple edit-config requests towards a particular device without waiting for it to first complete the one outstanding. If the server assigns the entity-tag values, the client will not know what value to use for the If-Match header/attribute until the previous operation completes. For a high performance client that sends NETCONF messages to large numbers of servers every minute, having to stop and wait, possibly for several minutes, while a particular server to completes a request before it can send the next is .. not ideal.

ii) If the client can assign the entity-tag values, they can be used for tracking service orders and creating audit trails. This is a feature high on the wishlist of many people I know.

I hope this explains how I arrived at the proposed text in the draft. If you have other ideas of how we could make this a more exact match of the RESTCONF mechanism, I would be interested to hear.

Best Regards,
/jan

On Sep 21, 2021, at 7:59 AM, Benoit Claise <[email-masked]<mailto:[email-masked]>> wrote:

Jan, all,

Feedback?

Regards, Benoit

_______________________________________________
netconf mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netconf

---

From: Andy Bierman <[email-masked]>
Date: Tue, 28 Sep 2021 10:11:01 -0700

Hi,

I support discussion of the problem statement, but not this solution.
Note that the RESTCONF ETag features operate on a single resource, not an
arbitrary set of edits. This feature expansion needs careful thought.

There is some text in the draft about client-provided transaction-ids.
This is not at all how ETag works (for good reason).

If I delete a node in resource /foo/A such that data outside my transaction
in /bar/B changes
(e.g., false when-stmt) does the transaction-id for /bar/B change?
If I change /foo/A/AA doesn't that also change the txid for /foo/A and /foo?
If this feature is standardized then it must be clear to developers exactly
how these transaction-ids are changed.

Andy

On Fri, Sep 24, 2021 at 1:17 PM Kent Watsen <[email-masked]> wrote:

> I support this work, though I prefer to see it exactly align with
> RESTCONF’s ETag and Timestamp.  Coincidentally, I recently filed this
> NETCONF-Next issue: https://github.com/netconf-wg/netconf-next/issues/12.
>
> K.  // contributor
>
>
>
>
> which would increase when the config changes; this would solve the concrete
> operational problem to know when a server configuration changed.
> modifying the same router based on their conflicting policies, not aware of
> each other. When logging the transaction Id + NETCONF client responsible
> for the transaction directly in the router, we can find back who is
> responsible for the latest change, along with the specific request details
> directly from the NETCONF client.
> change would become an event for whoever tracks the network state.  And if
> we design this mechanism correctly, I hope we could link the transaction ID
> with a orchestrator service request. So it could help logging network wide
> service requests history and help the closed loop automation in the end.
>
> Feedback?
>
> Regards, Benoit
>
>

---

From: "Jan Lindblad (jlindbla)" <[email-masked]>
Date: Wed, 29 Sep 2021 07:37:55 +0000

Andy, WG,

Very nice. I look forward to working out a good solution together with you.

Absolutely. When I started thinking about this, my initial thought was to propose an "exact translation" of the RESTCONF ETag mechanism into NETCONF. Two reasons made me deviate from this. The first is that there is no "exact translation". The other that when I examined the results of the approximation I came up with, I noticed that it didn't yield the improvements I had hoped for in the use cases I was looking at. When I changed the source of the transaction ids, I saw several strong benefits (client pipelining, audit trails), as mentioned earlier in the thread.

If you feel the client side assignment of transaction ids is problematic, we could make that an optional feature. It may only be important for systems that participate in high cadence automation scenarios, but in those scenarios I believe this principle is crucial for good performance.

If I delete a node in resource /foo/A such that data outside my transaction in /bar/B changes
(e.g., false when-stmt) does the transaction-id for /bar/B change?

Good point, an example demonstrating this should be added.

Yes, of course. Both the client and server knows the YANG model in play, and are aware that changes in /foo/A will affect /bar/B.

Certainly. I believe the examples in the draft are already showing this.

Completely agree.

/jan

On Fri, Sep 24, 2021 at 1:17 PM Kent Watsen <[email-masked]<mailto:[email-masked]>> wrote:
I support this work, though I prefer to see it exactly align with RESTCONF’s ETag and Timestamp.  Coincidentally, I recently filed this NETCONF-Next issue: https://github.com/netconf-wg/netconf-next/issues/12.

K.  // contributor

Jan, all,

Feedback?

Regards, Benoit

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Sep 2021 02:48:49 -0700

On Wed, Sep 29, 2021 at 12:38 AM Jan Lindblad (jlindbla) <[email-masked]>
wrote:

> Absolutely. When I started thinking about this, my initial thought was to
> propose an "exact translation" of the RESTCONF ETag mechanism into NETCONF.
> Two reasons made me deviate from this. The first is that there is no "exact
> translation". The other that when I examined the results of the
> approximation I came up with, I noticed that it didn't yield the
> improvements I had hoped for in the use cases I was looking at. When I
> changed the source of the transaction ids, I saw several strong benefits
> (client pipelining, audit trails), as mentioned earlier in the thread.
>
> If you feel the client side assignment of transaction ids is problematic,
> we could make that an optional feature. It may only be important for
> systems that participate in high cadence automation scenarios, but in those
> scenarios I believe this principle is crucial for good performance.
>
>

This should not be used at all.
Explain what problem this solves.
It creates problems (e.g., storage requirements, duplicates).

> Yes, of course. Both the client and server knows the YANG model in play,
> and are aware that changes in /foo/A will affect /bar/B.
>

Andy

---

From: Benoit Claise <[email-masked]>
Date: Wed, 29 Sep 2021 14:06:07 +0200

Hi Jan,

> A single per server top-level transaction id is actually not very useful in a broad range of use cases where multiple clients are involved. In environments where people care about transaction ids,…
I believe that a single per server top-level id solves one of my 
specific problem.
The router config was just changed, who is responsible for that?
Or going back in the past, I start to see weird behaviours in my network 
today at 2:00 AM. Who was the last client who modified the config, what 
was the transaction Id => let me review the applied changes.
> That was my initial thought as well, and we can certainly discuss this. When I thought about this I found it quite useful that the transaction id was assigned by the client, not the server. This wa…
I understand where you're coming from. The (customer) service request 
ids is important to me as well.
> I'd be interested to hear your thoughts on notifications in this context.
I'd like to think about the following lines: the client assigns the 
transaction Id (as you proposed) but the server keeps the following 
mappings:
     - a single increasing id per server +associated time
     - which client applied the modification, along with transaction Id 
(ideally a service Id)
     - and maybe more info such as the "touched" subtree, the protocol 
(NETCONF, RESTCONF), the status (finished, ongoing, etc.)
So basically a YANG module in the server.
Add the on-change notification on the above YANG module, for 
communication with an entity monitoring the network state (which is, 
most of the time, NOT the orchestrator/controller) and we would have a 
flexible solution.

> We have RFC 6470, and in my mind, much of the reason for my work here is to remove the need for config change notifications, as they can easily get quite costly, both to send and to process as a receiver. Plus they are always "arriving late", so a client can never rely on being up to date regardless. You may have use cases that warrant some new form of change notifications, however, so I'd be interested to hear more.
>
Regards, Benoit

---

From: "Jan Lindblad (jlindbla)" <[email-masked]>
Date: Wed, 29 Sep 2021 13:55:56 +0000

Andy, all,

Certainly.

Let's say we have a NETCONF client orchestrating a network, with a queue of incoming orders. The client translates each order into a set of transactions towards a collection of NETCONF servers. As part of the transaction mechanics, the client will lock, check that the relevant server configuration has not changed, then go ahead with an edit-config towards the server.

The "check that the server configuration has not changed" can happen in many ways. Potentially, the client could issue a get-config and check everything. This is a bit heavy, so many servers implement some sort of global transaction id (txid) that can be fetched using an action or get. RESTCONF allows the server to do the check locally. The client just adds an If-Match header with the last known txid, and the server guarantees to only do the change if the txid matches. At the end of the change, it returns a new txid to use in the If-Match header in the next change.

The downside with this approach is that a client is effectively unable to issue another request towards the device until the previous request has been completed, since the client isn't informed of the txid value to use in the If-Match header until that time. I'm sure this delay is unimportant in many use cases, but in high cadence orchestration, waiting many seconds or a few minutes for a server to complete a configuration change is often very inconvenient, and this mechanism will be the gating factor that determines the pace at which the whole network can be orchestrated.

If we allow the client to supply the txid, this problem goes away. The client already knows the next txid to use at the outset. In addition, the client can then use the txid for tracking things, such as customer names or service order numbers, which I know many find useful.

May I ask you to elaborate on the problems you hint at above?

When it comes to storage requirements, I don't think there is much difference between what I propose and what's already required by RFC8040. If a server tracks the timestamp of last change for resources, as described in RFC 8040, it would not take much storage or processing to also keep a map of timestamp to txid strings. The txid strings would only need to be stored once per server.

/jan

Completely agree.

/jan

Andy

K.  // contributor

Jan, all,

Feedback?

Regards, Benoit

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Sep 2021 07:26:22 -0700

On Wed, Sep 29, 2021 at 6:56 AM Jan Lindblad (jlindbla) <[email-masked]>
wrote:

> Let's say we have a NETCONF client orchestrating a network, with a queue
> of incoming orders. The client translates each order into a set of
> transactions towards a collection of NETCONF servers. As part of the
> transaction mechanics, the client will lock, check that the relevant server
> configuration has not changed, then go ahead with an edit-config towards
> the server.
>
> The "check that the server configuration has not changed" can happen in
> many ways. Potentially, the client could issue a get-config and check
> everything. This is a bit heavy, so many servers implement some sort of
> global transaction id (txid) that can be fetched using an action or get.
> RESTCONF allows the server to do the check locally. The client just adds an
> If-Match header with the last known txid, and the server guarantees to only
> do the change if the txid matches. At the end of the change, it returns a
> new txid to use in the If-Match header in the next change.
>
> The downside with this approach is that a client is effectively unable to
> issue another request towards the device until the previous request has
> been completed, since the client isn't informed of the txid value to use in
> the If-Match header until that time. I'm sure this delay is unimportant in
> many use cases, but in high cadence orchestration, waiting many seconds or
> a few minutes for a server to complete a configuration change is often very
> inconvenient, and this mechanism will be the gating factor that determines
> the pace at which the whole network can be orchestrated.
>
> If we allow the client to supply the txid, this problem goes away. The
> client already knows the next txid to use at the outset. In addition, the
> client can then use the txid for tracking things, such as customer names or
> service order numbers, which I know many find useful.
>
>

The only way the client can be sure the config has not changed is to lock
it,
and so the transaction-id is irrelevant.  Whether the client is reading the
current txid
or setting a new artificial txid, no other client can change anything in
the locked config.
So if this client can decide what edit to send. No other edit can show up
instead
so picking the id does not help.

Retrieving the current txid does not take minutes, but rather milliseconds.
Operations (like lock) could be replaced so the current ID is returned,
removing even this 1 roundtrip.

> May I ask you to elaborate on the problems you hint at above?

Does the server check the txid to see if it is already used, or just allow
the id
to be used over? What if 2 clients happen to pick the same id?
What if the server has storage size limits?
Storage of an arbitrary string for every transaction can use a lot of
memory.

> When it comes to storage requirements, I don't think there is much
> difference between what I propose and what's already required by RFC8040.
> If a server tracks the timestamp of last change for resources, as described
> in RFC 8040, it would not take much storage or processing to also keep a
> map of timestamp to txid strings. The txid strings would only need to be
> stored once per server.
>
> /jan
>

Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 29 Sep 2021 07:56:19 -0700

> Explain what problem this solves.

I am trying to understand this new feature.
It seems locking is not involved.
It the client has the config locked then nobody else can change it,
so IfMatch and ETags are not needed.

If locking is not used, then the ETag becomes relevant.
The client (1) retrieves the resource with its current ETag and
(2) creates an edit based on the returned data node values.
Then it can (4) send an edit with If-Match(etag). The server rejects
the edit if the resource was changed since (1).

If the client sends an edit with If-Match(my-etag-value) then it will always
fail because the current target resource ETag has a different value and the
If-Match
test will not pass.  That seems to be the biggest problem with
client-provided ETags.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Wed, 29 Sep 2021 15:16:24 +0000

Clarifying my earlier “exact match” statement: I wish it to be possible that that server supporting both NC and RC may implement one (NC or RC) on top of the other.  The HTTP ETag/Timestamp attributes are quite good and absolutely trounces RFC 5717 partial locking when the attributes are sprinkled throughout the data tree.  RC lacks locks, which prevents both 2-phase commits (and efficient successive bulk-updates), so likely RC is on top of NC for now.  [Aside: several proposals for adding locks to RC have been floated on the list in the past, but never pick up enough traction.  I’m still optimistic that this will occur someday.]

Customer transaction tracking, which I’m not opposed to, is a different kind of problem.  That said, I don’t think such an ID should be used for consistency checking, for reasons Andy mentioned, as well as because it differs from RC.

K.

> On Sep 29, 2021, at 10:56 AM, Andy Bierman <[email-masked]> wrote:
> 
> fail because the current target resource ETag has a different value and the If-Match
> test will not pass.  That seems to be the biggest problem with client-provided ETags.
>

---

From: "Jan Lindblad (jlindbla)" <[email-masked]>
Date: Thu, 30 Sep 2021 10:23:39 +0000

Kent, Andy, Benoit, WG,

Kent wrote:

Yes, I agree that makes good sense. There will be some aspects that are NC specific because of the differences between NC and RC. Then we have the debate about which party assigns the txids. I'm fine with that as an optional feature, that some server implementors support. This rides on top and would be compatible with the RC ETag solution.

The main reason for me proposing that clients should have the possibility to assign the txids are efficiency. In high cadence management scenarios, forcing clients to wait for (slow) servers is not very scalable. The possibility to tag transactions with service order numbers (or whatever) is a real world feature already implemented by some servers because of customer demand.

Andy wrote:

It seems to me that you assume locks would be held for extended periods of time. That is not the case in the use cases I work with. An orchestrator client is not going to hold locks towards thousands of servers ever, and not going to hold locks for any extended period of time even towards the few servers involved in a current transaction.

The client would take a lock in order to obtain the current txid and ensure it is still valid a moment later when following up with an edit-config (assuming the txid matched the client's expectation).

Even with a If-Match feature implemented, the client would probably lock the server, just to get transactional behavior between the edit-config, validate, commit and confirm.

The timeline with steps (1), (2) and (4) matches the use case I have in mind, but I think it may be worth pointing out that (1) may happen quite some time before (2), say 17 days. Not a stretch of time you'd like to keep a lock for.

If the client sends an edit with If-Match(my-etag-value) then it will always
fail because the current target resource ETag has a different value and the If-Match
test will not pass.  That seems to be the biggest problem with client-provided ETags

There are two mechanisms at play here. One is the If-Match feature, where the client can adorn the edit-config with transaction ids for select containers, lists and list entries. If the server sees that any of them are amiss, the transaction is aborted. This is similar to how things already work in RFC8040, except that only one If-Match header can be supplied in RC. That approach needs a bit of translation in NC. I propose to use attributes, and in that case we might as well allow adding more than one If-Match attribute.

If the server goes through with the edit-config, it needs to update the ETag and timestamp value of all changed nodes. It could either make up a random value, or it could accept a suggestion from the client, if provided. There are use cases where allowing the client to suggest the new value makes a big difference in scalability.

Andy also wrote:

The only way the client can be sure the config has not changed is to lock it,
and so the transaction-id is irrelevant.  Whether the client is reading the current txid
or setting a new artificial txid, no other client can change anything in the locked config.
So if this client can decide what edit to send. No other edit can show up instead
so picking the id does not help.

See discussion about locks above.

While removing roundtrips is always nice (and this roundtrip would be removed by the proposed draft), I'm not worried about this roundtrip. That is not the point here.

Does the server check the txid to see if it is already used, or just allow the id
to be used over?

There is no such check in the proposal. If a client wants to do this for whatever (good or bad) reason, I don't see this being important enough to prevent.

What if 2 clients happen to pick the same id?

I'd say that's the client's problem.

What if the server has storage size limits?
Storage of an arbitrary string for every transaction can use a lot of memory.

All servers I have worked with have had a storage size limit. Clients that don't need the ability to assign the txid are free not to. If that saves any measurable amount of memory is unclear to me, but I do know that quite a few NC users out there today, when given the choice, have decided that these are bytes well worth spending.

Benoit wrote:

I believe that a single per server top-level id solves one of my specific problem.
The router config was just changed, who is responsible for that?
Or going back in the past, I start to see weird behaviours in my network today at 2:00 AM. Who was the last client who modified the config, what was the transaction Id => let me review the applied changes.
...
I'd like to think about the following lines: the client assigns the transaction Id (as you proposed) but the server keeps the following mappings:
    - a single increasing id per server +associated time
    - which client applied the modification, along with transaction Id (ideally a service Id)
    - and maybe more info such as the "touched" subtree, the protocol (NETCONF, RESTCONF), the status (finished, ongoing, etc.)
So basically a YANG module in the server.
Add the on-change notification on the above YANG module, for communication with an entity monitoring the network state (which is, most of the time, NOT the orchestrator/controller) and we would have a flexible solution.

I agree that the functionality you describe is useful and note that many servers implement something proprietary along these lines. I also think that the transaction-id mechanism would a good base component to import into a standard model for this purpose. The transaction-id draft is a NETCONF specific, low-level protocol mechanism. I think much of the functionality you describe here should go in a separate module, and probably discussed in NETMOD.

Best Regards,
/jan

---

From: Andy Bierman <[email-masked]>
Date: Thu, 30 Sep 2021 08:05:44 -0700

On Wed, Sep 29, 2021 at 8:16 AM Kent Watsen <[email-masked]> wrote:

> Clarifying my earlier “exact match” statement: I wish it to be possible
> that that server supporting both NC and RC may implement one (NC or RC) on
> top of the other.  The HTTP ETag/Timestamp attributes are quite good and
> absolutely trounces RFC 5717 partial locking when the attributes are
> sprinkled throughout the data tree.  RC lacks locks, which prevents both
> 2-phase commits (and efficient successive bulk-updates), so likely RC is on
> top of NC for now.  [Aside: several proposals for adding locks to RC have
> been floated on the list in the past, but never pick up enough traction.
> I’m still optimistic that this will occur someday.]
>

I am not aware of any actual usage of RFC 5717.
Global locking is too coarse a lock and rarely gets used.
The lightweight edit collision detection provided by the well-known ETag
and associated headers like If-Match could be useful in NETCONF.

> Customer transaction tracking, which I’m not opposed to, is a different
> kind of problem.  That said, I don’t think such an ID should be used for
> consistency checking, for reasons Andy mentioned, as well as because it
> differs from RC.
>
>
Adding a comment to an edit transaction does not really provide any
consistency checking.
It is completely unrelated to the problem of detecting changes to a target
resource or datastore
due to multiple clients modifying the configuration at once.

Andy