Subject: [Netconf] the real problem with defaults

From: Andy Bierman <[email-masked]>
Date: Fri, 07 May 2010 15:50:51 -0700

Hi,

There is a real problem to be solved, and it isn't GUI rendering tags
for thin clients.  (Use YANG or read the module and hard-code the GUI.)

The problem with defaults is that they are defined in YANG
to NOT exist in the conceptual database.  This has a major
impact on the create and delete operations, because they
are only allowed if the data does not exist or exist (respectively).

What does that mean for the client developer?

 basic-mode       problem exists?
  report-all          yes
  explicit            no
  trim                yes

The problem is that report-all and trim do not distinguish between
nodes that exist and do not exist in their filtering algorithm.

The information that is needed is really the 'explicit' mode reply.
If you receive a node <foo> in an 'explicit' reply, you know
it can be deleted with nc:operation='delete'.  If the node does not exist
in the reply, you know that it can be created with nc:operation='create'.

The information that is needed is not whether the node equals
the schema default, but rather whether it was set explicitly,
or whether it is a conceptual default that only exists in the
reply, not in the database.

Only a basic-mode=explicit server is required to save that information.
Maybe explicit mode should be mandatory instead of report-all.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 08 May 2010 10:00:23 +0200

> to NOT exist in the conceptual database.  This has a major

Where does the draft say so? On the contrary, section 7.6.1 states:

When the default value is in use, the server MUST operationally
behave as is if the leaf was present in the data tree with
the default value as its value.

Also, in XPath evaluations the missing leaves with default values are conceptually added.

If it walks like a duck and quacks like a duck, then it's a "conceptual duck".

Lada

> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 8 May 2010 10:42:43 +0200

On Sat, May 08, 2010 at 10:00:23AM +0200, Ladislav Lhotka wrote:
> Andy Bierman p????e v P?? 07. 05. 2010 v 15:50 -0700:
> > Hi,
> > 

"MUST operationally be have as if the leaf was present" is not the
same as the leaf exists in the configuration datastore.
 
> Also, in XPath evaluations the missing leaves with default values

Yes, conceptionally. The wording was not choosen randomly.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 08 May 2010 11:19:57 +0200

> same as the leaf exists in the configuration datastore.

Andy wrote about *conceptual* datastore. I think we agreed in a Jabber
conference some time ago that the conceptual datastore does contain the
default values.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sat, 8 May 2010 11:52:28 +0200

On Sat, May 08, 2010 at 11:19:57AM +0200, Ladislav Lhotka wrote:
> Juergen Schoenwaelder p????e v So 08. 05. 2010 v 10:42 +0200:

If you want to make an argument that explicit mode contradicts
NETCONF, then you have to argue based on what is written down in the
specs; there have been careful wording exercises to deal with
this. And if you were successful, you will of course declare deployed
implementations non-compliant. Good luck with that.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 08 May 2010 13:08:13 +0200

> NETCONF, then you have to argue based on what is written down in the

Please have a look at the text above: did I say anything in this sense?
I only have hoped that we agreed on what the *conceptual* datastore
contains. If it is not the case, I give up.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 08 May 2010 05:07:01 -0700

> contains. If it is not the case, I give up.

Nobody has answered the relevant question.
What is the correct standard response that every
NETCONF server will give the same response:

   leaf foo {
      type int32;
      default 42;
   }

   <rpc>
      <edit-config>
         <target><running/></target>
         <config>
              <foo nc:operation="create">42</foo>
         </config>
     </edit-config>
   </rpc>

If 'conceptually exists' means the same as 'exists',
then this MUST fail with a data-exists error.

If it doesn't fail, and the 'create'  works instead, then
what happens the 2nd time the client sends this create PDU?
Does it fail this time?  It must, if the server OKed the first create PDU.

But it can't exist, because defaults don't exist in the database.
Yet when the server reboots, the default is in the database
in an explicit-mode server.

The same paradox exists for the 'delete' operation.
Can you delete a leaf set to its default value?
What happens when you delete a leaf with a default, but it
is not set to its default value?  Does the server send <ok>
or send a 'data-missing' error?

Does every type of defaults-handling server implement create/delete
operations the exact same way so a client can use NETCONF,
or is NETCONF an unusable mess because we can't even decide
that the create/delete operations work the same way on all nodes?

If there is a difference between client-set-to-default
and server-set-to-default, then how does a client know
which values were set by some client, and which were set
by the server?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sat, 08 May 2010 14:46:44 +0200

> then this MUST fail with a data-exists error.

Yes, "the server MUST operationally behave as if the leaf was present".
Of course, we have this weasel word "operationally" here, but it should
include replies to operations, shouldn't it?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Sat, 08 May 2010 09:14:04 -0700

> include replies to operations, shouldn't it?

Do the edit-config operations apply to the conceptual database
or to the YANG notion of a database + defaults off to the side.

I agree with you, but I am not sure the standard is clear
or the entire WG agrees.

So any create operation at all (doesn't have to be the schema default)
on a leaf with a YANG default MUST fail with a data-exists error.

If the data exists, does that imply that deleting a leaf with a YANG
default MUST succeed (except for other constraint violations),
even though the server will create that leaf immediately with
the YANG default value?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 08 May 2010 11:49:39 -0700

Ladislav Lhotka wrote:
> ...

The important thing is that every NETCONF operation
work the same on every server implementation.
Since every server with a writable database
must implement edit-config, it is important that
create and delete work the same every time.

There are 2 choices:
 
  1) a default leaf is NEVER in the database (for edit purposes)
  2) a default leaf is ALWAYS in the database (for edit purposes)

The problem with (1) is that a basic-mode=report-all server
is not required to maintain any notion of a default leaf.
We already use (2) for XPath (must/when) validation purposes, so it
seems consistent to do the same for editing purposes.

If (1), a create on a leaf==default will work.
If (2), a on a leaf==default will fail.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 08 May 2010 22:51:08 +0200 (CEST)

> create and delete work the same every time.

Don't they work the same?  If you do a get-config on the leaf and it
is there, then create will fail.  If it is not there, create will
succeed (*).  The difference is that if you create the parent node,
the result of a following get-config will be different, but that is
expected.  A client can adapt by checking the basic-mode.

(*) the weird thing is that in trim mode, create with a value that
matches the default value is a noop, and in report-all mode delete on
a leaf with default is a noop.

Remember that we're trying to support 3 modes that we know that
existing devices use today.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sat, 08 May 2010 14:21:43 -0700

> expected.  A client can adapt by checking the basic-mode.

So now you have to do a get-config first to know
if a top-level leaf is there or not?  The YANG draft
did not make this clear.

Also, unless the server implements the :with-defaults
capability, it does not have to adhere to any particular
basic-mode for defaults handling.

> existing devices use today.

I don't see how a client-set-default can exist in the database
according to the YANG spec.  It clearly says that defaults
do not exist in any database.  That clearly means that
the create operation MUST always succeed on a leaf that
contains the default value, since it cannot be a default
and also be included in a YANG defined NETCONF database.

Either that, or the edit-config works on a
conceptual database (like must-stmt) so
the create operation does not change if the leaf
happens to contain a default, or happens to be implemented
as a place-holder instead of a real leaf.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sat, 08 May 2010 23:39:18 +0200 (CEST)

> if a top-level leaf is there or not?  The YANG draft

This has nothing to do with YANG; this is what we get from trying to
support the three modes in the with-defaults spec.

> basic-mode for defaults handling.

Right, and that problem is what with-defaults is trying to solve, and
this is why we recommend that implementations support with-defaults.

> do not exist in any database.

It says:

  The default value of a leaf is the value that the server uses if the
  leaf does not exist in the data tree.

But it does not prohibit a report-all server to *create* the leaf,
with the default value found in the YANG module as the value.  But
after that the leaf exists in the data tree, just as any other leaf.

And it does not prohibit a client to set the leaf with a value that
happens to be the default value found in the YANG module.

> That clearly means that

How do you know if a leaf contains the deafault value -- you do a
get-config.  In all modes, if the reply contains the default value you
cannot create it (of course in trim mode it will enever contain the
default value...)

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sat, 08 May 2010 14:56:28 -0700

>   leaf does not exist in the data tree.

It also says:

   When the default value is in use, the server MUST operationally
   behave as is if the leaf was present in the data tree with the
   default value as its value.

This would lead people like Lada to believe that the server
MUST reject a create on a default value in use.

> cannot create it (of course in trim mode it will enever contain the

I can live with this solution, but I don't think the
YANG text supports it.

The with-defaults draft also needs to say that the server MUST NOT
change its basic-mode while any sessions are active (skirt
the capability-changed as optional issue).

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 09 May 2010 00:04:09 +0200 (CEST)

> MUST reject a create on a default value in use.

I thought we agreed on the "operationally behave" text, but if it
isn't clear, what can we write instead?

> I can live with this solution, but I don't think the

Because of the "operationally behave" text, or something else?

> the capability-changed as optional issue).

Ok.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sat, 08 May 2010 16:29:42 -0700

> isn't clear, what can we write instead?

I don't know what text you thought we all agreed on.
I know what the draft says now, and the text
'MUST operationally behave' sure sounds to me
like it applies to all protocol operations.
If it only applied to get-config, it would
say that, but it doesn't.

I think we need a definition of 'operational behavior'
in the terminology section, and the IETF needs to
agree on its meaning, if it doesn't mean all
protocol operations.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 9 May 2010 12:02:28 +0200

> agree on its meaning, if it doesn't mean all

To me, 'operationally behave' has nothing to do with NETCONF protocol
operations per se. If I have

      container vt220 {
          color {
              type enum { enum green; enum amber; }
              default green;
          }
      }

and vt220/color does not exist in explicit config, then the termial
will use green color. It operationally behaves as if the vt220/color
data node would exist and have the value green. In other words, there
is no config state, just operational state.

If we would know how to deal with operational state in general, I
believe defaults would fall out naturally. (And one option could be to
have a get kind of operation that returns the union of config state
and operational state and tagging might help to distinguish data nodes
representing config and operational state. Another alternative could
be separate get kind of operations that just retrieve operational
state. Yet another alternative is to introduce another special
datastore <operational/> to access operational state. And I believe we
discussed even more ideas in the past.) The question is whether people
here are willing to solve the underlying problem or just defaults,
postponing the other question until it pops up when people write data
models that need to deal with mixtures of operational and config
states, e.g. forwarding and routing tables as prominent examples.

/js

---

From: Andy Bierman <[email-masked]>
Date: Sun, 09 May 2010 06:43:16 -0700

> states, e.g. forwarding and routing tables as prominent examples.
Why can't your example be config?
I had an old monitor with an 'amber/green' switch
about 20 years ago.  I controlled the color.
That makes it config, not operational state.

The issue is the text the Lada and I pointed out
in the section about default leafs.  It clearly
says 'operationally behave' as if the leaf exists.

Rather than start all until we have a solution
that happens to fit a poorly defined 'explicit' config,
we need to clarify how create and delete operations
are expected to work in the standard, given the normative
text that exists.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 09 May 2010 15:53:17 +0200 (CEST)

> I don't know what text you thought we all agreed on.

This was part of the looong "meaning of default" threads.

> like it applies to all protocol operations.

Before "operationally behave" we had "internally use" or something
similar.

Do we have a common term that refers to the application code /
instrumentation code behind the NETCONF server on the device?  The
intention of the text is to say that this code (the application on the
device that is being configured), MUST behave as if the leaf was
explicitly set to its default value, even if it is not there in the
config.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Sun, 09 May 2010 08:06:02 -0700

> explicitly set to its default value, even if it is not there in the

Why doesn't it refer to the NETCONF protocol operations?
I think the text should say "server instrumentation behaves",
not "operationally behaves".  I also think the draft does not
make it clear that a server MAY choose to store a default
in the database (or rather its external XML representation).

I suppose the answer for all the standards fragility is
"Just use merge.  Create is picky.  Merge isn't."

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 08:33:39 +0200

> Why can't your example be config?

If can, but without explicit config, the system will choose green
color.

> That makes it config, not operational state.

Sorry for picking an example that does not match your old monitor.

> says 'operationally behave' as if the leaf exists.

I tried to explain what "operationally behave" means, look like I
failed.

> are expected to work in the standard, given the normative

Look at your favourite server software and chances are good that if
you leave out explicit config in the config file, it will pick a
default value. Explicit config models this config file approach which
has worked for 20+ years. You may call this 'poorly designed' - thats
your choice.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 10 May 2010 08:41:25 +0200

> Don't they work the same?  If you do a get-config on the leaf and it

No, there are subtle differences in their behaviour.

> succeed (*).  The difference is that if you create the parent node,

But as you write in (*) below, *repeated* creation of the same leaf with
the default value will succeed for the trim mode and fail for the
explicit mode.

> existing devices use today.

My understanding so far was that we agreed the conceptual data tree
contains all "active" configuration data (that are also used in XPath
evaluations), i.e., including default values. The sentence

   When the default value is in use, the server MUST operationally
   behave as if the leaf was present in the data tree with the
   default value as its value.

then makes perfect sense to me. I believe servers operating in all three
default-handling modes could easily accommodate to this "mental" model
and behave identically while keeping the internal representation of
their database. The trim- and explicit-mode servers would have to keep
an eye on the data model though, so it may require adjustments to the
behaviour of *existing* server implementations. This is, I guess, the
reason that we are not able to come to a real consensus after all the
long discussions.

However, allowing server implementors to work with different contents of
the *conceptual* data tree really means that the behaviour will differ,
too. If it was allowed, the servers MUST report their basic-mode of
defaults handling and this should be stated in 4741bis. The client
cannot learn about the server's mode simply by getting the configuration
(once), it would have to perform some experiments.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Bert (IETF) Wijnen" <[email-masked]>
Date: Mon, 10 May 2010 13:57:38 +0200

WG participants, I have now read the postings of the last week twice.

I can't say that I am reading any consensus. I also can't say that all 
the arguments
that have seen used are 100% clear to me (but that is because I am far 
less versed
in this material than the implementers).

In fact, it seems to me, that we clearly interpret the text related to 
"defaults" and with-defaults"
differently. And that is... the people who actually developed the text 
and have implemented
most (if not all of it). If even those people have different 
interpretations, what does that mean
for someone new who must implement this? Or someone new who needs to 
write a
data model using YANG?

---

There was also discussion on merging all (or part of) with-defaults with 
4741bis.
Again no clear consensus, although there I would be ready to conclude 
(WG chair
reading of the discussions) that I think that there is enough support to 
stick to our charter
and deliver the documents as described there. Or better, that I do not 
see enough support
to actually deviate from hat weagreed upon in the charter. So no merging.

---

But I am reluctant to declare we have clarity and consensus on the whole 
defaults/with-defaults
story at this point in time.

---

As was stated a few times, we thought we reached consensus on this quite 
a while ago.
It seems that the way we documented it is not clear enough.

Is anyone ready to do a comprehensive proposal as to what text should be 
changed or added
in order to try to see if we can agree on such changes?

Bert

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 10 May 2010 14:27:38 +0200

> for someone new who must implement this? Or someone new who needs to 

It may help to think of it in terms of the model-view-controller
paradigm. The conceptual data tree is the model and "trim", "explicit"
and "report-all" styles are different views of the model. The
"controller" part are then the NETCONF operations.

In my view, the operations can work consistently for all implementations
only if the model is the same for everybody, so an agreement on a common
model is IMO badly needed but has never happened. It needn't be the
"report-all" style, but since this style is already used in YANG for
XPath evaluations, it seems to be a good candidate.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 16:08:40 +0200

On Mon, May 10, 2010 at 01:57:38PM +0200, Bert (IETF) Wijnen wrote:
 
> ---
> 
> There was also discussion on merging all (or part of) with-defaults
> with 4741bis.  Again no clear consensus, although there I would be
> ready to conclude (WG chair reading of the discussions) that I think
> that there is enough support to stick to our charter and deliver the
> documents as described there. Or better, that I do not see enough
> support to actually deviate from hat weagreed upon in the
> charter. So no merging.

I continue to believe that it is a bug that 4741 does not tell a
client what it does wrt defaults and I would prefer that if we can
avoid that with 4741bis clients. The current with-defaults ID as such
might not be implemented because it does more than just announcing the
behaviour and thus does not solve the problem.

> ---
> 
> But I am reluctant to declare we have clarity and consensus on the
> whole defaults/with-defaults story at this point in time.

And this is why a decision to stick to the charter is IMHO premature.
Lada, for example, made a proposal to move the with-defaults behaviour
announcement to 4741bis while keeping the additional argument for
operations in the with-defaults document. We should not rule out this
option. (But perhaps this also was not your intention, but this is
unclear.)

> ---
> 
> As was stated a few times, we thought we reached consensus on this
> quite a while ago.  It seems that the way we documented it is not
> clear enough.
> 
> Is anyone ready to do a comprehensive proposal as to what text
> should be changed or added in order to try to see if we can agree on
> such changes?

Perhaps the concensus was not clear or it was only clear up to the
point where some implementors decided what is in the extra document
does not need to be implemented - so they can have concensus by
ignoring it. Not sure this is what we want at the end.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 10 May 2010 16:18:10 +0200

> behaviour and thus does not solve the problem.

+1. If the server behaviour was allowed to depend on the strategy of
handling the defaults, it certainly is a bug not to report the strategy.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 May 2010 08:28:18 -0700

Bert (IETF) Wijnen wrote:
> with 4741bis.
> reading of the discussions) that I think that there is enough support
> to stick to our charter
> quite a while ago.
> Is anyone ready to do a comprehensive proposal as to what text should
> be changed or added

This debate is really over how standard the NETCONF protocol
is going to be.  Should the create and delete operations behave
the same way on all NETCONF servers, going forward?
We know pre-standard that everybody does 'defaults'
differently.  That doesn't mean post-standard that we are
trying to preserve this technical diversity.

Sometimes we compromise (like with-defaults, candidate, startup)
and we end up with a small number of acceptable solutions.
Martin is arguing that is the best we can do in this case.
Perhaps he is right.  The heuristic he suggests seems like
an acceptable compromise -- the server MUST honor
its own basic-mode for create/delete.  If a get-config
in basic-mode contains the node, the server considers
the node to be present.  If the WG agrees, I can propose
some new text for the with-defaults draft.

I agree that a 'new' server MUST advertise its basic-mode.
That doesn't mean a 4741bis server -- it just means a server
that implements with-defaults.  I do not want to change the
RFC packaging of the NETCONF standard.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 17:38:54 +0200

> RFC packaging of the NETCONF standard.

But this won't fly if people do not implement with-defaults because it
also adds additional parameters to operations they do not want to
support.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 May 2010 08:53:11 -0700

> also adds additional parameters to operations they do not want to

Too bad -- the WG already agreed that a server SHOULD
support report-all.  If the vendor ignores the standard
and says to its customers "You already know how to
look that info up on our WEB site, so you don't need
that info from the server."

Every NETCONF server knows its own total set of
supported leafs, so every server can implement
report-all.

The diversity we are maintaining is the ability for
the server to choose its basic mode, not the ability
to decide that report-all is never needed.  The client
will decide whether to use report-all or not.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 10 May 2010 18:01:22 +0200

> RFC packaging of the NETCONF standard.

What will then be the value of 4741bis conformance, if a configuration
script that works for server A (using "trim") fails for server B (using
"explicit"), both servers fully supporting 4741bis and the same set of
YANG modules?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 May 2010 09:11:42 -0700

> "explicit"), both servers fully supporting 4741bis and the same set of

I do not see how the particular RFC numbers affect
protocol conformance.  An implementer has to
deal with SSH details in RFC 4742 that are even
more critical to interoperability.

The theoretical problem is that a server will change
its basic-mode and a client will get confused.
In reality, this is not going to happen.  There are some
code optimizations a client can use if it knows
the basic-mode, but certainly not critical to the protocol.

In standards-think, of course a server MUST advertise
its basic-mode.  But as long as the server sticks to
just 1 basic-mode, the client can get by.  In standards-think,
of course the server MUST stick to 1 basic-mode
at a time.  This is a CLR that vendors already follow.

So (in summary) I take back 'server MUST advertise
its basic-mode'.  I think the WG already made the right
decision with 'SHOULD implement with-defaults'.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Mon, 10 May 2010 12:03:33 -0400

"Bert (IETF) Wijnen" writes:
>changed or added in order to try to see if we can agree on such changes?

Perhaps the best path would be to turn the various issues raised
by our endless discussions on defaults into content in the with-defaults
draft, which seems like a place where it should be covered.  The
current coverage in the draft is fairly brief and does not give
the reader any sense of the joy we've been having on this and the
netmod mailing lists.

I think our discussions on the issue of defaults get very complex
very quickly, but the root issues are both minor and straight
forward:

(a) what happens when you "create" a leaf that has as default value?
(b) what happens when you "merge" that leaf to the default value?
(c) what happens when you "delete" a leaf that has been set to the default value?
(d) what happens when you "delete" that freshly deleted leaf a second time?

What happens if you perform these operations in sequence on the
same leaf?  Which of these are errors?  The "natural" answer depends
on how your database represents default values.

I use the "explicit" model in JUNOS and strongly distinquish between
default values and human- (or client-) provided data.  The database
holds "real" data, and default values are provided to internal
software components that query the database (and even that API will
tell the software if the value is explicitly set or a default).
This is a very natural way for CLIs to work and seems to match
customer, user, and programmer expectations.  So the "explicit"
answers to the above questions are:
  (a) create succeeds; the leaf now exists in the database
  (b) merge succeeds; the leaf still exists in the database
  (c) delete succeeds; the leaf is no longer in the database
  (d) delete fails; the leaf cannot be deleted since it does not exist

The "trim" model is drawn from IOS CLI (and similar clones) where
leafs that are set to their default values are discarded from the
configuration database.  So the "trim" answers to the above questions
are:
  (a) create succeeds; the leaf now exists in the database
  (b) merge succeeds; the leaf is removed from the database
  (c) delete fails; the leaf cannot be deleted since it does not exist
  (d) delete fails; the leaf cannot be deleted since it does not exist

The "report-all" model treats default values identically to "real"
data.  So default values appear automatically and removing a leaf
with a default value causes that leaf to reappear with the default
value.  The "report-all" answers to the above questions are:
  (a) create fails; the leaf already exists
  (b) merge succeeds; the leaf is unchanged (still has the default)
  (c) the delete succeeds; the leaf is recreated with the default
  (d) the delete succeeds; the leaf is recreated with the default

I would find it confusing to see a leaf that I just deleted reappear,
or to have a create fail for a list member that I just created.
If I make a list member and populate it with 2 leafs, I don't
want to see 20 leafs when I fetch it.  I want to see the data
that I explicitly set and nothing more.  I can find default values
in the appropriate YANG module, if my client even needs them.

IMHO this way of mixing defaults into configuration data is similar
to the SNMP world, where there was no way to distinquish between
config data and operational data.  Maybe this approach to defaults
is also tied to SNMP MIBs, where default values were suggestions,
not part of the contract both sides were expected to follow.

In any case, there is are three distinct camps and no side is willing
to live by the rules that would be enforced if one model were chosen.
We have marched through this ground repeatedly and I don't see much
value to crawling back into these trenches.  Let's try to live by
the rules of the last treaty and give each other the flexibility
to support our specific model.

Thanks,
 Phil

P.s.:  With any luck, the following YANG module and NETCONF opeation
will fill in some details about (a)-(d) above.
P.p.s: Please don't worry about namespace/etc issues, as this is
really more for discussion than for real testing.

module test {
    container test {
        list peer {
            key name;
            leaf name {
                type string;
            }
            leaf version {
                type uint8;
                default 1;
            }
        }
    }
}

<rpc>
    <edit-config>
        <configuration>
            <test op="merge">

		<!-- (a) what happens when you "create" a leaf that
		  -- has as default value?  -->

                <peer op="create">
                    <name>fred</name>
                    <version op="create">2</version>
                </peer>

		<!-- (b) what happens when you "merge" that leaf
		  -- to the default value?  -->

                <peer>
                    <name>fred</name>
                    <version>1</version
                </peer>

		<!-- (c) what happens when you "delete" a leaf that
		  -- has been set to the default value?  -->

                <peer>
                    <name>fred</name>
                    <version op="delete"/>
                </peer>

		<!-- (d) what happens when you "delete" that freshly
		  -- deleted leaf a second time?  -->

                <peer>
                    <name>fred</name>
                    <version op="delete"/>
                </peer>

            </test>
        </configuration>
    </edit-config>
</rpc>

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 10 May 2010 18:30:52 +0200

> more critical to interoperability.

RFC4741bis says: "A NETCONF implementation MUST support the SSH
transport protocol mapping [4]." Will it also say that a NETCONF
implementation SHOULD implement with-defaults?

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 May 2010 09:52:49 -0700

> implementation SHOULD implement with-defaults?

IMO, there is no need for a normative reference
from 4741bis to with-defaults.  That is for the IESG
to decide.

I think it could be argued that with-defaults provides
additional monitoring functionality to a NETCONF server.
We don't say that a server SHOULD implement
NETCONF Monitoring in 4741bis.  I don't think
circular normative references exist here either.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 May 2010 10:08:13 -0700

>> changed or added in order to try to see if we can agree on such changes?

This is a nice summary of the details.
I would just change 'merge' to 'merge or replace'.

Clearly we are using <get-config> to report
the configured value and/or the operational value in use.
Depending on your use-cases, this is either a problem
or completely irrelevant.

I do not find report-all confusing.
If a YANG leaf has a default value,
then deleting it is where the confusion lies.
It is requires to operationally exist after the deletion.

The confusion is that different vendors do not agree
on the meaning of configuration.

The 'explicit' camp calls it anything set by the client.

The 'trim' camp calls it anything the server would not
normally do on its own.

The 'report-all' camp calls it any node that is
currently in use, whether the client or the
server provided the value.

I don't think any 1 of them is right or better
for every possible use case.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 19:25:46 +0200

On Mon, May 10, 2010 at 05:53:11PM +0200, Andy Bierman wrote:
> support report-all.

I want a "MUST announce" as part of 4741bis such that my client can
learn how a server will react to basic edit-config operations. Knowing
how a server handles defaults clarifies how NETCONF works. The
additional with-defaults parameter can happily proceed in a separate
document.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 19:29:28 +0200

> (d) what happens when you "delete" that freshly deleted leaf a second time?

[...]
 
>   (d) delete fails; the leaf cannot be deleted since it does not exist

I do not understand (a) and (b) in the trim model - please explain.

> config data and operational data.

I agree - we need to look at the larger operational state problem.

/js

---

From: Phil Shafer <[email-masked]>
Date: Mon, 10 May 2010 13:17:41 -0400

>I do not understand (a) and (b) in the trim model - please explain.

In the "trim" model, setting a leaf to the non-default value creates
the leaf.  I guess I should have been clear that (a) is referring to
the non-default value:

- (a) what happens when you "create" a leaf that has as default value but
      give it a value that is not equal to the default value?
+ (a) what happens when you "create" a leaf that has as default value?

Setting a leaf to the default value removes it from the configuration
database.  So when I set "version" to "1", is disappears.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Mon, 10 May 2010 13:25:22 -0400

>the configured value and/or the operational value in use.

Given that one of our goals was "to clearly distinguish between
configuration data and operational data" [rfc3535], this is mixing
is clearly a bad thing.

>It is requires to operationally exist after the deletion.

I delete it but it's still there.  That seems to be an obvious
source of confusion.  (Here I'm not referring to reporting all when
getting config (which is uninteresting) but to servers that treat
defaults as part of the configuration (in a not conceptual way)).

>for every possible use case.

Agreed.  So can we stick with the agreement we made the last time
we walked through this subject?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Mon, 10 May 2010 10:53:56 -0700

> additional with-defaults parameter can happily proceed in a separate

I do not agree at all that 4741bis has to be the document
that contains this capability.  I do not even agree that MUST announce
is needed.  A simple <get-config> will answer this
question, and simply using 'merge' instead of 'create'
avoids most the problems completely.

The with-defaults capability cannot simply be moved to another document.
That will require a massive rewrite an re-reviewing everything. 
The 4741bis document already has a long list of open issues,
and adding new ones is not going to help.

I think Phil's argument about the client ignoring defaults
is valid.  If the server sticks to 1 basic-mode, the contents
of the <get-config> reply indicate how to server will
treat edits to its database.  There are so many constraints
possible in YANG, that requiring a <get-config> first to
make precise editing decisions is not unreasonable.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 20:34:40 +0200

> avoids most the problems completely.

It is a bug if the semantics of 4741 operations are not clear from the
4741 capability announcements. I believe this needs fixing in 4741bis.
I do not want to do experiments in order to probe the server's
behaviour. I thought we move past SNMP where this unfortunately was
common practice.

> and adding new ones is not going to help.

Not true. If you take out the changes to operations, then the
capability announcing the server behaviour boils down to one page of
text. The bigger part is probably the description what it means, which
even in the current with-defaults may not be good enough, see Phil's
posting.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 20:35:29 +0200

> database.  So when I set "version" to "1", is disappears.

OK, this makes it clear.

/js

---

From: Kent Watsen <[email-masked]>
Date: Mon, 10 May 2010 11:58:10 -0700

> handling the defaults, it certainly is a bug not to report the strategy.

+2 - I think 4741 should be updated to 1) define the three basic-modes and 2) advise how NETCONF operations on defaults can vary for each (i.e. Phil's examples).

I'm on the fence as to if 4741 should go so far as to define a way for a server to publish its basic-mode, since 4741 says this:

   5.2. Data Modeling

   Data modeling and content issues are outside the scope of the NETCONF
   protocol.  An assumption is made that the device's data model is
   well-known to the application and that both parties are aware of
   issues such as the layout, containment, keying, lookup, replacement,
   and management of the data, as well as any other constraints imposed
   by the data model.

It seems incorrect for 4741 to define a mechanism for publishing a server's basic-mode for default-handling without also providing a means to discover what the server's defaults are...

Would it make more sense for YANG to define a field for the server to publish its basic-mode?  - instead of putting it into a capability string?

Thanks,
Kent

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 10 May 2010 12:09:45 -0700

Hi -

> From: "Phil Shafer" <[email-masked]>
> To: "Bert (IETF) Wijnen" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Monday, May 10, 2010 9:03 AM
> Subject: Re: [Netconf] WG Chair question: where are we with defaults
...
...
>   (d) the delete succeeds; the leaf is recreated with the default

I don't see how you arrived at the conclusion that the leaf would
reappear.  There are *two* pieces of information: whether or not
something exists, and its value if it does exist.  Perhaps this is
because I'm coming at this with a CMIP / SNMP / SQL mindset,
but I think of "default" as being used to supply a value when
something is created, *not* bound up in the circumstances under
which something is created.  Netconf extends "default" to apply to
user interface / bandwidth optimizations, and potentially to
a data storage optimizations.  However, in this discussion these
optimizations have become like the tail wagging the dog, and, in
the analysis of "report-all" (if that analysis is indeed correct) seem
to have really diverged from what "default" means anywhere else.

> or to have a create fail for a list member that I just created.

Totally agreed.

> in the appropriate YANG module, if my client even needs them.

This is the use-case for using knowledge of defaults to eliminate
on-the-wire redundancy.  HOWEVER, this is also where a conflict
arises.  *If* it is possible to delete a leaf with a default value
it is not possible for the client to distinguish between the case where
the leaf has been deleted and the leaf has been suppressed from the
response.  If the server instantly re-creates a deleted defaulted leaf,
then it means that the model implementation cannot represent
whatever semantic was supposed to be represented by the
absence of that leaf.

Randy

---

From: Kent Watsen <[email-masked]>
Date: Mon, 10 May 2010 12:09:59 -0700

> Would it make more sense for YANG to define a field for the server to
> publish its basic-mode?  - instead of putting it into a capability string?

...or maybe add it to the "netconf-state" subtree defined by the netconf-monitoring I-D?

Thanks,
Kent

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 10 May 2010 21:37:08 +0200

> Would it make more sense for YANG to define a field for the server to publish its basic-mode?  - instead of putting it into a capability string?

I do not see the issue. If you can explain the three different default
modes in 4741bis, then it should be possible to announce the
basic-mode in 4741bis. How defaults are discovered is out of scope,
independent of whether 4741bis includes the announcement of the
basic-mode or not. For understanding how protocol operations behave,
you only need a description of the three modes plus the announcement.

/js

---

From: Phil Shafer <[email-masked]>
Date: Mon, 10 May 2010 15:42:39 -0400

"Randy Presuhn" writes:
>reappear.

Check the mailing list archives for the proponents of this model.
This is the behaviour they wanted (for the node to exist as a
real node in the database).

>There are *two* pieces of information: whether or not
>which something is created.

This is *not* the way YANG works.  Defaults values are used when
that leaf does not exist.

>Netconf extends "default" to apply to
>a data storage optimizations.

No, NETCONF adopts the classic CLI definition of default values,
which is something operators are very familiar with.  If you don't
give an explicit value in the configuration, then the system will
proceed using the default value.

For example, if the default ttl is 30 and you configure a peer that
does not specify the ttl knob, then the system will use 30.  This
value will not show up in the configuration data, but is implicit
by the absence of the ttl knob.  If you make a peer, set the ttl
knob and the delete that knob, then the system will behave as it
you had never set the knob at all.

This is a very different model from CMIP, SNMP, and SQL, but one
that has always worked well for networking gear.

>However, in this discussion these

This is not a storage optimization.  This is a combination
of easy integration, matching expectations, and not polluting
the configuration database that the human user needs to look
at.  Reducing noise in the configuration data is one of my
highest priorities, so pumping in meaningless default values
is IMHO a very wrong direction.

>on-the-wire redundancy.

This is built into YANG.

>HOWEVER, this is also where a conflict

You've lost me here.  If the absense of the leaf has a semantic
meaning then there should not be a default value for that leaf.

Thanks,
 Phil

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 10 May 2010 13:45:22 -0700

Hi -

> To: "Randy Presuhn" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Monday, May 10, 2010 12:42 PM
...
> meaning then there should not be a default value for that leaf.

The logical conlusion of that line of reasoning is that a leaf with
a default value must not be deletable, because deleting such a leaf
would have no semantic.  In some metamodels, this makes perfect sense.
For example, in CMIP land one cannot create or delete attributes
independently of the object to which they belong, so the question
of deleting a defaulted attribute never arises.

My problem is that I don't see how this metamodel constraint is ever
made explicit in netconf-land.

Randy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 10:05:49 -0700

Juergen Schoenwaelder wrote:
> ....

This is not really true.
The only mode in which the meta-data tells you enough
is basic-mode=report-all.  Do any NETCONF vendors
even use report-all as their basic mode?  I doubt it.
The proponents of report-all (including me) want it
as a client-initiated debug mode.

In the explicit and trim modes, a client needs to check the
current value of the leaf.  What if it doesn't
contain a default?  Then create will fail for sure.
Same for delete working for sure. 

Knowing the basic-mode does not tell the client anything
definitive, except in the basic-mode nobody actually uses.
Advertising with-defaults basic-mode doesn't really
solve the create/delete ambiguity problem at all.
IMO, there is no reason to make this advertisement a
MUST implement for all 4741bis servers.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 11 May 2010 13:12:56 -0400

>as a client-initiated debug mode.

If this true, then I don't understand your objection to
Kent's request to tag leafs that aren't "real" config
data with an attribute letting the client know these
values aren't real.  If your client wants debug
mode, isn't that a real key piece of information?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 11 May 2010 13:16:02 -0400

>made explicit in netconf-land.

It was never explicit or even implicit.  The issue only arises when
you start thinking about data models, which NETCONF does not consider.
With YANG, we hit this issue repeatedly and the w-d draft was
intended to address it in a way that isn't YANG specific.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 11 May 2010 13:19:11 -0400

Juergen Schoenwaelder writes:
>>.....

Should we add a more detailed version of this discussion to the
w-d draft?  Do we need to change the words "operationally behave"
to something that makes it clear we are talking about how the
device behaves, not how the server handles NETCONF operations?

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 10:48:25 -0700

> mode, isn't that a real key piece of information?

I don't want any YANG-derivable meta-data included
in the NETCONF standard <rpc-reply>.  An old-client
will not handle it at all, and not all new clients want
this meta-data.  Any 'server MAY send' is also
a 'client MUST accept'.  That's unacceptable.

The WG already rejected this proposal twice -- once
on the mailing list, and once at an IETF meeting.
There is no consensus or permission to expand the
charter of the with-defaults draft.

Andy

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 11 May 2010 10:52:46 -0700

Hi -

> Sent: Tuesday, May 11, 2010 10:16 AM

(banging head against wall)  This is the point.  Whether you like it
or not, a configuration management protocol necessarily makes
assumptions about the metamodel.  To pretend that a protocol
makes no assumptions or imposes no constraints is to invite a
real mess, as demonstrated by the current discussion.

Randy

---

From: Phil Shafer <[email-masked]>
Date: Tue, 11 May 2010 13:35:43 -0400

>a 'client MUST accept'.  That's unacceptable.

Old clients won't pass the new with-defaults parameter
to the server, so they won't get this new attribute.

>charter of the with-defaults draft.

I don't recall this.  I'm not sure this has been even been written
up as a concrete proposal.  Can the chairs confirm please if the
WG has rejected this proposal twice?

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 11 May 2010 13:41:01 -0400

>real mess, as demonstrated by the current discussion.

(with no head banging) No, one can carry anything in NETCONF,
including CMIP, SNMP, CLI, or YANG-based data.  There's no need to
pretend that a single view of the meaning of default values will
apply to all these models.  YANG is free to define how YANG-based
models deal with default values.  If you make a "CMIP over NETCONF"
draft, you can freely define how CMIP handles default values.  These
views can be very different.

This "one size fits all" view of the world is IMHO the source of
your bruised forehead  x;^)   (where "x" is the briuse)

Thanks,
 Phil

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 11 May 2010 11:41:49 -0700

Hi -

> Sent: Tuesday, May 11, 2010 10:41 AM
> including CMIP, SNMP, CLI, or YANG-based data.

Not true.  The semantics of NETCONF don't support SNMP get-next,
for example, nor CMIP set-to-default, nor CMIP cancel-get.  (The latter
could not even be emulated by created a new RPC.)  Nor can
it handle CMIP's create-with-reference-object.  I'm not arguing that
it should, but simply that, as we're seeing now, there *are* assumptions
about the models it can handle, even if folks didn't think them through.

>  There's no need to

This would lead one to conclude that the behaviour of operations in netconf
is both implementation and data model specific.  Doesn't sound like
much help for interoperability.

Randy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 11 May 2010 21:52:15 +0200

> device behaves, not how the server handles NETCONF operations?

We definitely need this text, where it goes is still unclear.

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 11 May 2010 21:57:19 +0200

> as a client-initiated debug mode.

Did I say anything about meta-data in the text you quoted? I am kind
of lost.
 
> definitive, except in the basic-mode nobody actually uses.

For me, it makes a whole lot of a different to know whether a server
is explicit or trim.

> MUST implement for all 4741bis servers.

The advertisement tells me what behaviour my client can expect. I do
not see why not informing the client about the server's behaviour can
make things simpler. We are going in circles.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 11 May 2010 22:26:45 +0200

> device behaves, not how the server handles NETCONF operations?

I think so, although IMO it's a slippery slope. I would prefer if the
adjective "operationally" was removed entirely.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: "Bert Wijnen \(IETF\)" <[email-masked]>
Date: Tue, 11 May 2010 22:31:23 +0200

Inline

----- Original Message ----- 
From: "Phil Shafer" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, May 11, 2010 7:35 PM
Subject: Re: [Netconf] WG Chair question: where are we with defaults

> WG has rejected this proposal twice?

I do not recall a concrete proposla either. 
I suggest that the person who claims that auch a proposal was rejected
(so Andy in this case) provides a ptr to the suggested proposal and at
least some time indication as to when it was discussed. Then I may
be able to find it.

Bert

---

From: Phil Shafer <[email-masked]>
Date: Tue, 11 May 2010 16:21:47 -0400

"Randy Presuhn" writes:
>it handle CMIP's create-with-reference-object.

Um, what about:

<rpc>
  <snmp:get-next>1.2.3.4.5.6.7</snmp:get-next>
</rpc>

I'm not a CMIP-head, but next IETF maybe we should talk
face-2-face about how CMIP operations are impossible
to represent over an RPC-based protocol.

>much help for interoperability.

The NETCONF WG elected to dodge the data modeling bullet, and
left that for later.  The NETMOD WG has produced YANG, which
is IMHO a great way to model data for networing devices.  Put
the two together and I think you have an emmense improvement
on network management, and if accepted and used by the IETF,
this will be a monstrous help for interoperability.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Tue, 11 May 2010 16:25:05 -0400

>For me, it makes a whole lot of a different to know whether a server

This is the part I'm missing.  How will you use the knowledge that
a particular server is trim or explicit in your code?  The only
difference is that trim will not return leafs whose value is equal
to the default.  If your client simply understand this is optional
behavior, what further differences will it make for the client code?

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 11 May 2010 22:52:25 +0200

> behavior, what further differences will it make for the client code?

Configuration scripts that work for a "trim" server may fail for an
"explicit" server and vice versa.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 14:38:12 -0700

Bert Wijnen (IETF) wrote:
> Inline
>
> ----- Original Message ----- From: "Phil Shafer" <[email-masked]>
> To: <[email-masked]>
> Cc: <[email-masked]>
> I do not recall a concrete proposla either. I suggest that the person
> who claims that auch a proposal was rejected

I do not recall the email thread conclusion.
I do recall the discussion at the IETF meeting,
and the room decision was not to add any XML
attributes to the with-defaults reply.
The only person who objected to the room decision
on the ML was the person who proposed the change.

I do not recall any emails or WG meeting minutes that
declared that the with-defaults charter has been expanded
to include XML attributes in the rpc-reply.
Can you point out that email?

IMO, expansion of the charter for this document has not
been agreed upon, and the problem/solution proposal has
not been properly presented to the WG.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 14:45:27 -0700

> "explicit" server and vice versa.

Can you expand on this generalization?
It doesn't matter for merge/replace.
For create/delete, the client really needs to do a get-config
first.  I don't see how the client knows if the leaf
contains the default or normal value already, so it knows
whether create/delete should work or not.

How realistic is it for a client to do creates or deletes
without first doing a get-config?

BTW, one of the use-cases for report-all is to retrieve
an XML document that can be used for off-line validation.
How does the possible inclusion of XML attributes in
this XML document affect off-line validation?
Seems like this would cause a problem.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 14:58:07 -0700

> to the server, so they won't get this new attribute.

I'm still confused why clients need an attribute added
to the reply.  That's why a real draft should be submitted,
so the WG can understand this problem more.

It seems rather strange that the company that
insists 'report-all' is pointless and does not want to
implement it, also insists that the report-all reply
needs XML attributes added to it.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 11 May 2010 15:24:23 -0700

> This is the part I'm missing.  How will you use the knowledge 
> that a particular server is trim or explicit in your code?  
> The only difference is that trim will not return leafs whose 
> value is equal to the default.  If your client simply 
> understand this is optional behavior, what further 
> differences will it make for the client code?

Primarily to allow an NMS provide a better user-experience.  For instance, if user sets a value same as default, an NMS can note that the operation has no affect on report-all/trim servers.  Likewise, if value is already default, an NMS can disable "clear to default" option on report-all/trim servers and highlight that the value is explicitly set on an explicit server.

Thanks,
Kent

---

From: "Randy Presuhn" <[email-masked]>
Date: Tue, 11 May 2010 15:30:36 -0700

Hi -

> Sent: Tuesday, May 11, 2010 1:21 PM

That's not netconf.  That's a protocol you've overlayed on netconf,
and has nothing to do with the semantics of netconf.  When you've
explained how the behaviour of this RPC in the presence of access
control would match that of SNMP, you'll only have scratched the
surface.  But you could do the same thing with FTP or HTTP, and
I'd find those equally unpersuasive.

The particular case of cancel-get cannot even be emulated with
RPC, because its behaviour is at odds with netconf's request/response
flow.  Cancel-get would be used while a long series of linked replies
is coming down the pipe.  In CMIP, a get may result in many responses.
This operation provides a way to say "no more, I didn't mean to
ask for a recursive listing of the entire tree."

> to represent over an RPC-based protocol.

CMIP specifics are a moot point.  The issue is that protocol
design DOES make assumptions about models.  The lack of
attention given to this will come back to haunt netconf.

> this will be a monstrous help for interoperability.

I am amazed by your faith, and hope you are correct.

Randy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 11 May 2010 15:29:43 -0700

> without first doing a get-config?

I tend to agree.  I'm personally less worried about failing protocol operations as an NMS's basic "update device" algorithm will likely diff against the device to calc what the <edit-config> should look like, and thus should never create a node that already exists or delete a node that doesn't

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Tue, 11 May 2010 15:57:33 -0700

> It seems rather strange that the company that insists 
> 'report-all' is pointless and does not want to implement it, 
> also insists that the report-all reply needs XML attributes 
> added to it.

That is a mischaracterization - our request has consistently been to *either* remove the "MUST implement report-all" or make it safe by tagging defaults.  Also, we argued that report-all is useful for thin clients (i.e. not pointless).

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 16:00:47 -0700

Kent Watsen wrote:
>
>
>> How realistic is it for a client to do creates or deletes without
>> first doing a get-config?
>
>
> I tend to agree.  I'm personally less worried about failing protocol
> operations as an NMS's basic "update device" algorithm will likely
> diff against the device to calc what the <edit-config> should look
> like, and thus should never create a node that already exists or
> delete a node that doesn't
>

What may be good for your thin client is bad for my thin client.
I have modular applications and the program that knows
about get-config requests/replies may not be the same program
as the the one that would have to strip XML attributes out
of the reply.

The former does not know how to strip attributes out of the
reply, and the latter does not correlate requests with replies,
so it doesn't know it should strip out particular XML attributes.

So unless my thin client can request report-all without any
XML attributes added, it breaks.  These attributes cause
false diffs, and cause unknown-attribute errors during
off-line validation.

> Thanks, Kent

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 11 May 2010 16:37:37 -0700

> I have modular applications and the program that knows about 
> get-config requests/replies may not be the same program as 
> the the one that would have to strip XML attributes out of the reply.

Huh? I thought this fork in the thread was debating the reason why advertising the basic-mode might be a MUST...  Also, note that I was agreeing with you!  ;)

> reply, and the latter does not correlate requests with 
> replies, so it doesn't know it should strip out particular 
> XML attributes.

I don't understand how why client coded to request report-all would be unable to strip out attributes (s/wd:default=\"true\"//g)

>From what I wrote on 5/7:

<quote>
If we allow tagging defaults, I think we should let trim/explicit servers advertise if the tagging is:
  - not supported by the server
  - supported, but must be asked for
  - default, may be disabled
  - always on, no way to disable
</quote>

Your arguing that, if we support tagging defaults, you'd want to remove the last option - right?

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 17:08:56 -0700

Kent Watsen wrote:
>>> I tend to agree.  I'm personally less worried about failing 
>>>       
>> protocol 
>>     

The code is separated, so it can't strip out XML attributes.
I don't know why your code can't diff 2 XML trees.
The point is that my thin client has just as much right to
NOT get XML attributes as your thin client has a right
to get them.  Why should my thin client that already
works with the current WG solution break for some
new addition that my thin client does not want?

I want the server to respond exactly as I request.
If the server MAY add stuff I don't want, then
the operation is no use to me.

> Your arguing that, if we support tagging defaults, you'd want to remove the last option - right?

I do not want the server to decide what to send.
I want the client to always decide.  I also want
the server to be able to not implement this XML attribute.

  basic-mode = [trim | explicit | report-all]

  also-supported = 0 or more of
         [trim | explicit | report-all | report-all-tagged]

   <with-defaults>[full-list-of-enums]</with-defaults>

This fixes the problem.
If the server wants to support this XML attribute,
it will list it in its also-supported parameter. 
If the client wants report-all-tagged, it will ask for it.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 17:21:41 -0700

> If the client wants report-all-tagged, it will ask for it.

Actually, the XML attribute tagging does not work with YANG.
YANG does not support XML attributes, and a hack like the one
in ietf-netconf.yang to support <filter> applies to a specific
XML element.  I don't know how to write a YANG extension
that would allow a YANG validation program to properly deal
with NETCONF PDUs that contain the XML attribute in
random elements.

All the more reason for you to write a draft explaining
the solution in detail.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Tue, 11 May 2010 17:46:46 -0700

> in ietf-netconf.yang to support <filter> applies to a 
> specific XML element.  I don't know how to write a YANG 
> extension that would allow a YANG validation program to 
> properly deal with NETCONF PDUs that contain the XML 
> attribute in random elements.
> 
> All the more reason for you to write a draft explaining the 
> solution in detail.

This is why on 4/5 I asked:

<quote>
One thing I'm not clear on is the extent that a YANG module definition must itself use YANG - specifically for protocol modifications.  For instance, the with-defaults I-D has YANG statements like:

    augment /nc:get-config/nc:input {
        uses with-defaults-parameters;
    }

Are these *required* because YANG compilers MUST be 100% data-driven?  - or could this I-D have alternatively left it developer to read the RFC cited by the module's "reference" statement?  
</quote>

Are you saying your "YANG validation program" will be 100% data-driven?

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Tue, 11 May 2010 19:03:54 -0700

> Are these *required* because YANG compilers MUST be 100% data-driven?  - or could this I-D have alternatively left it developer to read the RFC cited by the module's "reference" statement?  

We wanted some hack to justify using XML attributes in
a specific XML element, so we have this extension in
the ietf-netconf.yang module:

     extension get-filter-element-attributes {
       description
         "If this extension is present within the
          an 'anyxml' statement named 'filter', which must be
          conceptually defined within the RPC input section
          for the 'get' and 'get-config' RPC operations,
          then the following unqualified XML attribute is
          supported within the 'filter' element, within
          a 'get' or 'get-config' protocol operation:

            type : optional attribute with allowed
                   value strings 'subtree' and 'xpath'.
                   If missing, the default value is 'subtree'.

          If the xpath feature is supported, then the
          following unqualified XML attribute is
          also supported:

            select: optional attribute containing a
                    string representing an XPath expression.
                    The 'type' attribute must be equal to 'xpath'
                    if this attribute is present.";
     }

> Are you saying your "YANG validation program" will be 100% data-driven?

yes -- for the NETCONF content layer,
which happens to be the layer that YANG standardizes
now.  YANG does not support XML attributes.

An XML validation program looks at all the XML.
If the schema doesn't allow any XML attributes
to be present, then why is it OK for a server to
just add them to the reply?  This violates the
Postel Principle.

NETCONF before YANG: no standard content layer.
No standard content.  Vendors do whatever they want,

NETCONF after YANG; standard content layer.
What does that mean?

How are these XML attributes part of the standard
content layer, but the NETCONF standard data modeling
language has no support for XML attributes?

Perhaps we should scrap YANG and use RelaxNG,
if we want to use XML attributes in the standard NETCONF
content layer.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 12 May 2010 09:14:32 +0200

> behavior, what further differences will it make for the client code?

The client will know that setting this leaf to a value (which might be
the default value) and not getting it back is not a bug of the server
but considered a feature by whoever implemented the server. In the
simples case, my client could generate a warning "this netconf server
is trimming defaults, be careful". Believe me or not, this kind of
information is operationally useful.

Have you seen an operator knocking his head against the wall for hours
because when he sets a specific value the value disappears in the
config? Don't answer me he should have read the YANG / NETCONF RFCs
and he should understand the details of the data model.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 12 May 2010 13:15:30 +0200

> first.  I don't see how the client knows if the leaf

This may not help. Consider a leaf X with a default defined in the data
model. If the client get a configuration where X is not present, then

1. creating leaf X with the default value and then deleting it will
succeed for an "explicit" server but fail for a "trim" server;

2. creating leaf X with the default value and then creating it again
with any value succeeds for a "trim" server and fails for an "explicit"
server.
   
> whether create/delete should work or not.

Yes, but who wants to retrieve the configuration after every operation?

> an XML document that can be used for off-line validation.

The DSRL schema created by the DSDL mapping from YANG module(s) is able
to generate the "report-all" version from any valid configuration.

> Seems like this would cause a problem.

Why? I think it should be doable, e.g., if the attributes reside in a
dedicated namespace.

Lada

-- 
Ladislav Lhotka, CESNET
PGP Key ID: E74E8C0C

---

From: Phil Shafer <[email-masked]>
Date: Wed, 12 May 2010 08:08:03 -0400

>needs XML attributes added to it.

IIRC one of the cornerstone of the IETF is that we appear as
individuals, not representatives of our current employers.  Clearly
Kent and I disagree on the neccessity of w-d, and I think that's
completely healthy.

My main interest is that the w-d draft be suitably written so that
if I am pushed to implement it, no harm will come to my users.
Kent's proposal is meant to reduce the possibility of harm, which
is why I see it as a Good Thing.  Again, I think this is completely
healthy.

In contrast, your claims that this issue has been rejected multiple
times seem like fabrication, which is clearly not healthy.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 12 May 2010 05:39:08 -0700

> is why I see it as a Good Thing.  Again, I think this is completely

As individuals, you cancel each other out.
You cannot argue one minute that with-defaults
is harmful, and also argue it has to done 'your way' or the highway.

IMO, there is no need to encode the YANG file on-the-wire.
We didn't need that in SNMP and we don't need it in NETCONF.
YANG does not support XML attributes, and simply ignoring
all XML attributes does not seem like a standardized
content layer to me.

> times seem like fabrication, which is clearly not healthy.

I recall the conclusion from the IETF meeting:
No consensus to add the XML attribute.  There
was no consensus established on the mailing list
either.  If you have proof of this, point it out.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Wed, 12 May 2010 08:22:49 -0400

>information is operationally useful.

This assumes that your client will set the leaf and then check the
leaf, which is not the behavior I'm used to seeing in clients.  Most
clients are "fire and forget", so if the operation succeeds, they
move on and assume the right thing has happened.

I could see some utility in comparisons on a higher level, when I
asked to load a zillion import policies and what to fetch them all
and make sure the work was really done, but teaching teaching the
policy generator about defaults would be easier than teaching the
compare function to handle trimmed default values.

>and he should understand the details of the data model.

Trust me;  I had the same mental anguish when I first used IOS
(which is why I did my CLI differently ;^)

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 12 May 2010 08:27:47 -0400

>I am amazed by your faith, and hope you are correct.

Considering that these (or similar) tools and approaches have been
used for ~10 years in Juniper products, I don't see this as faith.
I see it as knowledge sharing.  I know this stuff works.  I've used
it for years.  It's quite nice.  Honest.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 12 May 2010 08:37:19 -0400

>As individuals, you cancel each other out.

I don't see it that way.  I see us as individuals contributing to
help the WG develop a better way.

>is harmful, and also argue it has to done 'your way' or the highway.

If I saw the world in this silly way, I don' think I'd
have stuck around this long ;^)

>We didn't need that in SNMP and we don't need it in NETCONF.

I didn't think "YANG file on-the-wire" was the current discussion,
but I think the monitoring draft does allow this.

If you are seeing the defaults discussion as "YANG file on-the-wire",
my view has always been that intermixing defaults with "real"
data is unneccessary, since these defaults can be found
in the YANG modules.

I guess I'm not following your argument here.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 12 May 2010 08:38:52 -0400

>Can you point out that email?

Do documents _have_ charters?  I had no idea.....

Thanks,
 Phil