Subject: [netconf] YANG attributes in a datastore

From: Michal Vaško <[email-masked]>
Date: Tue, 10 Dec 2019 14:12:49 +0100

Hello,
we are thinking about adding support for arbitrary attributes [1] in our YANG/NETCONF datastore. To my knowledge, only the representation in XML or JSON is defined and standardized but there are no guidelines for editing (CRUD) these attributes.

So, my question is a rather general one, regarding implementations that support editing these attributes, would you mind sharing some basic ideas or principles used? Or is this outside any consensus and each implementation is free to handle this completely in its own fashion? Thanks for any input.

Regards,
Michal

[1] https://tools.ietf.org/html/rfc7952

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Dec 2019 14:31:06 +0100

On Tue, 2019-12-10 at 14:12 +0100, Michal Vaško wrote:
> Hello,
> we are thinking about adding support for arbitrary attributes [1] in our
> YANG/NETCONF datastore. To my knowledge, only the representation in XML or
> JSON is defined and standardized but there are no guidelines for editing
> (CRUD) these attributes.
> 
> So, my question is a rather general one, regarding implementations that
> support editing these attributes, would you mind sharing some basic ideas or
> principles used? Or is this outside any consensus and each implementation is
> free to handle this completely in its own fashion? Thanks for any input.

I guess this should eventually be standardized in each protocol, so it makes
sense to figure out the current practices.

Lada

> _______________________________________________
> netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 10 Dec 2019 15:03:33 +0100 (CET)

Hi,

Michal Vaško <[email-masked]> wrote:
> Hello,
> we are thinking about adding support for arbitrary attributes [1] in
> our YANG/NETCONF datastore. To my knowledge, only the representation
> in XML or JSON is defined and standardized but there are no guidelines
> for editing (CRUD) these attributes.
> 
> So, my question is a rather general one, regarding implementations
> that support editing these attributes, would you mind sharing some
> basic ideas or principles used? Or is this outside any consensus and
> each implementation is free to handle this completely in its own
> fashion? Thanks for any input.

I think this depends on the attribute defintion.  Some attributes are
read-only from the northbound's protocol point of view.  Internally in
the server you may need some API to set/change them of course.  Some
other attributes are read-write.

In our implementation we support a few read-write attributes (and some
read-only).  Here's a simple example (in an edit-config):

  <foo annotation="hello"/>

This would set/change the "annotation" attribute's value to
"hello", regardless of its previous value.

To delete an annotation, you would do:

  <foo annotation=""/>

Each of our attributes has a special value that means "delete" (in
most cases it is the empty string.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Dec 2019 15:24:34 +0100

> most cases it is the empty string.

Hmm, why don't you do simply

<foo/>

to delete the attribute?

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 10 Dec 2019 15:55:08 +0100 (CET)

> Hmm, why don't you do simply

B/c that would mean that all attributes would have to be repeated just
to set a value; I don't want:

  <foo>42</foo>

to erase all attributes on foo.

/martin

---

From: Balázs Lengyel <[email-masked]>
Date: Tue, 10 Dec 2019 14:58:59 +0000

Hello,
If we try to standardize editing metadata, we will face a number of issues
that need to be clarified:

- Will you model whether the attribute (metadata) is writable?  Config is
not an allowed substatement in https://tools.ietf.org/html/rfc7952#section-3
- Can a metadata have other additional properties? (mandatory, cardinality,
uniqueness of some kind?)
- If it is modified, will it be subject to validation, error-messages? Will
a bad metadata result in a failed error-config?

Regards Balazs

-----Original Message-----
From: netconf <[email-masked]> On Behalf Of Martin Bjorklund
Sent: 2019. december 10., kedd 15:04
To: [email-masked]
Cc: [email-masked]
Subject: Re: [netconf] YANG attributes in a datastore

Hi,

I think this depends on the attribute defintion.  Some attributes are
read-only from the northbound's protocol point of view.  Internally in the
server you may need some API to set/change them of course.  Some other
attributes are read-write.

This would set/change the "annotation" attribute's value to "hello",
regardless of its previous value.

  <foo annotation=""/>

Each of our attributes has a special value that means "delete" (in most
cases it is the empty string.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Dec 2019 16:31:41 +0100

> not an allowed substatement in https://tools.ietf.org/html/rfc7952#section-3

In my view annotations should never be configuration because they should not
influence the device state. The read-only/read-write propertly might be useful.

> uniqueness of some kind?)

By design, annotations do not have a specific place in the schema, so they
cannot be mandatory.

Cardinality: if you mean annotations with a list of values, this was discussed
but rejected due to difficulties with XML encoding.

> a bad metadata result in a failed error-config?

I would say yes because otherwise it makes no sense to define them and their
properties.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Dec 2019 07:42:11 -0800

Hi,

YANG already has the "annotation" statement to define metadata.
YANG has data-def-stmt to define data.  It would be very unwise
to use metadata inside a leaf as if it were a leaf inside a container.

Andy

On Tue, Dec 10, 2019 at 7:31 AM Ladislav Lhotka <[email-masked]> wrote:

> > not an allowed substatement in
> https://tools.ietf.org/html/rfc7952#section-3
>
> the

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Dec 2019 16:50:08 +0100

> to use metadata inside a leaf as if it were a leaf inside a container.

A typical example is a "comment" annotation - you want to permit it basically
everywhere but don't want to clutter the schema with "comment" leaves.

Lada

---

From: Kent Watsen <[email-masked]>
Date: Tue, 10 Dec 2019 16:19:41 +0000

> everywhere but don't want to clutter the schema with "comment" leaves.

These are what I refer to as "cross-cutting" metadata attributes.  Another example
is the "enabled" attribute defined in draft-kwatsen-conditional-enablement.

Kent // contributor

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Dec 2019 08:25:04 -0800

> everywhere but don't want to clutter the schema with "comment" leaves.
I understand the difference between annotation-stmt and leaf-stmt.
If the semantics can apply to all data nodes then annotation is appropriate.
We should not change the protocols and YANG to reinvent a leaf-stmt.

When we created YANG 1.0, we explicitly rejected the then-current style of
XML

   <address street="123 Elm" city="Anytown" zipcode="100036" />

We rejected the use of attributes as leafs. We should not change that
decision now.

Lada
Andy

> > > > - If it is modified, will it be subject to validation,
> error-messages?
> > > Will

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 10 Dec 2019 17:33:35 +0100

> We should not change the protocols and YANG to reinvent a leaf-stmt.

Then I don't understand the point of your previous message. I think we've been
talking about annotations as per RFC 7952 from the beginning of this thread
(Michal just called it somewhat confusingly "attribute").

Lada

> 
> When we created YANG 1.0, we explicitly rejected the then-current style of XML
> 
> We rejected the use of attributes as leafs. We should not change that decision
> now.
> 
> 
> > Lada 
> > 
> 
> Andy
>  
> > > 
> > > 
> > > Andy
> > > 
> > >

---

From: Andy Bierman <[email-masked]>
Date: Tue, 10 Dec 2019 08:41:24 -0800

> (Michal just called it somewhat confusingly "attribute").
OK, but there is no way to prevent "annotation zipcode", etc.
The use of foo="value" to set and foo="" to delete seems fine, but not sure
any new
standards are needed for that.

Lada

Andy

> decision
> > > > > > If we try to standardize editing metadata, we will face a number
> of
> > > issues
> > > > > > - Will you model whether the attribute (metadata) is writable?
> Config
> > > is
> > > > > In my view annotations should never be configuration because they
> should
> > > not
> > > > > influence the device state. The read-only/read-write propertly
> might be
> > > > > useful.
> > > > >
> > > > > By design, annotations do not have a specific place in the schema,
> so
> > > they
> > > > > cannot be mandatory.
> > > > >
> > > > > Cardinality: if you mean annotations with a list of values, this
> was
> > > > > discussed
> > > > > I would say yes because otherwise it makes no sense to define them
> and
> > > their
> > > > > properties.
> > > > >
> > > > > Lada
> > > > >
> > > > > >
> > > > > > Regards Balazs
> > > > > >
> > > > > > > we are thinking about adding support for arbitrary attributes
> [1]
> > > in
> > > > > > > basic ideas or principles used? Or is this outside any
> consensus
> > > and

---

From: Kent Watsen <[email-masked]>
Date: Tue, 10 Dec 2019 17:06:23 +0000

> The use of foo="value" to set and foo="" to delete seems fine, but not sure any new

Metadata is not data.  Whilst syntacally possible, and some may attempt to [mis-]use
it as such, we must hold the line that any IETF-defined metadata/annotations do not
encode data.

Kent // contributor

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 11 Dec 2019 08:59:17 +0100

> it as such, we must hold the line that any IETF-defined metadata/annotations

While I agree with this, the original question remains: how to edit such
metadata in an interoperable way. Deleting the annotation with foo="" may not
always work, and indicating deletion with a special magic value is also brittle
because such a value is not specified in the annotation's definition.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 11 Dec 2019 09:30:12 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> While I agree with this, the original question remains: how to edit
> such metadata in an interoperable way. Deleting the annotation with
> foo="" may not always work, and indicating deletion with a special
> magic value is also brittle because such a value is not specified in
> the annotation's definition.

So far, noone is proposing to standardize anything.  The question was
if any implementation is doing this today, and if so, how.

/martin

---

From: Kent Watsen <[email-masked]>
Date: Wed, 11 Dec 2019 19:37:45 +0000

> So far, noone is proposing to standardize anything.  

I must've replied to the wrong message.  I was reacting to this message:
https://mailarchive.ietf.org/arch/msg/netconf/bHLGqeK-Yo3cYDAXic83m7oMAGY <https://mailarchive.ietf.org/arch/msg/netconf/bHLGqeK-Yo3cYDAXic83m7oMAGY>.

> The question was if any implementation is doing this today, and if so, how.

Juniper supports metadata today.  Specifically, the "inactive" annotation.  Sadly JUNOS defines an asymmetric "active" annotation to unset it, which is what thing that we were trying to fix with the "enabled" annotation in the conditional-enablement draft. https://www.juniper.net/documentation/en_US/junos/topics/reference/tag-summary/junos-xml-protocol-inactive-attribute.html <https://www.juniper.net/documentation/en_US/junos/topics/reference/tag-summary/junos-xml-protocol-inactive-attribute.html>.

Note that the ''report-all-tagged' retrieval mode in RFC 6243 was defined so that JUNOS wouldn't promote a default value to configuration.  This illustrates round-tripping, in that "configured" annotations are indeed expected to be returned to the server in write-operations.

Kent

---

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Dec 2019 12:35:18 -0800

On Tue, Dec 10, 2019 at 11:59 PM Ladislav Lhotka <[email-masked]> wrote:

> > Metadata is not data.  Whilst syntacally possible, and some may attempt
> to
> > [mis-]use
> > it as such, we must hold the line that any IETF-defined
> metadata/annotations
> > do not
> > encode data.
>
I think empty string is the least worst solution for deletion.
It does not work for type 'empty' os an administrative string that
allows empty-string in the value-set.

RFC 7952 is under-specified wrt. data type encoding

1) it is not even clear that the type-stmt is a mandatory sub-statement of
the annotation-stmt.
There is no example showing this usage.

2) The data type encodings are not correct for the empty type

      The attribute value SHALL be encoded in the same way as the value
      of a YANG leaf instance having the same type; see Section 9 of
      [RFC7950] <https://tools.ietf.org/html/rfc7950#section-9>.

This does not work for 'empty' type which is encoded as an empty element in
XML and a null array in JSON.
This encoding cannot be used in an XML or JSON attribute.

I assume the solution is constrained to only use valid XML, so inventing
some hack to identity the deletion operation
will not work (unless it is valid XML)

Lada
Andy

---

From: Andy Bierman <[email-masked]>
Date: Wed, 11 Dec 2019 12:48:55 -0800

> There is no example showing this usage.
I found the last-modified example so disregard comment (1)

> This does not work for 'empty' type which is encoded as an empty element
> in XML and a null array in JSON.