Subject: [yang-doctors] List key model definition order and results w/ pyang/yanglint

From: Ebben Aries <[email-masked]>
Date: Wed, 24 Jan 2018 20:16:50 -0700

Possibly bringing up a subject that may have been discussed before but
could not find anything on it so sending to the group.

RFC 6020 and 7950 both have the same wording on the subject around list
keys per

https://tools.ietf.org/html/rfc6020#section-7.8.2
https://tools.ietf.org/html/rfc7950#section-7.8.2

>From a model definition standpoint, there is nothing mandatory
surrounding that list keys be the first leaf definitions underneath.

>From a content encoding standpoint, the following XML Mapping Rules in
https://tools.ietf.org/html/rfc6020#section-7.8.5 do dictate an order
that the keys be defined before other subelements.  JSON obviously does
not utilize the same encoding technique.

   The list's key nodes are encoded as subelements to the list's
   identifier element, in the same order as they are defined within the
   "key" statement.

   The rest of the list's child nodes are encoded as subelements to the
   list element, after the keys.  If the list defines RPC input or
   output parameters, the subelements are encoded in the same order as
   they are defined within the "list" statement.  Otherwise, the
   subelements are encoded in any order.

Given the following simple example:

module test {
    namespace "http://test";
    prefix test;

    container foo {
        leaf bar {
            type string;
        }
        list car {
            key "far mar";

            leaf par {
                type string;
            }
            leaf far {
                type string;
            }
            leaf mar {
                type string;
            }
            leaf nar {
                type string;
            }
        }
    }
}

leaf 'par' here is defined before the 'far' and 'mar' keys, however if
we render out a sample-xml-skeleton with pyang, we do not take into
account the XML encoding rules defined in Section 7.8.5 and merely keep
the order as defined in the model.

$ pyang test.yang --strict -f sample-xml-skeleton
<?xml version='1.0' encoding='UTF-8'?>
<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <foo xmlns="http://test">
    <bar />
    <car>
      <par />
      <far />
      <mar />
      <nar />
    </car>
  </foo>
</data>

With yanglint, given the following XML document

$ cat test.xml
<foo xmlns="http://test">
    <bar>BAR</bar>
    <car>
        <par>PAR</par>
        <far>FAR</far>
        <mar>MAR</mar>
        <nar>NAR</nar>
    </car>
</foo>

The rendered XML output will actually reorder the list keys to be first

$ yanglint -f xml test.yang test.xml
<foo xmlns="http://test">
  <bar>BAR</bar>
  <car>
    <far>FAR</far>
    <mar>MAR</mar>
    <par>PAR</par>
    <nar>NAR</nar>
  </car>
</foo>

Personally, from a BCP standpoint, defining list keys first eases
readability and want to solicit others thoughts on this (rather than
nested down in the list or via a grouping reference further down).
Generation to yin does not rearrange in either pyang or yanglint which
means any processing of yin would need to undergo some reordering
possibly depending on how it is absorbed.

For the JSON encoding case, there may be no strict ordering but
following a BCP where folks *should* send list keys before other
elements can ease implementation efforts in many cases.  It is also
likely that payloads will generally be constructed in the order in which
they are defined in the model.

So either I see making the definition of such either mandatory or at a
bare minimum pyang sample-xml-skeleton output to take into account
ordering list key leafs prior to other subelements.

Thx

/ebben

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 25 Jan 2018 08:30:58 +0100 (CET)

Hi,

> nested down in the list or via a grouping reference further down).

I agree.

> possibly depending on how it is absorbed.

I don't understand what you mean.  Yin follows the same rules as basic
YANG syntax; it is the "key" statement that defines the XML element
order.

> elements can ease implementation efforts in many cases.

I disagree.  Implementations MUST handle object members in any order
(std JSON), so they can never use a simpler implementation that is
built on the presumption that keys always are sent first.

(However, a clever implemention could check if the keys are sent
first, then it can parse more efficiently; it wouldn't have to buffer
data.  But this is the opposite of a simple implemention.)

> It is also

Yes.

> ordering list key leafs prior to other subelements.

I think the bug in pyang should be fixed!

/martin

---

From: Ebben Aries <[email-masked]>
Date: Mon, 29 Jan 2018 18:12:05 -0700

inline...

On Jan 25 08:30 AM, Martin Bjorklund wrote:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc6020-23section-2D7.8.2&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=s0ueqUyec_0t88MK81g6lUGSHXLivei_aFPMFmdUATQ&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc7950-23section-2D7.8.2&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=tnRmYIIjLA3KSnysiPYAaVjjLJ7GLK-No5AFu9bJ1hg&e=
> > 
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc6020-23section-2D7.8.5&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=El_GT294heczTOGSYxV-s0MdzjwhB-vEBEVCQNYEdzs&e= do dictate an order
> >     namespace "https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=";
> >     prefix test;
> > 
> >     container foo {
> >         leaf bar {
> >             type string;
> >         }
> >         list car {
> >             key "far mar";
> > 
> >             leaf par {
> >                 type string;
> >             }
> >             leaf far {
> >                 type string;
> >             }
> >             leaf mar {
> >                 type string;
> >             }
> >             leaf nar {
> >                 type string;
> >             }
> >         }
> >     }
> > }
> > 
> >   <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> >     <bar />
> >     <car>
> >       <par />
> >       <far />
> >       <mar />
> >       <nar />
> >     </car>
> >   </foo>
> > </data>
> > 
> > <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> >     <bar>BAR</bar>
> >     <car>
> >         <par>PAR</par>
> >         <far>FAR</far>
> >         <mar>MAR</mar>
> >         <nar>NAR</nar>
> >     </car>
> > </foo>
> > 
> > <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> >   <bar>BAR</bar>
> >   <car>
> >     <far>FAR</far>
> >     <mar>MAR</mar>
> >     <par>PAR</par>
> >     <nar>NAR</nar>
> >   </car>
> > </foo>
> > 

Should we add text to rfc6087bis to suggest such?  Any reason not to?

> YANG syntax; it is the "key" statement that defines the XML element

Just merely pointing out that the same handling that would be done for
XML encoding rules would need to be done for potential processing of YIN
if the consumer of said data needs to process ordered elements
in whatever application.

Understood this is lossless and a direct mapping

> built on the presumption that keys always are sent first.

Agreed

> data.  But this is the opposite of a simple implemention.)

This was precisely my point but agree, the assumption can never be made
due to this

A BCP could very well be put into code for --ietf, -f yang (and other
flags) that *could* enforce this ordering within the model itself.  If
we do add text to rfc6087bis, this should come along w/ imo.

> I think the bug in pyang should be fixed!

Have the fix in a local branch - pull request now at:

https://github.com/mbj4668/pyang/pull/359

/ebben

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 30 Jan 2018 16:09:00 +0100 (CET)

Hi,

Thank you for contributing with a pull request, I will have a look at
it!

/martin