Subject: [Netconf] 4741bis-04 has non-deterministic schema

From: Kent Watsen <[email-masked]>
Date: Thu, 2 Dec 2010 20:18:20 -0800

I loaded the schema from 4741bis-04 into XMLSpy (2006) and got the following error:

'<xs:element ref="rpc-error">' makes the content model non-deterministic against '<xs:any namespace-"##any">'.

Removing the <rpc-error> element as follows:

     <xs:group name="rpcResponse">
       <xs:sequence>
-        <xs:element ref="rpc-error"
-                    minOccurs="0" maxOccurs="unbounded"/>
         <xs:any namespace="##any" processContents="lax"
                 minOccurs="0" maxOccurs="unbounded"/>
       </xs:sequence>
     </xs:group>

Then gives the error:  '<xs:element name="ok">' makes the content model non-deterministic against '<xs:any namespace-"##any">'.

Removing the <ok> element as follows:

     <xs:complexType name="rpcReplyType">
       <xs:choice>
-         <xs:element name="ok"/>
         <xs:group ref="rpcResponse"/>
       </xs:choice>
       <xs:attribute name="message-id" type="messageIdType"
         use="optional"/>
       <!--
         Any attributes supplied with <rpc> element must be returned
         on <rpc-reply>.
       -->
       <xs:anyAttribute processContents="lax"/>
     </xs:complexType>

Enables the XSD to be valid.  While this schema will still validate everything from before, it's not ideal either as it doesn't explicitly show the <ok> or <rpc-error> elements anymore, but I don't see how else to make it valid...

BTW, I believe there is an error in the netconf-monitoring RFC, shouldn't this example be updated as follows?

       <rpc-reply message-id="101"
           xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
-        <data>
           <netconf-state
             xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">
             <schemas>
               <!-- SNIP -->
             </schemas>
           </netconf-state>
-        </data>
       </rpc-reply>

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Thu, 2 Dec 2010 20:35:57 -0800

Hi,

These XSD changes do not reflect the correct syntax.
Maybe another variant is needed because you can't just remove <ok> from the schema.

Also, the monitoring example is correct.
The response is for the following <get>.
The <get> operation returns everything within the element <data>.

<rpc message-id="101"
     xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
  <get>
    <filter type="subtree">
      <netconf-state xmlns=
      "urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">
        <schemas/>
      </netconf-state>
    </filter>
  </get>
</rpc>

Andy

From: [email-masked] [mailto:[email-masked]] On Behalf Of Kent Watsen
Sent: Thursday, December 02, 2010 8:18 PM
To: [email-masked]
Subject: [Netconf] 4741bis-04 has non-deterministic schema

Thanks,
Kent

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 03 Dec 2010 12:31:11 +0100 (CET)

Hi,

Kent Watsen <[email-masked]> wrote:
> '<xs:element ref="rpc-error">' makes the content model
> non-deterministic against '<xs:any namespace-"##any">'.

Ugh.  I've validated the XSD with libxml, but it doesn't complain :(
I agree this is an error though.

One option is to change the ##any to ##other:

  <xs:group name="rpcResponse">
    <xs:sequence>
      <xs:element ref="rpc-error"
                  minOccurs="0" maxOccurs="unbounded"/>
      <xs:any namespace="##other" processContents="lax"
              minOccurs="0" maxOccurs="unbounded"/>
    </xs:sequence>
  </xs:group>

But then we have the problem that the "data" element is not listed
here, since it is defined in the Operations Layer as rpc output to get
and get-config.  This becomes a problem since the XSD and YANG model
both provide elements to the same namespace.

We could handle this by putting data back:

  <xs:group name="rpcResponse">
    <xs:sequence>
      <xs:element ref="rpc-error"
                  minOccurs="0" maxOccurs="unbounded"/>
      <xs:element name="data" type="dataInlineType" minOccurs="0"/>
      <xs:any namespace="##other" processContents="lax"
              minOccurs="0" maxOccurs="unbounded"/>
    </xs:sequence>
  </xs:group>

  <xs:complexType name="dataInlineType">
    <xs:complexContent>
      <xs:extension base="xs:anyType"/>
    </xs:complexContent>
  </xs:complexType>

But then we're mixing layers... 

An alternative would be to use the substitution technique that we
already use for the rpc operation:

  <xs:group name="rpcResponse">
    <xs:sequence>
      <xs:element ref="rpc-error"
                  minOccurs="0" maxOccurs="unbounded"/>
      <xs:element ref="rpcResponse" minOccurs="0" maxOccurs="unbounded"/>
    </xs:sequence>
  </xs:group>

  <xs:complexType name="rpcResponseType"/>
  <xs:element name="rpcResponse"
              type="rpcResponseType" abstract="true"/>

And then if you want to define your own rpc-reply element, e.g. "data"
you would do:

  <xs:element name="data" substitutionGroup="nc:rpcResponse">
    <xs:complexType>
      <xs:complexContent>
        <xs:extension base="nc:rpcResponseType">
          <xs:sequence>
            <xs:any minOccurs="0" maxOccurs="unbounded"
                    namespace="##other" processContents="lax"/>
          </xs:sequence>
        </xs:extension>
      </xs:complexContent>
    </xs:complexType>
  </xs:element>

/martin

---

From: Kent Watsen <[email-masked]>
Date: Fri, 3 Dec 2010 09:27:24 -0800

> The <get> operation returns everything within the element <data>.

Ah, I see now the YANG schema for <get> and <get-config> having:

     output {
       container data {
         presence
           "An empty data container indicates that the
            request did not produce any results.";
         description
           "Copy of the source datastore subset which matched
            the filter criteria (if any).";
       }
      }

Martin, this means that we do not want to define a <data> element in <rpcResponse>

Thanks,
Kent

---

From: Andy Bierman <[email-masked]>
Date: Fri, 3 Dec 2010 09:39:07 -0800

Hi,

I think the YANG for <get> and <get-config> output needs to change to anyxml,
otherwise a YANG validator will complain that container 'data' has no child nodes
defined.

I agree that the <data> element should be left out of the XSD.
It is not hardwired in.  The WG decided the <data> node was part
of the <get> or <get-config> definition.  No other 4741 operation returns data.
The XSD should align with the YANG expectations (output/foo --> /rpc-reply/foo).

Andy

From: Kent Watsen [mailto:[email-masked]]
Sent: Friday, December 03, 2010 9:27 AM
To: Andy Bierman; [email-masked]
Subject: RE: 4741bis-04 has non-deterministic schema

     output {
       container data {
         presence

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Fri, 3 Dec 2010 14:57:48 -0800

>              type="rpcResponseType" abstract="true"/>

XMLSpy 2011 says that the 4741bis-4 schema is valid after this update

>                    namespace="##other" processContents="lax"/>

Right, saving the 4741bis-4 schema into file called "4741bis-4.xsd", I then created another file called "my-rpc.xsd" as follows:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema
   xmlns:xs="http://www.w3.org/2001/XMLSchema"
   xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"
   xmlns:dmi="http://xml.juniper.net/dmi"
   xmlns="http://xml.juniper.net/dmi"
   targetNamespace="http://xml.juniper.net/dmi">

  <xs:import namespace="urn:ietf:params:xml:ns:netconf:base:1.0" schemaLocation="4741bis-4.xsd"/>

  <xs:element name="my-rpc" substitutionGroup="nc:rpcOperation">
    <xs:complexType>
      <xs:complexContent>
        <xs:extension base="nc:rpcOperationType">
          <xs:sequence>
            <xs:element name="name" type="xs:token"/>
          </xs:sequence>
        </xs:extension>
      </xs:complexContent>
    </xs:complexType>
  </xs:element>
  
  <xs:element name="my-rpc-reply" substitutionGroup="nc:rpcResponse">
    <xs:complexType>
      <xs:complexContent>
        <xs:extension base="nc:rpcResponseType">
          <xs:sequence>
            <xs:element name="name" type="xs:token"/>
            <xs:element name="value" type="xs:anyType"/>
          </xs:sequence>
        </xs:extension>
      </xs:complexContent>
    </xs:complexType>
  </xs:element>

</xs:schema>

Which then allowed me to create:

<nc:rpc message-id="101"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"
        xsi:schemaLocation="urn:ietf:params:xml:ns:netconf:base:1.0 4741bis-4.xsd">
   <dmi:my-rpc xmlns:dmi="http://xml.juniper.net/dmi"
               xsi:schemaLocation="http://xml.juniper.net/dmi my-rpc.xsd" >
      <name>token</name>
   </dmi:my-rpc>
</nc:rpc>

And:

<nc:rpc-reply message-id="101"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"
        xsi:schemaLocation="urn:ietf:params:xml:ns:netconf:base:1.0 4741bis-4.xsd">
  <dmi:my-rpc-reply xmlns:dmi="http://xml.juniper.net/dmi"
                    xsi:schemaLocation="http://xml.juniper.net/dmi my-rpc.xsd">
	<name>token</name>
	<value>anytype</value>
  </dmi:my-rpc-reply>
</nc:rpc-reply>

All validating with XMLSpy 2011

BTW: is the NetConf schema post somewhere online? - or will projects always need to copy/paste it from the spec and distribute it as part of there schema?

Thanks,
Kent

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 05 Dec 2010 14:16:01 +0100 (CET)

> XMLSpy 2011 says that the 4741bis-4 schema is valid after this update

I validated this schema with libxml as well.

So, does anyone object to this change?

Since the current schema in draft -06 is broken we need to do
something.  I believe this is the best fix; it actually makes the XSD
a bit more useful, since we get an explicit mechanism for handling
custom rpc-replies (just as we already have with custom rpcs).

Tools that generate XSD compatible with 4741bis from YANG models
(e.g. pyang) should be updated to use this new substitution group
mechanism, but this should be ok.

> BTW: is the NetConf schema post somewhere online? - or will projects always
> need to copy/paste it from the spec and distribute it as part of there schema?

IANA keeps it at
http://www.iana.org/assignments/xml-registry/schema/netconf.

But in general copy & paste from the RFC is done also for other
formats; mibs, yang models, abnf grammars etc.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Sun, 05 Dec 2010 14:16:32 +0100 (CET)

> otherwise a YANG validator will complain that container 'data' has no child

+1

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 6 Dec 2010 10:10:55 -0800

Hi,

The way this is handled in yuma is not perfect YANG,
but it allows better auto-validation than anyxml.

  container config {
     ncx:root;
  }

The 'root' extension tells the tools that this is a config root,
not a container with no children defined.  The allowed content
is the set of available top-level YANG data nodes (known to the tool).
This works for <my-custom-edit> or <edit-config> -- any container
tagged as ncx:root.

We can't really do something like this in 4741bis.
But IMO we need to standardize on YANG as the NETCONF content layer,
and stop assuming a NETCONF config can be 'anyxml'.

Andy

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Sunday, December 05, 2010 5:17 AM
To: Andy Bierman
Cc: [email-masked]; [email-masked]
Subject: Re: [Netconf] 4741bis-04 has non-deterministic schema

+1

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 06 Dec 2010 20:30:19 +0100 (CET)

> This works for <my-custom-edit> or <edit-config> -- any container

But the tool could also do

  anyxml config {
      ncx:root;
  }

> and stop assuming a NETCONF config can be 'anyxml'.

I would argue that anyxml is the correct construct to use even if all
content was described in YANG modules.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 06 Dec 2010 12:14:30 -0800

On 12/06/2010 11:30 AM, Martin Bjorklund wrote:
> Andy Bierman<[email-masked]>  wrote:
>> Hi,
>>
>> This works for<my-custom-edit>  or<edit-config>  -- any container
>> tagged as ncx:root.
>

good point -- I should change ncx:root to use anyxml.

> content was described in YANG modules.

Yes -- an NP container is its own special YANG creation.
A real <config> node might have the wd:default attribute in it,
or other XML attributes.

I would prefer to keep mixed XML content, or a text <config> node, out
of the standard.  NETCONF is complicated enough with just YANG as the content.

Andy

---

From: Kent Watsen <[email-masked]>
Date: Mon, 20 Dec 2010 14:55:31 -0800

Just a quick update (nothing effecting 4741bis-04) to correct my previous posting, in case someone looks at this in the future...

Noticing that the NETCONF XML example draft-ietf-netmod-yang-13#section-4.2.9 doesn't have any namespace prefixes:

    <rpc message-id="101"
          xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
       <activate-software-image xmlns="http://acme.example.com/system">
         <image-name>acmefw-2.3</image-name>
      </activate-software-image>
    </rpc>

means that what I posted on Dec 3rd shouldn't have either.  To fix this, I modified the schema attributes in the "my-rpcs.xsd" file as follows:

<xs:schema
   xmlns:xs="http://www.w3.org/2001/XMLSchema"
   xmlns:nc="urn:ietf:params:xml:ns:netconf:base:1.0"
-  xmlns:dmi="http://xml.juniper.net/dmi"
   xmlns="http://xml.juniper.net/dmi"
   targetNamespace="http://xml.juniper.net/dmi"
+  elementFormDefault="qualified">

Which then allows (according to `xmllint --schema my-rpc.xsd my-rpc.xml`) the rpc to omit the prefixes.  Here is the updated rpc sans prefixes:

<rpc message-id="101" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" >
   <my-rpc xmlns="http://xml.juniper.net/dmi">
      <name>token</name>
   </my-rpc>
</rpc>

Notes:
  - XMLSpy could not validate this anymore since the schema no 
    longer specifies the XSD to validate against
  - removing the "xmlns:dmi" line wasn't technically required, but
    I did anyway as it was no longer needed

Thanks,
Kent