Subject: Re: [OPSAWG] XSDMI

From: Phil Shafer <[email-masked]>
Date: Tue, 13 Nov 2007 12:13:10 -0500

[Joining the conversation late, but .....]

"David Harrington" writes:
>Juergen has raised the issue that the XSD produced by the XDMI effort
>and the XSD produced by the YANG effort might be different, and having
>two different XSD equivalents for MIB datatypes would be suboptimal. I
>agree.

Cool....

>However, I have the concern that while vendors wait for YANG to be
>completed, they will need to develop proprietary data models for use
>with netconf. Some vendors will want to reuse datatypes found in the
>15 year compilation of IETF NM experience found in the standard MIB
>modules, and supported by existing implementation code.

Can this concern be addressed by altering XSDMI to follow the
encodings of YANG?  There are already SMI->YANG translators and
YANG->XSD translators, so the SMI->XSD path is covered.  How
different are the XSDs produced by XSDMI and SMI-based YANG?

>The XSDMI effort is about taking the MIB, which operators have
>complained is not well suited to configuration, and translating
>aspects of the MIB into equivalent XSD.

IMHO translating MIBs into XML will not make them instantly
configuration friendly.  There's a lot more that needs to
happen.

>The YANG approach for
>netconf focuses not only on configuration, but also on developing an
>information modeling language which can then be used by tools to
>generate syntax-specific data models for a variety of purposes, such
>as Netconf-suitable XSD data models, SNMP-suitable MIB modules, plus
>data models for other modeling systems.

I don't think we're that ambitious.  ;-)

Quite the opposite, we tried to minimize the problem domain we were
focusing on, in order to be able to define a solvable problem.  We
leveraged the knowledge gained from having solved this problem in
multiple proprietary implementations to create a data modeling
language that NETCONF modelers can use to define their models.  We've
put features in that have allowed our previous implementations to
achieve the scale and power that is required in our problem space.
The utility of YANG outside that problem space is not interesting to
us.

>Just to be clear, I support the YANG effort because I believe it could
>lead to better consistency of data modeling across protocols, but it
>has yet to be proven to be viable, and other efforts to converge or
>harmonize data models have had mixed success at best.

Yes, YANG is boldly going where many have tried before, but the strength
of YANG is that it's based on working products that use capabilities
of the modeling language to support industrial-strength NETCONF
implementations.  Think of this as field-trials, meaning the
YANG-style solution's viability has been proven in the core for many
years.

>If the YANG approach proves to be
>viable, then the XSDMI translations can be modified as needed to work
>with YANG in the future. But XSDMI could give us XSD datatypes to
>develop data models, possibly years before consensus on YANG will be
>achieved.

IMHO this puts the cart before the horse, giving an encoding/syntax
(XSD) that modeling languages like YANG will have to encode to.  I
can't see a new XSDMIv2 being churned out that becomes
YANG-compatible.  If you want to be compatible, start now.

Thanks,
 Phil

_______________________________________________
OPSAWG mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/opsawg

---

From: "David Harrington" <[email-masked]>
Date: Tue, 13 Nov 2007 16:47:33 -0500

inline 

> -----Original Message-----
> From: Phil Shafer [mailto:[email-masked]] 
> 

Here's what I know about the XSD encodings of YANG:
from 
"It is possible to generate an XSD document representing the data
   model defined in YANG.  The XSD will specify the structure of the
   model, the used types, cardinality, etc. but a good part of the
   information in YANG can not be represented in standard XSD
constructs
   e.g. config, status, default, keyref.

   The above information will be added to XSD inside annotation
   statements (which can not be handled by standard XML tools).  The
   exact form of annotations is for further study.

   Data models defined in YANG might have multiple top-level elements.
   To make it possible to validate the XML encoded data, both for the
   generated XSD describing the data model and for the config and
state
   data received in NETCONF RPCs a single top level element <data>
shall
   be added.

   From a YANG model many different XSD schemas can be generated
   defining the same data model.  The exact rules to generate XSD from
   YANG are outside the scope of this document, however some general
   considerations are needed.
"

The WG has absolutely no idea how different the translations of XSDMI
and YANG are because we have never seen the XSD output from YANG.

> configuration friendly.  There's a lot more that needs to

And it is not our intention to make MIB modules
configuration-friendly. The XSDMI translations are not about Netconf.
It is solely about making SMI datatypes and textual conventions
avaiable for use in XML-based environments that **choose** to use the
XSDMI translations.

> The utility of YANG outside that problem space is not interesting to

But it is interesting to others who may want to use the XSD
equivalents of SMI datatypes and TCs for non-YANG and non-Netconf
purposes.

> YANG-compatible.  If you want to be compatible, start now.

We have published our XSD encodings in the IETF. YANG has not. 

If there are SMI->YANG and YANG->XSD tools, then why aren't you
publishing the specs and the output so we can see the translations so
we can compare them? If yours are better than ours, which may be the
case, then by all means we can try to be compatible starting now, but
we have not seen your encodings; we cannot start trying to be
compatible until you do publish your encodings.

I have seen three different proposals for doing SMI-XSD translations,
not including the YANG approach, and each approach has advantages and
disadvantages. I would like to have the OPSAWG be able to consider all
the different tradeoffs and decide as a community which approach is
best. 

I do not like abdicating all responsibility to a closed design team to
make the decisions for the IETF. Are all your design decision based on
achieving vendor-neutrality, or are some decisions favorable to the
existing implementations of certain vendors? Are any of the designs
encumbered by IPR? Are you making decisions to trade away some feature
that others may want without even consulting those who might be
impacted by the decision?

I am not expecting XSDMI to become a standard in the next two months
or the next six, and probably not the next year (this is the IETF
after all). So there is plenty of time to try to build compatibility
between the approaches, if you publish your specs and output. If YANG
can deliver on its promises of a viable solution, and the industry
accepts it, I have no issue with deprecating XSDMI. But I do not want
to **not** do XSDMI, and not have that point of comparison, just
because you promise a better solution. I've dealt with too much
vaporware in my life to want to put all my eggs in one promised
basket.

I'm not trying in any way to require YANG to use the XSDMI encodings,
or otherwise contrain the design of YANG. I think YANG and SMI are in
different applicability spaces. I resent that members of the YANG team
want to force the rest of us to use only their encodings and to
constrain the IETF from doing any work on an SMI-XSD translation that
is not the YANG translation.

>  Phil

dbh

---

From: Phil Shafer <[email-masked]>
Date: Tue, 13 Nov 2007 17:32:05 -0500

"David Harrington" writes:
>make the decisions for the IETF.

I don't think we're asking for that, we've just burning
the midnight oil trying to get the YANG spec out.  Now
we're finishing up tools and such.  More will be available
shortly.

>Are all your design decision based on
>existing implementations of certain vendors?

I think the group has done an incredible job of designing YANG with
the benefit of hindsight on existing projects, with foresight toward
future uses, and with vision for the growth of NETCONF.

>Are any of the designs
>encumbered by IPR?

No.

>Are you making decisions to trade away some feature

Absolutely.  That's the purpose of using a small design group.  Well
considered design decisions without consulting the entire population.

>If YANG
>because you promise a better solution.

Cool.  Take a read of the spec, and we can sit down and walk
thru features point by point in Vancouver.

>I think YANG and SMI are in
>different applicability spaces.

I guess I'm confused about what space XSDMI is aiming for then.
Who is your target audience?  What protocol(s) will use it?

Thanks,
 Phil

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 13 Nov 2007 23:45:38 +0100

> compatible until you do publish your encodings.

Get the latest libsmi sources and run smidump -f yang to turn all your
MIB modules into yang. There is currently no specification how this is
done - just running code. ;-) For the yang to xsd translation, there
is a running implementation as well, but it is not yet public. Please
give us a few more days; working out the details of the language,
writing a specification, implementing tools, bug fixing tools, dealing
with IETF decision bodies, preparing slide ware, ... all takes time
and energy.
 
> impacted by the decision?

A clear "no" to all your concerns. Starting with a small group of
technical people was the only way to get this going and we are now in
the process of putting all details and implementations on the table
for consideration by the IETF and the NETCONF community. There is no
hidden agenda.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 15 Nov 2007 12:00:03 -0800

> with IETF decision bodies, preparing slide ware, ... all takes time

I can concur with Juergen that writing a good YANG to XSD translator
is hard enough.  Writing down every algorithm is very difficult.
Free tools should be good enough for now, but I agree with Dave
that a standard XSD representation of SMIv2 is needed, and the
YANG to XSD translation needs to match, for all SMIv2 TCs and types.

I think there is a big difference between alignment between
SMIv2 and YANG data types, and aligning the SMIv2 and YANG languages.
The latter is a non-goal, and translation from YANG to SMIv2 is
not supportable.

There is a significant issue created by romascanu-datatypes,
because it is a 'best hits' collection of TCs from other MIB modules.
I would much rather have an architecture based on the SMI to YANG
to XSD conversion path, in which the original modules are preserved.
Creating a best-hits module is going to create import conflicts if
the original modules are ever converted to XSD as well.

> for consideration by the IETF and the NETCONF community. There is no

Nobody thinks the IETF should rubber-stamp YANG.
IMO, YANG should be published as an Experimental RFC ASAP,
to get some experience with the DML and to develop some
more free tools.

The standards-track path can be separate, and be as
slow and deliberate as necessary.  By the time the IETF
figures out what to do about NETCONF and XSD, running code
will probably overtake events anyway.

> /js

Andy

---

From: "David Harrington" <[email-masked]>
Date: Thu, 15 Nov 2007 17:47:47 -0500

> YANG to XSD translation needs to match, for all SMIv2 TCs and types.

I agree that a standard XSD representation of SMIv2 is needed, but I
do not fully agree that the YANG-to-XSD needs to match the XSDMI
transaltions.

While it would appear beneficial for all management interfaces to use
the same datatypes, experience has taught us this is not necessarily
true. That is why Netconf is looking to use XML/XSD instead of SMI,
and why RADIUS and Diameter use named attributes, and IPFIX uses IEs,
and syslog uses SDEs, and so on. Different purposes may benefit from
different types of data models, and different datatypes for the same
concept.  

At the same time, two different standard protocols may use the same
basic encoding rules, such as ASN.1 or XML, without having to use
exactly the same language for modeling. 

We learned that in a smaller way in SNMPv2. We started out believeing
that there should be one MIB for SNMPv2 and all aspects of SNMPv2
should use those same objects. But as we got into more detailed
design, we found that actually the different-purpose portions of
SNMPv2 had slightly different assumptions about the managed objects in
the MIB, and that caused conflicts as people working on different
parts tried to slightly redefine the managed object, and caused other
parts to no longer work properly. In SNMPv3, we deliberately chose to
separate the MIB module for access control from the MIB for
authentication, even though both MIB modules included the name of the
authenticated principal.

There are multiple use cases for an XSD translation of SMIv2. I do not
believe that YANG should be constrained to produce XSD that will be
viable for all use cases. I think YANG should focus on YANG-XSD
translations that are useful for configuration, including new data
modeling approaches for configuration with new aspects such as nested
structures. 

If romascanu-netconf-datatypes had not been proposed first in the
netconf community, we could have had these translations to XSD done
years ago. But any data modeling work, even for non-netconf use cases
was held up by "wait until the netconf protocol is finshed"; now you
want "wait until the YANG tools are finished, and then wait until we
get around to documenting the algorithms, and then bring the specs to
the IETF, so we can debate any variations used by different tools, and
then go through the whole standards process to get the IETF stamp of
approval." 

If there are use cases that are not germane to Netconf configuration,
why must we wait for YANG?

Think of a manager station that uses an XML-compatible database. The
manager application designers want to use XSD to validate SMIv2 values
received from SNMP agents before it puts those values into its
XML-based records. This saves them the cost and effort of developing
SMI datatype validation routines, since they can use the XSD
validation routines of their database. 

They need the XSD to accurately reflect the original semantics that
were specified in SMIv2. It would be useful to have the IETF provide
standard XSD for those validations, since we have the expertise about
the original semantics of the SMIv2 datatypes, and we can save people
the cost of figuring out and debugging the SMI-XSD translations
themselves.

And they shouldn't need a netconf-specific YANG toolkit to get these
translations.

> the original modules are ever converted to XSD as well.

Why don't we just have XSDMI use a prefix of smi: for its translated
datatypes, and have YANG use a prefix of yang: for its XSD output?
Wouldn't that solve the imports problem you raise?

> will probably overtake events anyway.

XSDMI is not about NETCONF!!!!
It is about providing XSD to accurately reflect the original semantics
that were specified in SMIv2, so people can utilize the XSD to
validate SMIv2 data that may not have anything to do with NETCONF.

---

From: Andy Bierman <[email-masked]>
Date: Thu, 15 Nov 2007 17:58:19 -0800

> do not fully agree that the YANG-to-XSD needs to match the XSDMI

It is less work to standardize a TC once.
In every case, the semantics (DESCRIPTION clause) are
cut and pasted from SMIv2 into the XML data model.  The only issues
are syntax-related.  Hopefully, the syntax differences can
be resolved.  If not, then 2 versions of the same TC would
be the result, hopefully with different names.

 >....

I keep hearing about the mystery protocols behind door #2.
What other protocols besides NETCONF are relevant here?

There is nothing NETCONF-specific at all about any of
the SMIv2 TCs in the YANG I-D.  A few TCs in romascanu-datatypes
are SNMP-specific, such as RowStatus.  It is unlikely that
a RowStatus object would have relevance outside the SNMP protocol.

However, a set of derived types is not even close to the
same thing as a data model.  Having a standard InetAddress
doesn't help that much unless there are standard objects
using that type.  Say one object is for the DHCP server address
and another is the default gateway address.  Just knowing
that they are both InetAddresses doesn't help much without
the data model.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 16 Nov 2007 08:57:40 +0100

> do not fully agree that the YANG-to-XSD needs to match the XSDMI

It does not have to, but it is really bad if it does not...

> validation routines of their database. 

I assume this manager station will have lots of fun once SMIv2 data
comes also in via NETCONF and the XML representation is different.
The IETF is about interoperability and multiple ways to represent the
same data has so far been considered not a good idea to achieve
interoperability.

/js

---

From: Andy Bierman <[email-masked]>
Date: Fri, 16 Nov 2007 09:52:26 -0800

Hi Dave,

>>...

I told the IESG I did not object to their no-YANG-BoF decision,
based on the NETCONF-only factor.  But it seems you are right.
Different protocol WGs are not encoding their data structures
the same way at all, and that is not likely to change.

YANG gets some of its power (like all app-specific DMLs)
by abstracting away, or hard-wiring, various aspects of
the protocol/application mapping.

In YANG, the data definitions are hard-wired
to be placed inside <config> parameters within NETCONF RPC
operations.  The rpc and notification definitions fit
within the <rpc> and <notification> PDUs, respectively.

When I said another protocol WG could do the work to
define how seemingly generic data structure definitions
(like list, container, leaf) fit within a different
protocol PDU structure, I didn't mean YANG should be generic.
(It is definitely OPS-NM specific, like SMIv2, because
network operators need to read and understand the data structures.)
I just meant that the IESG is right to be concerned
about too much WG overlap, and inventing too many
ways to do the same thing.

But now I do not think there is any overlap.
NETCONF needs a DML that ordinary people can read.
People who do not write code or protocols for a living,
but run networks for a living.  YANG is simple enough
to meet that requirement.  It is also powerful enough
to model even the most complex NETCONF configurations,
including runtime constraints via Xpath expressions,
extensible templates, and conditional object augmentation.

Andy