Subject: [OPSAWG] XSDMI work

From: "David Harrington" <[email-masked]>
Date: Fri, 10 Aug 2007 12:56:32 -0400

Hi,

The XSDMI BOF proposed the creation of a WG to develop 1) the XSD
equivalents of datatypes and textual conventions from SMIv2, and 2)
algorithms to translate MIB module specifications into XSD
equivalents. The BOF demonstrated community support for these two work
items. 

People in the BOF volunteered to commit to being editors (3) and
reviewers (14) for these work items. Starting documents and
implementations already exist. The work is expected to take less than
12 months.

At Dan's suggestion, rather than starting a new WG for this work, we
have proposed that the SMI-to-XSD translation work be done in the
OPSAWG.

The OPSAWG chairs need to determine whether the OPSAWG WG should
accept this work. Please send email to [email-masked] with comments
about your support/lack of support for having the OPSAWG do this work.
(If you do not subscribe to opsawg, you might want to do so, in order
to follow the discussions.)

David Harrington
[email-masked]
[email-masked]

_______________________________________________
OPSAWG mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/opsawg

---

From: Andy Bierman <[email-masked]>
Date: Fri, 10 Aug 2007 10:19:03 -0700

> have proposed that the SMI-to-XSD translation work be done in the

I have an objection to this "translation" work.
You will make all kinds of assumptions about data organization
and data naming, and I'm not sure what people are supposed
to do with the XSDs.

The term "XSD translation" is misleading.
The XML on the wire is what is at stake here.
The smidump translation of a MIB has SNMP operations
built into it.  Is this work supposed to produce
a standard way to send an SNMP PDU over any protocol
designed to encode SNMP PDUs in XML?  Why do we need this?

Is the charter going to state that the XSD translation will
be suitable for use for any particular protocol?
NETCONF needs to standardize XML data organization,
namespace usage, and Xpath-based data naming.
Is this translation going to be compatible with that standard
when it eventually comes out?

I would like the charter to be clear on the purpose of this work item,
and what 'protocols' are signed up to use it.

> to follow the discussions.)

Andy

---

From: Ron Bonica <[email-masked]>
Date: Mon, 13 Aug 2007 10:32:27 -0400

Andy, David,

Permit me to ask a few dumb questions so that I can understand a) what
David is proposing and b) why Andy is objecting.

AFAIKS, David is proposing the following:

- a mapping from SMIv2 TCs to XSD types
- a mechanism to translate SNMP MIBS to XSD.

The XSD is an XML schema that is independent of the protocol that is
used to shuffle the data back and forth. In one respect, it is a bit
like Esperanto. It strength is that it omits the details that make it
inappropriate for one protocol or the other. But at the same time, it's
weakness is that it lacks the detail that would be required by any protocol.

David, Andy, do I have this right? If so, could you guys point out what
some of those omitted/lacking details are?

                                 /speaking as individual contributor
                                 /and by no means a data modeling expert
                                  Ron

---

From: Andy Bierman <[email-masked]>
Date: Mon, 13 Aug 2007 08:53:09 -0700

> weakness is that it lacks the detail that would be required by any protocol.

That's the pretty picture from 10000 feet

> some of those omitted/lacking details are?

The XSD is going to contain <key> clauses that specify how
data naming works for the multiple conceptual instances
of the 'table row' (i.e., some ancestor node of all index nodes).
In order to be useful for a particular protocol, this naming scheme
must be compatible with that protocol.
The conceptual data structures are going to use data types that must
be supported in that protocol, which impacts DM design.

In order to be useful for any protocol, another document
would be needed that maps every detail in the SMI-to-XSD algorithm
to every detail of the operations (verbs) for that protocol.
This is 90% of the work.

This generic SMI-to-XSD mapping is the other 10%.
IMO, this just complicates things.  I would rather
the 90% document be 100%, and map SMIv2 directly
to the protocol, rather than mapping from SMIv2 to an XSD info model,
and then to the protocol.

The current proposal contains lots of SNMP-specific details,
like engine-IDs, contexts, and max-repetitions.  This suggests
that the mapping is for SNMP, not SMIv2.  IMO, seamless integration
of MIB-based data into the NETCONF protocol is an interesting
problem to solve.  Using a NETCONF session to send SNMP PDUs in XML
to the NETCONF agent, who will forward it to the SNMP agent,
is not an interesting problem to solve.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 13 Aug 2007 09:07:29 -0700

Hi Ron,

BTW, there is a simple algorithm for data organization
that works even without namespaces.

We know that descriptors are not required to be unique
across modules, but module names are required to be globally
unique.

This simple Xpath expression works for any MIB object:

/MODULE-NAME/TableName/TableEntry[index components]/tableColumn

/IF-MIB/ifTable/ifEntry[ifIndex=17]/ifName

/FOO-MIB/fooTable/fooEntry[x=1][y=2][name='fred']/fooColumn

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 13 Aug 2007 08:53:09 -0700

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 13 Aug 2007 09:07:29 -0700

Hi Ron,

Andy

---

From: "David Harrington" <[email-masked]>
Date: Mon, 13 Aug 2007 13:25:08 -0400

Hi,

(Andy raised some points on the NGO list. I am responding on the
OPSAWG list, since the question is about accepting this work into the
OPSAWG WG.)

It is not just the XML on the wire that is at stake here. The proposed
work is almost solely about translating from SMI data model to XSD
models, independent of any protocols that might use it.

The SMIDUMP utility translations are documented in
draft-li-mib-convert-00. This was done to give us something concrete
to discuss. The SMIDUMP utility does mix some protocol-dependent
information into its XSD. We should try to make sure the SMI-to-XSD
translations are as protocol-independent as we can make it.

A few years ago, other SDOs decided they needed mappings from their
layer 2 protocols to IP, and decided to ask the IETF to do that work.
The IETF decided it would make more sense to define an IETF-standard
interface to IP that was independent of any specific layer 2 protocol,
and then the engineers who wanted specific layer 2 protocols to work
with IP could determine how to map from their protocol to the
protocol-independent interface. That is what I am proposing here -
creating a protocol-independent interface to MIB data. Then other
protocols can write to that interface if they choose. I am not trying
to require that other protocols utilize the resulting XSD, however. If
the designers of protocols have a better approach for accessing MIB
data for their purpose, have at it!

MIB modules are used in multiple ways. 

1) they are used as specifications for implementations on agents,
where the format of the data in a data model must be known so that,
for example, a Counter64 is allocated a 64-bit variable for counting,
and a SIZE 255 OCTET STRING can be allocated 255 bytes to storage, and
an object of a TEXTUAL-CONVENTION like SnmpAdminString can be properly
formatted using the rules associated with that TC, such as using UTF-8
encoding and no control codes.
	An SMI-to-XSD translation of datatypes is meant to make
SMI-compatible datatypes available for protocols that use XSD to
define datatypes. This will allow new data models to reuse existing
datatypes (such as Counter64) and Textual Conventions in their
designs, if they choose. This would be especially useful so a data
point in an SMI-compatible data model could be compared to a data
point in an XSD-compatible data model.
	Other protocols, such as XCAP, IDMEF, and ATOM use XML-based
data models and define XML Schemas for validating the data models.
Some of those schemas contain datatypes roughly similar to those
defined in the SMI datatypes and textual conventions. For example,
XCAP uses a display-name type; IDMEF specifies addresses in its XML.
The SnmpAdmninsTring type and the InetAddressType/Address/Port textual
conventions define rules about how to format addresses, and identify
address types. The SnmpAdminString and InetAddress* restrictions could
also be useful in an XML Schema to validate the data for an XML-based
data model. 
	These examples only show that complex types constructs  used
in the MIB (textual conventions and application-specific datatypes)
could also be useful in other data models, if converted to XSD. By
making the datatypes from SMI available to other data models in XSD,
the values used could be more consistent across data models.

2) the datatypes are used to identify the format of information being
passed on-the-wire using a protocol, such as SNMP. In a TLV entity
like an SNMP varbind, the datatype is used to identify the type of the
value.
	In XML, the datatype is not passed on the wire in the XML
element. Rather an external schema document, in XML Schema, or a DTD,
or RelaxNG is used to validate the XML.
	The IETF has spent twenty years developing the MIB, and there
is a lot of information about networking protocols embedded in its
information model, and modular data models, based on the skills and
experience of protocol developers, administrators, and equipment
vendors.
	If an XML-based protocol like Netconf chooses to leverage that
information by accessing a managed object in the MIB, the receiving
application (the Netconf client or the manager application using
Netconf to gather data) should know the schema of the  datatype of the
values being accessed so the data can be validated. 
	If the Netconf client or the manager, however, do not already
support SNMP and the SMI, it would be more convenient for the Netconf
client (the manager) to be able to perform validation of the data
using XSD that is compatible with the corresponding SMI definition.
	An SMI-to-XSD translation of datatypes is meant to make
SMI-compatible validation available for protocols that carry SMI-style
MIB objects in an XML format.	

3) MIB module specifications are "imported" by SNMP managers to
develop corresponding database records reflecting the structures of
the tables, the allowed value ranges, and so on; The MIB module
specifies restrictions, such as ranges. that allows the manager
application to validate information received over the wire, such as in
an SNMP varbind.
	Many databases now have XML interfaces, and can import XSD
directly to generate internal database record formats, and then the
database can validate XML data going into those records.
	For a developer that wants a new manager application to be
able to validate MIB information retrieved using an XML-based
protocol, who uses a database with an XML interface, and who doesn't
want to have to support an SMI-specific import facility, being able to
import the XSD equivalent of a MIB module specification would be
useful.
	A MIB-to-XSD translation would be helpful. One goal is to
provide a standard for translating MIB specification elements other
than datatypes and textual conventions for this "import" purpose, a
purpose which is possible using the SMI. 

4) MIB modules identify how to "address" a specific managed object, so
a manager can ask an agent for a specific object. The OID format used
for this is widely hated, but if a protocol needs to identify an
object in the MIB to access, there needs to be a way to translate the
OID addressing into an XML-compatible equivalent. 
	One way to aproach this is define a whole new data model using
a non-SMI data modeling language, such as XSD or the new data modeling
languages being worked on in netconf (NCX and YANG), and these new
languages would need to define how to address the undwelriny
instruentation directly without going through the SMI-style interface.
There is always the possibility that doing this will result in
slightly different semantics or modeling, which would be undesirable.
It would also be indesirable to ignore all the existing MIB data
models and define all new data models to access the same information
already being made available on device implementations.
	Translating the OID addressing to an XML equivalent addressing
should be a relatively straight-forward algorithmic translation of the
hierarchical OID registration values to a corresponding XML
hierarchical naming scheme using the SMI object descriptor names. That
would be one way to do it; other ways to address the same information
might be possible depending on the protocol. The XSDMI effort would
probably need to define one algorithmic approach, but that does not
mean protocls could not use a different approach if that was
justified.

5) MIB modules are frequerntly used to provide information to NMS
users as help screens. The SMI if displayed, so NMS users can read
what the purpose and restrictions are for the data. Part of the
purpose of moving toward XML and the XML family of protocols is that
CS students are being trained in these protocols, but not in SMI.
Assuming administrators will begin to have a better understanding of
XSD, NMS developers might display thew XSD related to a data model
definition. So if Netconf defines a data model in XSD, unrelated to
the MIB, but also accesses data from the MIB, it would be unfortunate
if the help screen for one managed object was in XSD, with th eother
was in SMI; it would seem better to make it possible for NMS
developers to be able to display the XSD equivalent of the SMI, if
desired.
	The XSDMI work should define one translation from SMI to XSD.
Other efforts, such as Netconf's YANG or NCX proposals may have their
own protocol-specific translations as well.

David Harrington
[email-masked]
[email-masked]

> -----Original Message-----
> From: Andy Bierman [mailto:[email-masked]] 
> Sent: Friday, August 10, 2007 1:19 PM
> To: David Harrington
> Cc: [email-masked]; [email-masked]; [email-masked];
[email-masked]
> Subject: Re: [NGO] XSDMI work
> 
> David Harrington wrote:
> > Hi,
> >

---

From: "David Harrington" <[email-masked]>
Date: Mon, 13 Aug 2007 15:21:08 -0400

Hi,

I will respond inline 

> Sent: Monday, August 13, 2007 2:26 PM
> To: David Harrington
> Cc: [email-masked]; 'Ron Bonica'
> I don't really want to respond inline, so
> I'll just point out some NETCONF details:
> 
>   - NETCONF configuration is an extension of proprietary CLI,
>     not SNMP. The SNMP protocol, architecture, transport, security
>     and access control model are not factors in the network
>     configuration problem.  But there is some MIB-based 
> monitoring data
>     that has become part of the operational framework of many 
> networks.

Agreed, and the translationof SMI datatypes to XSD datatypes, and the
translation of MIB documents into XSD Schemas has nothing to do with
the SNMP architecture, SNMP transports, SNMP security, or SNMP access
control. A MIB module defines an SMI-based interface to the underlying
instrumentation, and an XML-based data model can used XML Schema to do
the same. For the same instrumentation, it would be a good thing if
the XSD was the equivalent of the SMI interface, if the underlying
data is the same.

Over twenty years, we have standardized a wide range of knobs and
vernodrs have provided hooks in their implementations to provide the
information for those knobs. Vendorsd would like to be able to reuse
those some of those same hooks rather than starting the twenty year's
worth of data modeling all over again.
 
> 
>   - NETCONF data modeling has been a real challenge
>     because there is no pre-existing agreement on the CLI,
>     so standard config, whether encoded in XML or Morse code,
>     requires a lot of agreement on the standard knobs first.

Agreed. and hopefully the Netconf community will be able to drive
consensus on those standard knobs for configuration, which the IETF
has been totally unable to do so far.

On the other hand, over the course of twenty years we have reached
consensus on a wide variety of knobs useful for performance and fault
monitoring and useful for notifications, and it would be a share to
ignore all that work we have alreadty reached consensus on just
because no consensus was reached on configuration knobs. It is often
quite adequate to monitor a certain subset of the instrumentation, but
to do configuration requires knowing all the relevant knobs. 

I am in the camp that believes non-standard configuration knobs will
never go away because that is how vendors differentiate their
products, and only reaching conensus on a **subset** of configuration
knobs will probably not be sufficient, as was shown in the case of MIB
data modeling; that's one major reason why SNMP is not good for doing
configuration. I cannot configure a VLAN using the identical
configuration commands on a Cisco device and an Enterasys device,
because they require slightly different parameters, but I can
certainly monitor how many VLANs have been created and which IDs they
have been assigned, and what protocols they support, even if I cannot
configure them using a standard data model.

> 
>   - There is a big difference between reusing SNMP, reusing SMIv2,
>     reusing conceptual MIB data, and reusing agent SNMP 
> instrumentation.
>     Depending on your goal, you will see the solution space 
> differently.

And the goal is to define how to reuse the MIB data, by defining a
standardized interface to that MIB data that would be useful for
XML-based protocols.

> 
>   - There is a use case for NETCONF reading MIB data 
> occasionally within the
>     the context of provisioning new services, verifying
configuration
>     changes, etc.

And if Netconf wants to use the XSDMI translations, that might be
beneficial to Netconf. or not.

> 
>   - There is some usefulness in a standard way to convert a
>     DESCRIPTION clause to a <documentation> element, etc.,
>     just as the romascanu-datatypes RFC will specify how to
>     convert an OCTET STRING data type to an xs:string, or 
> BITS to xs:list, etc.

That's what we're shooting for ...

> 
>   - It is not clear how much SNMP this new WG is trying to add to
>     other protocols, or how much SMIv2 it is trying to
>     turn into something protocol-neutral, or how a real protocol
>     utilizes the output XSDs.  I am just asking for clarification
>     on these issues in the charter, that's all.

Personally, I am shooting for no SNMP-centric stuff; I am also
shooting for no Netconf-specific stuuf; I am shooting for
protocol-independent. Most MIB data models (except of course the ones
for managing SNMP) are independent of SNMP, and as a MIB Doctor I have
worked to keep them that way.

MIB data models are defined using SMI, and XML-based applications and
protocols could benefit from using the existing data models for
purposes of monitoring and notifications and other things. We need to
understand how to translate the SMI to XSD in order to fully
understand a MIB data model and make it easier to work with in an
XML-based environment. For those XML-based applications that want to
validate the XML data they get, it would be useful to have the XSD
equivalent of MIB data model datatypes and textual conventions and
table specifications and notification specifications and so on. 

That said, a MIB data model is just an interface to the underlying
instrumentation, and there may be ways to ignore the MIB interface to
access the same underlying data directly. Operators and application
designers should choose the best combination of protocols and data
models to meet their needs.

SCSI is a standardized interface to an underlying storage device.
While some applications benefit from direct access, most find it
easier to use a standardized interface; others may prefer to use
different standard interfaces for storage device access. The XSDMI
effort is about providing **a** standard interface that can be used by
those who would prefer not having to design their own low-layer access
routines to utilize MIB data.

David Harrington
[email-masked]
[email-masked]

> >>> At Dan's suggestion, rather than starting a new WG for this
work,
> > we

---

From: Andy Bierman <[email-masked]>
Date: Mon, 13 Aug 2007 11:25:40 -0700

- NETCONF configuration is an extension of proprietary CLI,
    not SNMP. The SNMP protocol, architecture, transport, security
    and access control model are not factors in the network
    configuration problem.  But there is some MIB-based monitoring data
    that has become part of the operational framework of many networks.

  - There is a big difference between reusing SNMP, reusing SMIv2,
    reusing conceptual MIB data, and reusing agent SNMP instrumentation.
    Depending on your goal, you will see the solution space differently.

  - There is a use case for NETCONF reading MIB data occasionally within the
    the context of provisioning new services, verifying configuration
    changes, etc.

  - There is some usefulness in a standard way to convert a
    DESCRIPTION clause to a <documentation> element, etc.,
    just as the romascanu-datatypes RFC will specify how to
    convert an OCTET STRING data type to an xs:string, or BITS to xs:list, etc.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Sun, 19 Aug 2007 09:31:35 +0200

> about your support/lack of support for having the OPSAWG do this work.

I do not support this work much as I do not support the work to define
XSD standard data types at this point in time, regardless of where
this is hosted in the IETF.

I believe both work items conflicts with NETCONF data modeling work
and in particular with naming issues (how to identifiy XML elements
that serve as keys) and data organizational issues (although XML
namespaces somewhat help here to deal with potentially different
approaches).

To be clear: I do support work in this space as it is of high
practical relevance; I do not want to be seen as negative. I just feel
that some more core NETCONF SMI issues have to be sorted out before we
are ready to work on _standards_ for doing translations or _standard_
reusable data types.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>