Subject: Re: An action plan for pushing the SMI to Standard

From: "C. M. Heard/VVNET, Inc." <[email-masked]>
Date: Tue, 16 Jun 1998 17:53:37 -0700 (PDT)

On Tue, 16 Jun 1998, Harald Tveit Alvestrand wrote:

> The SNMP community has expressed a need to see the Structure of Management
> Information (v2), RFCs 1902, 1903 and 1904, advance to Standard status.
> 
> A number of hopefully minor issues have been raised related to these
> documents; it is clear that some work, hopefully just clarifications, is
> needed before we can advance the documents.
> 
> In order to get this work done rapidly and throughly, the O&M ADs have
> asked a small team to work out proposed new documents, well in advance
> of the August IETF, that can be reviewed there and hopefully advanced
> to Standard in a reasonably short time.
>
> [ ...]
>
> This work is NOT about adding new functionality [ ... ]

In the RMONMIB WG Minutes for IETF #40 (Sun, 4 Jan 1998,
from Andy Bierman) I notice this:

> 2.2) Gauge64 vs. Counter64
> 
> The RMON MIBs define some TCs:
>   ZeroBasedCounter32  (based on Gauge32, defined in RMON-2)
>   ZeroBasedCounter64  (based on Counter64, defined in HC-RMON)
>   ZeroBasedGauge64    (based on Counter64, defined in HC-RMON)
> 
> The use of Counter64 for the base type is not correct.  
> The correct approach for the HC-RMON MIB is to add an
> unsigned 64-bit data type to the SNMP SMI. This issue has 
> been deferred to the SNMPv3 WG.

I gather, on the basis of the words "this work is NOT about adding
new functionality" that this issue (and others like it) will NOT
be addressed by the SMI team.  Is this inference correct?

Thanks,

Mike
--
C. M. Heard/VVNET, Inc.
[email-masked]

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Wed, 17 Jun 1998 07:59:57 +0200

>be addressed by the SMI team.  Is this inference correct?
This inference - that the SMI team will not be adding new base types
to the SMI - is correct.
There is another, separate activity, that I believe Jurgen will spearhead,
that will collect "useful TCs" that are not part of the current set
and put them separately on the standards track.

Whether this activity can add new base types is something we will
have to discuss; I believe new base types are "controversial".....

                          Harald A

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

---

From: "Brian O'Keefe" <[email-masked]>
Date: Wed, 17 Jun 1998 10:18:56 -0600

> have to discuss; I believe new base types are "controversial".....

Understood. I'm fully supportive of every effort to advance
the SMIv2 to full-standard.  And while there are a number of
"enhancements" that I'd like to propose for SMIv3 work, the 
one that is absolutely critical to get on the standards track 
ASAP is  "Unsigned64".  Whether this addition can be done
without setting the entire v2 SMI (and protocol - rfc1905)
back to proposed is beyond my ability to say.  I do know that
such an addition would require modification, albeit minor, to
the manager-side SNMP API/stack that I work with in order for 
it to become fully supported in our code.   

None-the-less, we need to get this on the standard track in
some form ASAP. With this new SMI base type defined, separate
efforts can add TC variations to one's content.  But without
it, MIB designers will continue to kludge up work-arounds that
just make life simply difficult in the end.

bok

PS - will the SMIv3 effort (and mailing list) be announced
on the snmp or snmpv3 list?

---

From: Keith McCloghrie <[email-masked]>
Date: Wed, 17 Jun 1998 10:29:23 -0700 (PDT)

Brian,

What MIB designers need, so they can avoid having "to kludge up
work-arounds", to be able to write

      SYNTAX     Unsigned64

Observe that they can do this irrespective of whether Unsigned64 is
defined: a) as a base SMI type or b) as a TC.  Where to IMPORT it from
(an SMI or a TC MIB) is the *only* difference in the MIB between these
alternatives.

However, using a new SMI base type would generate an on-the-wire
encoding which is currently illegal, i.e., it effectively creates a new
SMI, requiring a much difficult transition strategy.  On the other
hand, using a TC requires no changes except in consenting method
routines and NMS applications.  So, if you want to be able to use this
anytime soon, ...

Keith.

PS. The same argument is true for Gauge64.

---

From: Bob Stewart <[email-masked]>
Date: Wed, 17 Jun 1998 14:31:27 -0400

>routines and NMS applications.  So, if you want to be able to use this

I don't see why having a new tag is any more traumatic than special-casing
an old one.  If anything I'd think a new tag would be cleaner and easier to
implement.

Without a new tag smart, helpful, general-purpose applications will
mishandle 64-bit integers or unsigned, thinking they're counters and
require calculating a delta.  In other words, special counter semantics go
with the existing tag.  To use it as a 64-bit unsigned integer violates that.

With a new tag we maintain a consistent, protocol-level distinction, and in
*both* cases it works only between consenting, upgraded agents and
applications.

	Bob

---

From: "Brian O'Keefe" <[email-masked]>
Date: Wed, 17 Jun 1998 12:59:43 -0600

Bob, thank you for replying to Kieth for me!!

Kieth, I agree that we're sort of in a pickle here;
but TC's are defined for "refining/tightening" the
semantics of the base type, not for "expanding/loosening"
the semantics.

Let's learn from our mistakes and be a bit more
forward-looking in adding data types to the protocol 
and SMI, rather than taking a protectionist approach
as was done in v2SMI. The new types will get used,
especially in the standards track, where and when 
appropriate with reasonable review. 

After all, who woulda thought 15 years ago that a
PC would need more than 640 Kbytes of RAM?

bok

---

From: Keith McCloghrie <[email-masked]>
Date: Thu, 18 Jun 1998 00:33:38 -0700 (PDT)

Bob,
 
> an old one.  If anything I'd think a new tag would be cleaner and easier to

I'm not talking about "cleaner and easier to implement".  I'm talking
about the practicality of getting products to support it.

It's 5 years since Counter64 was first published in an RFC, and 2.5
years since SNMPv2c specified how it could be used with community
strings.  Yet, there's still lots of systems which do not support it.

If a new SMI could be published by the end of this year (optimistic?),
we will still have many systems in the year 2001 which won't support
the new SMI.

In contrast, defining a new TC does not require a new SMI, and can be
supported in new method routines.  New method routines get added to
most products on every new release.

> with the existing tag.  To use it as a 64-bit unsigned integer violates that.

When the SMI was first published (RFC 1065), we defined NetworkAddress.
Later, we realised that this was a mistake because we couldn't keep
re-issuing the SMI every time we needed a new type of NetworkAddress.
Since then, we've defined things like DisplayString and RowStatus as TC's,
so that they can be put in SYNTAX clauses, and applications can apply
the extra semantics of the TC, even though they are sent on the wire as
OCTET STRING and INTEGER repectively.  Effectively, the underlying
syntax specifies the means of encoding them on the wire, and the SYNTAX
clause specifies the semantics.  That is, calculating a delta is a
function of the SYNTAX in the MIB, not what tag is used on the wire.

So, while you're right that to encode a Unsigned64 as a Counter64 is
a violation of the *letter* of the law, but this is a case where it will
be beneficial to bend the rules a little, for reasons of practicality,
and this won't be the first time that we've bent the rules a little.
(How many times has the Interfaces MIB WG bent the rules ?? :-)

Keith.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Thu, 18 Jun 1998 09:13:21 +0200

>with the existing tag.  To use it as a 64-bit unsigned integer violates that.

Are there examples of such "helpful" applications/APIs that do
things this differently for Unsigned32 and Counter32?

This seems like a good argument - but if there are no existing examples,
it seems to me that we might get away with just declaring the practice evil.

                    Harald

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 18 Jun 1998 10:38:19 +0200

>>>>> Bob Stewart writes:

Bob> I don't see why having a new tag is any more traumatic than
Bob> special-casing an old one.  If anything I'd think a new tag would
Bob> be cleaner and easier to implement.

Bob> Without a new tag smart, helpful, general-purpose applications
Bob> will mishandle 64-bit integers or unsigned, thinking they're
Bob> counters and require calculating a delta.  In other words,
Bob> special counter semantics go with the existing tag.  To use it as
Bob> a 64-bit unsigned integer violates that.

I agree that a new tag would be clean, relatively easy to implement
and avoid ambiguities in cases where the SNMP manager is not capable
to understand or simply does not have access to the relevant MIBs.

Bob> With a new tag we maintain a consistent, protocol-level
Bob> distinction, and in *both* cases it works only between
Bob> consenting, upgraded agents and applications.

The down-side of a new tag is that an agent which implements
Unsigned64 will cause a manager which does not implement Unsigned64 to
get an ASN.1 parse error. In other words: you would not be able to
`walk' the "new" agent with an "existing" SNMPv2c/SNMPv3 manager.
Adding a tagged Unsigned64 causes interoperability problems, even if
this is the solution we really really really like to have.

[The fundamental problem here is that the RFCs do no specify a "gentle
way" to handle unknown tags, e.g. by saying thy should be handled as
opaque values.]
							Juergen

Juergen Schoenwaelder  [email-masked] http://www.cs.tu-bs.de/~schoenw
Technical University Braunschweig, Dept. Operating Systems & Computer Networks
Bueltenweg 74/75, 38106 Braunschweig, Germany.        (Tel. +49 531 / 391 3283)

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 18 Jun 1998 10:40:55 +0200

>>>>> Brian O'Keefe writes:

Brian> PS - will the SMIv3 effort (and mailing list) be announced on
Brian> the snmp or snmpv3 list?

There is no SMIv3 effort as far as I know. There is a design team
which is chartered to bring SMIv2 to full standard status. I received
an announcement of the design team and the rules of the game over the
SNMPv3 mailing list.
							Juergen

---

From: "Brian O'Keefe" <[email-masked]>
Date: Thu, 18 Jun 1998 07:54:58 -0600

> (How many times has the Interfaces MIB WG bent the rules ?? :-)

...and I'm feeling the pains of several of those IF-MIB bends.
It must be coincidence that I was just referring to these IF-MIB
pains in a conversation I had yesterday about this SMI / new data 
type topic.

A number of applications that we ship no longer work because the
semantics of several objects have been *changed*, or objects 
that used to be supported (and adapted) to all interface types
are no longer supported by certain classes of interface types.
But I'll elaborate on the IF-MIB mailing list.

Looking forward and positive...

I would be supportive of a TC approach to get Unsigned64 and
Gauage64 into deployment asap; but I do not support approaches
that break backwards compatibility. 

Overall, I see two alternatives:

   1. a TC approach such as Dave Perkins has been investigating
      w.r.t. wrapping the new data types in OPAQUE; or

   2. adding an explicit new application-tag for Integer64, 
      Unsigned64 (and maybe other forseen important base types,
      e.g., float and/or double-float) to both a SNMPv3 SMI and
      updated SNMP protocol spec. Gauge64 can & should simply be 
      a TC based on Unsigned64.  The SNMPv3 Security work is at 
      proposed, get rfc1905 updated asap and re-submitted as
      proposed so that all the SNMPv3 security mplementations
      underway can roll in the new data types NOW.

bok

---

From: "Brian O'Keefe" <[email-masked]>
Date: Thu, 18 Jun 1998 07:37:03 -0600

> things this differently for Unsigned32 and Counter32?

Yes.  The OpenView Network Node Manager grapher and data collector
for example.  Both look at the data type of the queried object.
Integers, Unsigned32's and Gauge32's are plotted/stored as actual
values.  Counter32's and Counter64's are automatically translated
to rate of change and plotted/stored as such.

Now there is a configuration file that can be used to "coerce"
the data type of specific mib objects.  However, there's
not a plain ole Unsigned64 data type to coerce not-really-counter64
objects to, so the existing installed base would mis-handle all these
not-really-counter64s.  We could add this in a future release,
but that doesn't help the installed base (backward compatibility).

> it seems to me that we might get away with just declaring the practice evil.

I'd say that OpenView NNM is fairly widely deployed.  It has
supported Counter64 for apx 2 years now (circa SNMPv2c intro).

Further, this is *not* an "evil practice".  It is in fact
a "smart, helpful, general-purpose application" that 
releives the end user and other applications from having
to do this on their own.

bok

---

From: "Darren D. Smith" <[email-masked]>
Date: Thu, 18 Jun 1998 08:32:44 -0600

> it seems to me that we might get away with just declaring the practice evil.

Just as an example, HP OpenView NNM has graphing functions that look
at whether it is a counter or a guage, and behave differently.
Currently these functions don't have to look at the MIB syntax
to know what to do.  If Unsigned64 and Gauge64 are implemented
as textual conventions, these routines will have to access the
MIB meta-data to understand and graph a MIB value.  This means that
the MIB modules have to be loaded into our data store, and
numerous other things.  There is no doubt implementation wise
that *for us* doing the "right" thing is to implement the 64-bit
integers the same as the 32 bit integers.

As far as the backwards compatiability issue with new agents 
breaking old managers, this is a true issue.  However, new managers
can typically be rolled into the environment much easier than
rolling all the agents in the environment.  On the other hand, it
could be possible for a manager to also get a parse error from an
old agent, but that seems easier to handle on the management side,
which has to handle potential missing objects anyway.

--
Darren D. Smith 	[email-masked] 	OVSD, HP
--

---

From: "C. M. Heard/VVNET, Inc." <[email-masked]>
Date: Thu, 18 Jun 1998 12:32:12 -0700 (PDT)

[ since I started this thread ... ]

Keith McCloghrie wrote:

kzm> It's 5 years since Counter64 was first published in an RFC, and 2.5
kzm> years since SNMPv2c specified how it could be used with community
kzm> strings.  Yet, there's still lots of systems which do not support it.

That's true.  But is it not also true that the majority of the systems
now deployed support only SNMPv1, which does not support Counter64,
and not the now historic SNMPv2p (with which Counter64 was originally
specified) nor SNMPv2C (which is not on the standards track)?

Brian O'Keefe wrote:

bok> I would be supportive of a TC approach to get Unsigned64 and
bok> Gauage64 into deployment asap; but I do not support approaches
bok> that break backwards compatibility.

I agree.

bok> Overall, I see two alternatives:
bok>
bok>    1. a TC approach such as Dave Perkins has been investigating
bok>       w.r.t. wrapping the new data types in OPAQUE; or
bok>
bok>    2. adding an explicit new application-tag for Integer64,
bok>       Unsigned64 (and maybe other forseen important base types,
bok>       e.g., float and/or double-float) to both a SNMPv3 SMI and
bok>       updated SNMP protocol spec. Gauge64 can & should simply be
bok>       a TC based on Unsigned64.  The SNMPv3 Security work is at
bok>       proposed, get rfc1905 updated asap and re-submitted as
bok>       proposed so that all the SNMPv3 security mplementations
bok>       underway can roll in the new data types NOW.

I would prefer to see alternative 1 -- rehabilitation of the Opaque
type, and wrapping new data types in it -- rather than the use of new
application tags, because it addresses the following points raised by
Juergen Schoenwaelder:

Juergen> The down-side of a new tag is that an agent which implements
Juergen> Unsigned64 will cause a manager which does not implement Unsigned64 to
Juergen> get an ASN.1 parse error. In other words: you would not be able to
Juergen> `walk' the "new" agent with an "existing" SNMPv2c/SNMPv3 manager.
Juergen> Adding a tagged Unsigned64 causes interoperability problems, even if
Juergen> this is the solution we really really really like to have.
Juergen>
Juergen> [The fundamental problem here is that the RFCs do no specify a "gentle
Juergen> way" to handle unknown tags, e.g. by saying thy should be handled as
Juergen> opaque values.]

Double-wrapping new types behind an Opaque tag avoids this, and has the
additional advantage of being usable with SNMPv1.  That is no small
consideration, because SNMPv1 in all likelihood will be with us for years.

In saying this let me hasten to add that carefully specifying what
is allowed in the sequence of octets inside the Opaque wrapper is
an _essential_ part of its rehabilitation [I believe Dave Perkins uses
the word "domestication"].  In particular, one must not only specify
that a valid BER encoding is required, but must also specify which
tags are reserved for future standardization and which (if any) are
fair game for private use.  Dave Perkins has addressed these points
and has made some specific proposals in draft-perkins-opaque-01.txt
(available at http://www.snmpinfo.com/sismiclr.htm).

It is true that domesticating the Opaque type can only be done by
revising the the SMI as it is currently written in a way that is not
strictly backward compatible.  But so would adding a TC that allows
a Gauge64 type to be encoded with a Counter64 tag.  I would expect
that the latter change would be much more likely to cause backward
compatibility problems with existing applications because Counter64
is used in a number standard MIBs, whereas Opaque is not used in any.
Also, it is not difficult to find private enterprise MIBs which use
Counter64, but only once have I seen an Opaque type.

One of the reasons that I started this discussion is that I am working
_right now_ on an (SNMPv1-based) system that has a substantial number
of Gauge64-type objects in its private enterprise MIB.  The approach
I have taken is encode them like they were integers, but tag them as
octet strings.  I have done this rather than double-wrapping them
behind an Opaque tag because the existing undomesticated Opaque type
offers me no advantage over an octet string (and might even have slightly
more risk of causing a management application to fail).  However, if a
domesticated Opaque type offered a standard way to have a Gause64
I'd adopt it in a heartbeat and junk my current octet string-based TC.

By changing the SMI rules to rehabilitate the Opaque type we have an
opportunity to solve once and for all the fundamental problem that
Juergen so eloquently pointed out --

Juergen> [The fundamental problem here is that the RFCs do no specify a "gentle
Juergen> way" to handle unknown tags, e.g. by saying thy should be handled as
Juergen> opaque values.]

There is an identified need today for an Unsigned64 type, and the need for
other base types is likely to arise in the future.  I strongly feel that
addressing these problems should take higher precedence than advancing
SMIv2 to full standard, and indeed, one could argue that their existence
is evidence that such advancement is premature.  I respectfully urge the
ADs to reconsider their position the work of the SMI team "is NOT about
adding new functionality".

Thanks,

Mike
--
C. M. Heard/VVNET, Inc.
[email-masked]

---

From: Keith McCloghrie <[email-masked]>
Date: Fri, 19 Jun 1998 00:42:12 -0700 (PDT)

Brian,

> to rate of change and plotted/stored as such.
 
The snmptcl applications that Marshall and I wrote do the same thing.
However, defining Unsigned64 and Guage64 as TC's based on Counter64
will *not* break these snmptcl applications, because the applications
determine the data type by examining the SYNTAX clause in the MIB, not
by examining the ASN.1 tag on the wire.  I'd be surprised if your
applications were different.

Keith.

---

From: Mark Sylor <[email-masked]>
Date: Sun, 21 Jun 1998 22:23:20 -0400

Our application works much as Keith describes. It distinguishes between
Gauges and Counters based on the SYNTAX in the MIB, not the tag.

I believe Keith's suggestion is equivalent to redefining the meaning of the
Counter64 tag to mean Unsigned64 (the more primitive type), and then having
TCs for Counter64 (it wraps modulo 2^64) and Gauge64 (it latches if it
reaches 2^64-1). That would simplify implementations, and is preferable to
adding yet another tag.

Of course the problems of getting that to standard status need to be
considered.

			Mark

---

From: "Brian O'Keefe" <[email-masked]>
Date: Mon, 22 Jun 1998 07:47:37 -0600

Keith McCloghrie wrote:
> I'd be surprised if your

Surprise.   Our application is based on the notion that
a TC can only refine the base type, not expand it.

>From RFC 1903:

  "When designing a MIB module, it is often useful to define new types
   similar to those defined in the SMI.  In comparison to a type defined
   in the SMI, each of these new types has a different name, a similar
   syntax, but a more precise semantics."

We interpret this to mean that a TC is to its base type,
as a derived class is to its base class in OO programming.
That is, all of the operations you can do to the base class
can also be done with meaningful results to the derived class.

Casting Counter64 into a more general Unsigned64 is NOT making
the semantics more precise.  In fact, it broadens or loosens
the semantics. Gauge64 simply changes the semantics completely.
This means that the "rateOfChange()" operation, which is the 
only sensible operation for interpreting a Counter64, would no 
longer yield meaningful results for the derived class/type.

At what point does "bending the rules" degenerate into
"there are no rules"?

bok

---

From: "Brian O'Keefe" <[email-masked]>
Date: Mon, 22 Jun 1998 08:07:53 -0600

> reaches 2^64-1). That would simplify implementations, and is preferable to

I agree that this is how things SHOULD HAVE BEEN defined from the
beginning;
at at least when reviewing RFC 1443 for advancement as 1902.

However, if I may take your analogy a step further...  how would you
accomplish
such a redefinition?  You'd have to edit rfc 1905 to change the name and 
semantics of the Counter64 application tag to Unsigned64.  This change
requires
publishing a new document.  At what level?  PROPOSED-STANDARD.   Now,
given 
that RFC 1905 references Counter64 as its old semantics, it too would
have to
roll back to PROPOSED-STANDARD.

So where has this gotten us?  Back to square one.

...

Now on a more positive note, if you all wanted to "bend the rules" we
could 
*probably* respond in fairly short order with updates to our next
product 
release and maybe even patches for existing/deployed releases.  However, 
what is the difference between this course and defining a new
application tag for Unsigned64 (leaving Counter64 as is for backwards
compatibility), rolling the Pdu version number, and getting the whole 
thing rolled out in about the same timeframe.  The only difference I see
is
that one is "bending the rules" and one is "playing by the rules".  

bok