Subject: snmpconf-pm-04 notes

From: Wes Hardaker <[email-masked]>
Date: 08 Feb 2001 17:08:10 -0800

I haven't quite finished reading it, but I thought I'd jot these down
for as far as I've gotten.  I'll try to mark the more serious issues
(as opposed to editorial ones) with a '***'.

1)  section 4:
    "Policies always express a notion of:
       if (an object has certain characteristics) then (apply operation to
       that object)"

    should be replaced with:

    "Policies always express a notion of:
       if (an object has certain characteristics) then (apply an operation(s) to
       an object)"                                            ^^
       ^^
  Operations can apply to objects other than the one with the
  characteristic in question.  Now, since an action can contain
  multiple operations, the following would probably be the best, IMHO:

    "Policies always express a notion of:
       if (an object has certain characteristics) then (apply a set of
       operation(s) to a set of objects)"

2) section 4:
  roles, whereas the percent utilization of a link would not be.
     ->
  roles, whereas "the percent utilization of a link" would not be.

  (the other two examples in the paragraph are quoted)

3) section 4:

  Roles are human defined strings that can be referenced by policy
     ->
  Roles in this model are human defined strings that can be referenced by policy

4) section 4:
  Repeated use of the word 'download' is used to say that the manager
  downloads policies to the managed devices.  IMHO, this is
  "uploading".

***
5) section 4:
  There are security issues with not uploading policies to a managed
  object until they are discovered.  This issue comes up repeatedly in
  the document.  During the time that the policy is not in place it
  obviously can't be enforced (which could be serious if financial or
  highly secure information was transmitted (EG) during that time
  window).  In theory, you'd think, that this shouldn't happen that
  often and the window would be small.  But what if the
  pmNewRoleNotification isn't received by the management station?

6) 5.1

  therefore they can be modeled as one element type.
   ->
  therefore they can be defined in this model as one element type.

7) 5.2

  filter the last time it was checked (or it was a newly-
   ->
  filter the last time it was checked (or it is a newly-

  and just below it:

  the last check, it will remain in the set of elements that
  will whose policyAction will be run within the
    ->
  the last check, it will remain in the set of elements
  whose policyAction will be run within the

8) 5.3

   accessible from accessor functions, no state is remembered
   from the policy filter evaluation, nor from the previous
   filter/action invocation of this element nor from the previous
   invocation of the policy filter or action). If any syntax or
     ->
   accessible from accessor functions, no state is remembered
   from the policy filter evaluation, nor from the previous
   filter/action invocation of this element. If any syntax or

***
9) 5.3 and others
   If syntax or processing errors occur, the action will terminate
   immediately for this element.  I think failures need to be dealt
   with in a better way.  The document repeatedly references failing
   actions and that processing stops.

   First and most importantly, I think a failure notification needs to
   be sent out (if configured to do so), as there are security
   implications with policies that fail to run properly.

   Second, having a policy die in mid stream is a pretty serious event
   since half of the operations (or so) will have succeeded before the
   error occurred (yet another security consideration).  I'm not
   saying there is a way around this (there isn't), but it should be
   noted somewhere (IMHO).

***
10) I certainly understand the reason for not including function
    definitions, but the code duplication issues resulting from this
    are going to be a pain.

11) section 6: define BNF & EBNF somewhere (or state a reference to
    its definition).

12) section 6:

    "The use of comments and newlines are allowed and encouraged
    where they will promote readability of policy code."
    ^^^^^^^^^^^^^^^

    Oh sure, but I can't make comments with snide remarks in them?
    How about:

    "The use of comments and newlines are allowed and encouraged
    in order to promote readability of policy code."

13) section 8.1:

        Copies the value of b to a. After the copy a and
        b are equal but do not share storage.

    Compilers optimized for storage space could simply make a
    reference until a data request is made...

14) section 8.1:
    "Index (with range checking): a[i]"

    What happens when its out of range?

* (it's almost important)

15) 11.1.1.1
   If getint() is retrieving a unsigned int, I suggest calling it
   getunsigned() or getuint() instead.

16) repeatedly, starting in 11.1.1.1:

     The agent will retrieve the instance in the same SNMP context
     in which the element resides.

   What exactly does this mean?  In the SNMP notion of contexts, an
   item can exist in multiple contexts.

17) It's repeatedly stated that oids are specified as dotted-string of
    numbers, but doesn't say what happens if the oid can't be parsed.
    Action stops, like everything else?

18) 11.1.1.2
    the optional argument "value" should be dropped (as someone else
    pointed out).
     a) C has no optional arguments
     b) is not needed.

***
19) why have both longs and ints if they are the same size?  Just drop
    "long"s, or make them 64 bits and drop "long longs"

********
20) repeatedly:

     Note that no actual SNMP PDU needs to be generated and parsed
     when the policy MIB module resides on the same system as the
     managed elements.

    If this is the case, I can write policy scripts that will operate
    on objects that I don't otherwise have permission to modify?  Is
    there *no* access control checking going on here?  The
    DISMAN-EVENT-MIB handles this by using the authentication
    semantics of the person that performed the set operation putting
    it into existence.  Maybe something similar should be done here?

***
21) 11.1.2
    readVarbind() can return bogus values if nothing was ever written
    to that slot location before.  This goes against everything
    previously in the document where the default action was to stop
    processing for situations like that.  Undefined values are
    probably a very bad thing in a policy script.

-- 
Wes Hardaker
NAI Labs
Network Associates

---

From: Steve Waldbusser <[email-masked]>
Date: Thu, 22 Feb 2001 18:35:40 -0800

Wes,

  Thanks a lot (!) for the extensive comments. I'll try to finish my
reply to your second note soon.

My comments are inline.

>        operation(s) to a set of objects)"

What if it said:
"Policies are intended to express a notion of:"
and then we could use the original pseudo-code, which is designed to
show the intent of the policy-based management model rather than the
limits of what a *could* be done.

I think the "intent" also extends to:
... (apply operation to that object or a related object)
for example: "for every frame relay circuit with xxx characteristics,
apply config xxx to its containing interface/port"

Do you think this should be mentioned in this section?

>   (the other two examples in the paragraph are quoted)

Fixed

>   Roles in this model are human defined strings that can be referenced by policy

Done.

>   downloads policies to the managed devices.  IMHO, this is

Very interesting. You probably say uploading because the code is being
"pushed" from here to there. I say it is downloading because the code is
going from server to client (central to distributed).

I'm not sure I agree that it is "uploading" but it's clear that it's not
clear. I'm thinking using the word "install" instead. What do you think?

>   pmNewRoleNotification isn't received by the management station?

I agree. I'll make some mention of this as appropriate (at least in the
security section). How's this text?:

-- Note that using this algorithm to avoid installing "unnecessary"
-- policies may result in delays in having the policy available when
-- the policy becomes necessary. This delay could become extensive if
-- an interruption of communications prevents the notification from
-- being delivered and/or the policy from being downloaded, causing
-- the sytem to not be in compliance with policy for a period of
-- time. In particular, if the policy is enforcing security rules,
-- this could open up security vulnerabilities during this period of
-- time.

>   whose policyAction will be run within the

Both are fixed.

>    filter/action invocation of this element. If any syntax or

Fixed. 

>    implications with policies that fail to run properly.

For each policy, there's a rollup of the total number of instances of
the policy that are failing now (a gauge, pmPolicyAbnormalTerminations).
Also, when I send an updated draft next week, check out
pmTrackingPolicyToElementInfo which contains info about errors and
exceptions on a per-execution-context basis (i.e. per policy/element
pair). Of course, there's also the debugging table.

So we've made it easy to poll but haven't achieved sub-second
notification (only possible with a notification). And it's here that I'm
a bit reluctant to say that a notification is a requirement. To be
honest, I really don't have a sense of this yet. One more notification 
isn't that big a deal but I'm most worried about getting into ratholes
about how we can make sure that floods of notifications don't occur.

All we would absolutely need would be to have a notification that is
sent whenever the abnormalTerminations gauge goes from zero to non-zero
and no more than 1 notification per 60 seconds (or some similar
non-configurable constant). If we could keep it simple like this it
might be worth it.

> 11) section 6: define BNF & EBNF somewhere (or state a reference to

OK.

>     where they will promote readability of policy code."

Done.

>     reference until a data request is made...

Fixed.

> * (it's almost important)

Yes, I agree it's important. I'll specify.

Check this out in light of the loosely-typed variables in the next
draft.

>    item can exist in multiple contexts.

It's supposed to mean the context in which the element was discovered
(by the pmElementTypeRegTable).

>     Action stops, like everything else?

Yes. I'll say if it isn't of the form below, it is a run-time-exception.

oid:
  integer [ . integer ]

Done.

>     "long"s, or make them 64 bits and drop "long longs"

While I agree, this has been overtaken by events. Check out the next
draft.

>     it into existence.  Maybe something similar should be done here?

Exactly. Access-control checking MUST be done even if no PDU is
generated. I've had a long-standing todo to write the appropriate text.
The plan is to lift as much text as appropriate from the disman event
mib.

This has been changed.

Steve

---

From: Wes Hardaker <[email-masked]>
Date: 23 Feb 2001 10:47:10 -0800

>     if (an object has certain characteristics) then (apply an operation(s) to
Steve> What if it said: "Policies are intended to express a notion
Steve> of:" and then we could use the original pseudo-code, which is
Steve> designed to show the intent of the policy-based management
Steve> model rather than the limits of what a *could* be done.

What I didn't like about the original statement is that it implied to
me that operations always applied to the objects that triggered them,
which is hardly the case (if anything, its probably counter to how
they'd typically be used.  The operation is more likely to apply to a
near by object to the one that triggered the action).  Anyway, you can
do what you want of course.  The notion of applying multiple actions
is less important (IMHO) than the notion of applying the action to a
possibly different object.

>   downloads policies to the managed devices.  IMHO, this is

Steve> Very interesting. You probably say uploading because the code
Steve> is being "pushed" from here to there. I say it is downloading
Steve> because the code is going from server to client (central to
Steve> distributed).

Ahh.  To me it has always implied a direction associated with who is
performing the action.  The m-w.com dictionary defines it as:

   to transfer (information) from a microcomputer to a remote computer
   usually with a modem

which is just horrible out of date and could be interpreted either way
(though I'd argue that is directional).

However,

Steve> I'm not sure I agree that it is "uploading" but it's clear that
Steve> it's not clear. I'm thinking using the word "install"
Steve> instead. What do you think?

That works.

>> pmNewRoleNotification isn't received by the management station?

Steve> I agree. I'll make some mention of this as appropriate (at least in the
Steve> security section). How's this text?:

Steve> -- Note that using this algorithm to avoid installing "unnecessary"
Steve> -- policies may result in delays in having the policy available when
Steve> -- the policy becomes necessary. This delay could become extensive if
Steve> -- an interruption of communications prevents the notification from
Steve> -- being delivered and/or the policy from being downloaded, causing
Steve> -- the sytem to not be in compliance with policy for a period of
Steve> -- time. In particular, if the policy is enforcing security rules,
Steve> -- this could open up security vulnerabilities during this period of
Steve> -- time.

That looks good.  2 comments:

1) please don't put it in the MIB as a comment (see issues discussed
   in my second mail message with respect to putting functionality
   related discussions in MIB comments)

2) download -> installed    ;-)

>    implications with policies that fail to run properly.

Steve> For each policy, there's a rollup of the total number of
Steve> instances of the policy that are failing now (a gauge,
Steve> pmPolicyAbnormalTerminations).  Also, when I send an updated
Steve> draft next week, check out pmTrackingPolicyToElementInfo which
Steve> contains info about errors and exceptions on a
Steve> per-execution-context basis (i.e. per policy/element pair). Of
Steve> course, there's also the debugging table.

Steve> So we've made it easy to poll but haven't achieved sub-second
Steve> notification (only possible with a notification). And it's here
Steve> that I'm a bit reluctant to say that a notification is a
Steve> requirement. To be honest, I really don't have a sense of this
Steve> yet. One more notification isn't that big a deal but I'm most
Steve> worried about getting into ratholes about how we can make sure
Steve> that floods of notifications don't occur.

Steve> All we would absolutely need would be to have a notification
Steve> that is sent whenever the abnormalTerminations gauge goes from
Steve> zero to non-zero and no more than 1 notification per 60 seconds
Steve> (or some similar non-configurable constant). If we could keep
Steve> it simple like this it might be worth it.

IMHO, it is a serious security issue not to enable a way for failed
actions to be run incorrectly and not have a way to inform an
administrator of this immediately.  You're right that the counter
objects exist, but (IMHO) thats not enough.  I don't want to have to
rely on a management station monitoring 50,000 boxes to determine
which ones are failing.  The chances of monitoring it frequently
enough to ensure that my policies are being implemented properly
everywhere is slim.  HP-OV, the last time I ran it, defaulted to
polling objects only every 30 or 60 minutes (and I couldn't decrease
it from that because the polling was taking 20 minutes in the first
place (granted it was a slow box)).  I'd much rather have a trap sent
warning me of such a serious problem. 

The other possibility is to configure backup actions to be used when a
primary action fails.  This is what many other policy related
architectures are trying to do (specifically, the one I'm most
familiar with is the ipsec-policy WG and they're providing a list of
backup actions to take).  In theory, then, you could implement a
backup action that sent a trap.  Though I think making a specific
notification to handle failures is a better way to go.

>    item can exist in multiple contexts.

Steve> It's supposed to mean the context in which the element was discovered
Steve> (by the pmElementTypeRegTable).

It's already been too long since I read it.  I'll have to go rescan it
now, as this one is already hazy in my head.

-- 
Wes Hardaker
NAI Labs
Network Associates

---

From: Steve Waldbusser <[email-masked]>
Date: Fri, 23 Feb 2001 14:13:03 -0800

> possibly different object.

Your statement surprises me but I'm wondering if it's because I've
created another terminology problem. I happen to not mean object in the
SNMP sense (i.e. a variable, sort of) but in an object-oriented sense.
What I should really say is element (I wrote this text before I siezed
on element as the term - "thingie" was in the running for a while too
:-).

This should really be re-written to:
    "Policies are intended to express a notion of:
        if (an element has certain characteristics) then (apply an
operation to that element)"

After fixing my notational blunder, would you agree that an operation is
more likely to apply to the same element? Second most likely to a
related element? And unrelated elements are also possible but don't
really draw as many strengths from the architecture.

> 2) download -> installed    ;-)

Sigh. It seems it will take some time to purge that from my vocabulary.

> notification to handle failures is a better way to go.

Understood. Still thinking. And eager to hear others viewpoints.

Thanks,
Steve

---

From: Wes Hardaker <[email-masked]>
Date: 27 Feb 2001 23:03:00 -0800

>>>>> On Fri, 23 Feb 2001 14:13:03 -0800, Steve Waldbusser <[email-masked]> said:

Steve>     "Policies are intended to express a notion of:
Steve>         if (an element has certain characteristics) then (apply an
Steve>             operation to that element)"

Steve> After fixing my notational blunder, would you agree that an
Steve> operation is more likely to apply to the same element?

Yes.

Steve> Second most likely to a related element?

Yes.

Steve> And unrelated elements are also possible but don't really draw
Steve> as many strengths from the architecture.

Yes, additional and non-related objects are possible.  I'd still argue
"an element" be the right wording in the action clause, but I
understand your point in wanting to point to the default case.

However, this would leave me to believe that this is not appropriate
for a policy, since the objects/elements/whatever are completely
independent (aside from being tied through this policy):

if (packet arrives with bit 12 set) then (blow up a building somewhere)

>> administrator of this immediately.

Steve> Understood. Still thinking. And eager to hear others viewpoints.

Me too (on the view points idea, I'm done thinking about it ;-)

-- 
Wes Hardaker
NAI Labs
Network Associates

---

From: Wes Hardaker <[email-masked]>
Date: 27 Feb 2001 23:11:38 -0800

Throttling is a very very good thing, you're right.  They should be
throttled.  I wouldn't, however, base my notifications on the value 0
for the pmPolicyAbnormalTerminations object.  It implies that it won't
fire another notification till it wraps around to 0 again or until the
object is destroyed and recreated.  I'd say fire if it changes at all,
and don't fire another one till after the throttle factor.

And I disagree that it shouldn't be a configurable throttle value.  A
global notification throttle object (defaulting to, say, 60) should be
defined to deal with this.

(side note: I'd also really like to see multiple actions be available
per policy rule, and ideally fall back actions as well.  Or can a code
table call another code object in the same table? (hence making it
possible to write a function X that calls Y and Z, which could also be
called independently by another action).

-- 
Wes Hardaker
NAI Labs
Network Associates

---

From: Steve Waldbusser <[email-masked]>
Date: Wed, 28 Feb 2001 07:20:53 -0800

> and don't fire another one till after the throttle factor.

Ack. That pesky troublesome abnormalTerminations object again. This has
confused people in the past so I recently rewrote it:

pmPolicyAbnormalTerminations OBJECT-TYPE
    SYNTAX      Gauge32
    UNITS       "elements"
    MAX-ACCESS  read-only
    STATUS      current
    DESCRIPTION
         "The number of elements that, in their most recent filter or
         action execution, have experienced a run-time exception and
         terminated abnormally. Note that if a policy was experiencing
         a run-time exception while processing a particular element
         but on a subsequent invocation it runs normally, this number
         can decline."
    ::= { pmPolicyEntry 12 }

Let's say we have 20 elements. If, when policy P processes those
elements, the first iteration through them 3 of them failed, this object
would be set to 7. If on a later iteration through them policy P failed
on only 1 of them, this object would be set to 1. If on a later
iteration through them policy P succeeded on all of them, this object
would be set to 0. This object is a gauge.

In other words, this should be 0 on most policies. Any nonzero value
indicates a policy that is currently failing on some elements. If you
fix a policy or it "fixes itself", the value will go back to zero.
pmPolicyExecutionErrors is a counter and shows all errors the policy has
ever experienced.

So that's why I'm thinking about the 0 to 1 transition as the noteworthy
one.

> defined to deal with this.

The 0 to 1 transition should substantially limit the number of messages
and then all we need is some reasonable fixed interval to mop up any
issues regarding repetitively bouncing between 0 and 1.

Keep in mind that the MTTR of a broken script is fairly long because you
have to find a programmer, give him a few cokes, and wait. The reason we
want a notification is so that some automated remedial action can occur
instantly (disable the policy, install the last good version, shut down
the device, ...) until the repair is made some time later. So once we're
alerted, we're probably not going to find another notification
"interesting" for hours (until we think it's actually repaired). What
I'm trying to argue is that if we can guarantee an alert on the first 0
to 1 transition your pretty much done. Very little chance of
underreporting the problem. All that's left is to make sure you don't
overreport.

Regarding overreporting, we just need some protection from floods. I say
just pick a number. I realize it should be higher than I originally
said. Maybe 30-60 minutes. Why would you ever set it lower? In the
unlikely case that your programmer has instant response time and fixes
the problem in 10 minutes, why can't we just watch the error counters
for another 50 minutes?

I'm not saying it won't ever be useful, just that most of the time it
isn't useful and I can't see that it's ever necessary. I'm just trying
to avoid feature creep. It's easy to add objects now. It's easy to add
them later. But it's impossible to lower the cost after the fact.

> called independently by another action).

We only have one action. Fall-back actions are available through the use
of precedence:

Let's say I have a normal QOS, a gold QOS, and a videoconference QOS:
Filter                     Action                    Precedence
if(interface)              apply normal QOS          1
if (interface          
  && roleMatch("Gold"))    apply Gold QOS            2
if (interface #27
  && between 2 and 3 PM)   apply Videoconference QOS 3

So all interfaces get at least normal. Anything tagged with "Gold"
gets Gold QOS. If you delete the "Gold" tag, it reverts to normal. No
matter what the previous QOS was, at 2PM interface 27 gets set to
Videoconference. At 3PM interface 27 reverts back to whatever it 
deserves at that time.

Steve

---

From: Wes Hardaker <[email-masked]>
Date: 28 Feb 2001 14:57:10 -0800

>>>>> On Wed, 28 Feb 2001 07:20:53 -0800, Steve Waldbusser <[email-masked]> said:

Steve> pmPolicyAbnormalTerminations OBJECT-TYPE
Steve>          "The number of elements that, in their most recent filter or
Steve>          action execution, have experienced a run-time exception and
Steve>          terminated abnormally. Note that if a policy was experiencing
Steve>          a run-time exception while processing a particular element
Steve>          but on a subsequent invocation it runs normally, this number
Steve>          can decline."
Steve>     ::= { pmPolicyEntry 12 }

Steve> Let's say we have 20 elements. If, when policy P processes
Steve> those elements, the first iteration through them 3 of them
Steve> failed, this object would be set to 7.

Ah, that makes more sense.  Sorry.  (up until you mix 3 and 7, which I
think was a typo).

Steve> What I'm trying to argue is that if we can guarantee an alert
Steve> on the first 0 to 1 transition your pretty much done.

True, though if the code changed I'd be tempted to want another one
but its certainly not necessary.

Now, if that trap goes un-received, though, then the broken script may
not be noticed for a while.  I'd still be tempted to send them out
every X number of seconds (defaulting to 60 minutes, say, as you would
like).  If X is 0, only send one (note that this is a different
configuration item than discussed below where throttling from toggling
is discussed):

Steve> Regarding overreporting, we just need some protection from
Steve> floods. I say just pick a number. I realize it should be higher
Steve> than I originally said. Maybe 30-60 minutes. Why would you ever
Steve> set it lower?

Never underestimate the desire for a sysadmin to want something
different than what the MIB designer figured would be good enough.
IMHO, everything should be configurable unless there is a good reason
to do otherwise and I don't see one here.  The reason for not making
it configurable is that you don't see a value other than yours being
useful (and its doubtful you ever will, seeing as that its your choice)!

Steve> In the unlikely case that your programmer has instant response
Steve> time and fixes the problem in 10 minutes, why can't we just
Steve> watch the error counters for another 50 minutes?

You could, and I'm not saying otherwise.  I'm saying it should be
configurable.  The type of interface I like when managing 10000 nodes
is for them to watch themselves, or nearby nodes, and notify me when
something goes wrong.  I'd rather not run software that continually
monitors something if I don't need the data itself (I'm only looking
for errors not collecting statistics).  I'm likely to fix code and
forget about it (while working on something new) until another error
notification comes my way from the same node again.

Steve> It's easy to add objects now. It's easy to add them later. But
Steve> it's impossible to lower the cost after the fact.

It's also annoying to have written code to support one architecture
that could have been done differently after the later addition.

>> called independently by another action).

Steve> We only have one action. Fall-back actions are available
Steve> through the use of precedence:

Right, I'd like to see more than one "successful" action.  (Take a
look at what we did in the draft-ietf-ipsp-ipsec-conf-mib if it ever
gets published (it hasn't made it through ID processing yet))

Steve> Let's say I have a normal QOS, a gold QOS, and a videoconference QOS:
Steve> Filter                     Action                    Precedence
Steve> if(interface)              apply normal QOS          1
Steve> if (interface          
Steve>   && roleMatch("Gold"))    apply Gold QOS            2
Steve> if (interface #27
Steve>   && between 2 and 3 PM)   apply Videoconference QOS 3

Steve> So all interfaces get at least normal. Anything tagged with
Steve> "Gold" gets Gold QOS. If you delete the "Gold" tag, it reverts
Steve> to normal. No matter what the previous QOS was, at 2PM
Steve> interface 27 gets set to Videoconference. At 3PM interface 27
Steve> reverts back to whatever it deserves at that time.

And if you wanted to do:

                              && encrypt
                              && encrypt

Then, the Videoconference QOS and the Gold QOS code must both
implement the exact same code to provide encryption?  With multiple
actions, you can separate functionality and promote reuse.

-- 
Wes Hardaker
NAI Labs
Network Associates