Subject: Re: closing Script MIB issues

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 3 Apr 2000 15:57:40 +0200

>>>>> Randy Presuhn writes:

Randy> We need to get closure on the Script MIB issues so we can do a
Randy> WG last call before requesting advancement to Draft Standard.
Randy> Issues from the i-d:

Randy>    02: Restartable scripts

I think the strawman here was to have an smLaunchFlags objects of type
BITS which indicates whether a script is being restarted on
reinitialization.

Randy>    03: Error message during script retrieval or compilation (
Randy> smScriptError)

I think the strawman here was to add an smScriptError object which
contains the last error message generated.

Randy>    04: Scripts that can run forever without having to reset
Randy> smRunLifeTime

I think the strawman here was to give the max. smRunLifeTime value a
special meaning (which stops the timer from decrementing).

Randy>    05: Index smLangTable and smExtsnTable by name rather than
Randy> Integer32

Randy>    06: Dependencies between scripts and script versioning
Randy> (Eamonn)

Randy>    07: Storage type of script code versus storage type of
Randy> script meta information

Randy> All issues on which there are no contributions within the next
Randy> two weeks will be closed with a status of "no action".

I have a few more issues in my list:

   08: Final polling missing in procedures 7.6 and 7.7 (Frank)

I think a suitable clarification should be added to these sections.

   09: Add procedures for suspend/resume/purge to section 7 (Frank)

I think it is helpful to define these procedures.

   10: Any changes needed for editing in the SNMP code table? (Juergen)

There have been a few comments on the smCodeTable and its flexibility
regarding the numbering scheme and so on. Do we want to keep it the
way it is defined? Many implementations so far do not support it and
we may get problems with it anyway once we want to advance to Draft
standard.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

---

From: Eamonn McManus <[email-masked]>
Date: Mon, 03 Apr 2000 16:17:51 +0200

Juergen Schoenwaelder wrote:
> [...]
> [...]

I would like to re-suggest the scheme I described in my Simple Times article,
  http://www.simple-times.org/pub/simple-times/issues/7-2.html#tools

* Entries can only be added to the smCodeTable before the corresponding
smScriptAdminStatus is ``enabled.'' 

* The first entry to be added to the smCodeTable must have smCodeIndex 1, and
every subsequent entry must have smCodeIndex one more than the current largest
value. Each entry must have its smCodeRowStatus set to ``active'' before any
subsequent entry can be added.  Entries once added to the smCodeTable cannot be
deleted. 

These constraints mean that during download the agent does not have to store
smCodeRowStatus or smCodeIndex values anywhere. It does have to remember where
each smCodeText object is within the script, but only while the script is being
pushed. 

* Once the script has been pushed, the manager sets the smScriptAdminStatus to
``enabled.'' As at present, the fact that the code comes from the smCodeTable is
indicated by an empty smScriptSource and an smScriptStorageType that is
``volatile.'' 

* When the smScriptRowStatus is set to ``active,'' an agent may choose to delete
the smCodeTable, or it may reorganize it so that the same code is presented
using a different division into smCodeText objects. In particular it may decide
to chop up the presented text into fixed-length smCodeText objects so that it
does not have to remember the offsets corresponding to smCodeIndex values. This
means that the smCodeTable takes up no storage: the agent can construct replies
to SNMP get requests on the fly. 

If the smScriptStorageType is then made non-volatile, the script text will be
written to non-volatile storage. 

* The only way to modify the text of a script after it has been downloaded is to
delete the smScriptEntry and create a new one. 

Éamonn (speaking in a personal capacity)

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 04 Apr 2000 18:55:45 +0200

I'd like to add another Script MIB issue that has not been
discussed before:

    11: Script resource consumption indication (in the run table)

Scripts running on managed nodes consume resources. If you want
to avoid overloading of a node it is very helpful to get information
about the scripts' resource consumption. You might decide not to run
a script on a certain node, if you find out that it is almost
overloaded. Or you might decide to terminate the biggest consumer of
resources before you start your new script.

Similar to the Host Resources Running Software Performance Group
(hrSWRunPerfTable) in the Host Resources MIB, I suggest to extend
smRunEntry by two more objects:

    smRunPerfCPU    INTEGER
    smRunPerfMem    KBytes

smRunPerfCPU indicates the number of centi-seconds of the total
system's CPU resources consumed by the respective running instance
of the script. smRunPerfMem indicates the total amount of real
system memory allocated to the running instance. 

I see the problem, that this information might not available on all
systems implementing the Script MIB or that it might not be available
for all languages supported by a Script MIB implementation. However,
I consider it very helpful to get this information whenever it is
available. Unavailability could be indicated by value zero.

An alternative would be the introduction of a separate table or
even a separate (and optional?) group as it is the case in the
Host Resource MIB. But I do not like the idea of adding another table.
Implementing the existing tables already seems to be enough work.

    Juergen
-- 
Juergen Quittek    [email-masked]     Tel: +49 30 254230-19
NEC Europe Ltd., C&C Research Laboratories  Fax: +49 30 254230-99
Hardenbergplatz 2, 10623 Berlin, Germany  http://www.ccrle.nec.de

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 5 Apr 2000 13:04:44 +0200

>>>>> Juergen Quittek writes:

Juergen> I'd like to add another Script MIB issue that has not been
Juergen> discussed before:

Juergen>     11: Script resource consumption indication (in the run
Juergen> table)

[...]

Juergen> I see the problem, that this information might not available
Juergen> on all systems implementing the Script MIB or that it might
Juergen> not be available for all languages supported by a Script MIB
Juergen> implementation. However, I consider it very helpful to get
Juergen> this information whenever it is available. Unavailability
Juergen> could be indicated by value zero.

I believe that this is nearly impossible to implement and that the
numbers are hard to interpret on the manager's side. The Jasmin
implementation is capable to execute multiple scripts in one runtime
system (an operating system process) and it is really hard to
calculate how much CPU time is consumed by each thread or how much
memory consumption is caused by each thread. I believe that the only
metric that can be reasonably implemented are resource usage
statistics for the runtime systems but not for each running
script. However, this requires to make the mapping of running scripts
to runtime engines explicit (for examply by adding a column which
contains the PID of the underlying operating system process). Once
you have this mapping, you can actually look into the host resources
MIB to get the actual numbers.

Just to be clear: I certainly understand the need for resource
consumption indicators. However, I do not see how you can define
something which is sufficiently clear so that a manager can actually
base decisions on it and which is at the same time implementable for
arbitrary languages and runtime systems.

Do you think smRunPerfCPU and smRunPerfMem are implementable in the
Jasmin Java runtime without any guesses?

/js

---

From: Juergen Quittek <[email-masked]>
Date: Thu, 06 Apr 2000 13:47:14 +0200

> Jasmin Java runtime without any guesses?

Well, you can, but you do not want to. If you do so, you have to run
each script in its own JVM. This is prohobitive, because the JVM eats
up a lot of resources. You would get the information on used resources
by using significantly more than without measuring the usage.

If scripts share a JVM (as in the current Jasmin implementation), you
can measure smRunPerfCPU. The scripts run as individual threads in the
JVM and if you have kernel level threads, the CPU usage is accessible.
However, since all threads share address space of the same process,
you cannot measure individual usage of memory.

This holds for Java, but if you consider other script languages, such as
Tcl of python, the overhead of the interpreter is much less. For these
languages you can run scripts as individual processes and implement
smRunPerfCPU and smRunPerfMem.

Anyway, I also like the idea of just adding a PID (smRunPID) to the run
table. This is leaner and easier to implement, and it still provides
sufficient means to retrieve the performance information as long as you
also have support of the Host Resources MIB or another suited MIB.

    Juergen

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 6 Apr 2000 14:18:42 +0200

>>>>> Juergen Quittek writes:

Juergen> If scripts share a JVM (as in the current Jasmin
Juergen> implementation), you can measure smRunPerfCPU. The scripts
Juergen> run as individual threads in the JVM and if you have kernel
Juergen> level threads, the CPU usage is accessible. 

Not quite true. Not all systems which have kernel supported threads
treat them similar too processes. (Not all systems are running Linux
yet. ;-)

Juergen> However, since all threads share address space of the same
Juergen> process, you cannot measure individual usage of memory.

Juergen> This holds for Java, but if you consider other script
Juergen> languages, such as Tcl of python, the overhead of the
Juergen> interpreter is much less. For these languages you can run
Juergen> scripts as individual processes and implement smRunPerfCPU
Juergen> and smRunPerfMem.

I think we should not impose this restriction on runtime systems.

Juergen> Anyway, I also like the idea of just adding a PID (smRunPID)
Juergen> to the run table. This is leaner and easier to implement, and
Juergen> it still provides sufficient means to retrieve the
Juergen> performance information as long as you also have support of
Juergen> the Host Resources MIB or another suited MIB.

Fine. What do others think? Making the PID of the runntime system
accessible is IMHO relatively easy to implement (and may actually be
useful for debugging purposes) and it allows us to use the
HOST-RESOURCES-MIB or the APPLICATION-MIB to obtain performance
statistics without having to reinvent the wheel.

/js

---

From: Juergen Quittek <[email-masked]>
Date: Fri, 07 Apr 2000 17:44:29 +0200

I just browsed my note-book an found another Script MIB issue I ran into
some weeks ago:

    12: Setting row status to "active" is missing in procedure 7.5
        between steps 2 and 3.

    Juergen

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 7 Apr 2000 17:50:57 +0200

>>>>> Juergen Quittek writes:

Juergen> I just browsed my note-book an found another Script MIB issue
Juergen> I ran into some weeks ago:

Juergen>     12: Setting row status to "active" is missing in
Juergen> procedure 7.5 between steps 2 and 3.

Thanks. This is now in my issues list.

/js