Subject: RE: [NGO] NETCONF monitoring schema

From: "Mark Scott" <[email-masked]>
Date: Mon, 19 Nov 2007 13:51:15 -0500

Martin,

Reviving an old email chain here.

I have taken over the monitoring draft originally submitted by Sharon
(Chisholm).  The update is now at:
http://tools.ietf.org/id/draft-scott-netconf-monitoring-00.txt

I did incorporate some of your suggested content below (thank you) but
for now I excluded the portions which I think are too netconf or
implementation specific.  Items such as "inBadHellos" and
"inXMLParseErrors".

Since this schema will be used to monitor more than netconf sessions
(via netconf however) my thinking is we want the schema to be
'sessionType' agnostic (which maps to your 'contextType' below).  

So we could re-incorporate some of further suggestions below, but
perhaps should use more generic terms (ie: "inBadMessages",
"inParseErrors", etc) to ensure they are suitable for all session types.

Perhaps the more detailed stats should be gathered based on sessionType,
and only made available when querying a specific session, or session
type?

Mark

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Tuesday, August 07, 2007 8:20 AM
To: [email-masked]
Subject: Re: [NGO] NETCONF monitoring schema

Hi,

As input to the monitoring schema discussions, here's the statistics
schema we have.

(the format is our DML; you can view it as pseudo-XSD)

/martin

   <elem name="sessions">
      <desc>
	A list of the currently active user sessions in the system.
      </desc>
      <elem name="session" minOccurs="0" maxOccurs="unbounded">
	<elem name="id" type="SessionId" key="true"/>
	<elem name="user" type="xs:string"/>
	<elem name="srcIp" type="confd:inetAddressIP"/>
	<elem name="context" type="contextType"/>
	<elem name="protocol" type="protocolType"/>
	<elem name="login" type="xs:dateTime"/>
	<elem name="locks">
	  <desc>
	    A list of the locks currently held by the session.
	  </desc>
	  <elem name="lock" minOccurs="0" maxOccurs="unbounded">
	    <!-- Use a idx as key instead of the config name; this is
	         to allow for future partial locks. -->
	    <elem name="idx" type="xs:unsignedInt" key="true"/>
	    <elem name="configName" type="configNameType"/>
	  </elem>
	</elem>
      </elem>
    </elem>

   <elem name="statistics">
      <elem name="inSessions" type="xs:unsignedInt">
	<desc>
	  The total number of NETCONF sessions started towards the
	  NETCONF peer.
	  inSessions - inBadHellos = 
              'number of correctly started netconf sessions'
	</desc>
      </elem>
      <elem name="inXMLParseErrs" type="xs:unsignedInt">
	<desc>
	  The total number of messages that were unparsable and thus
	  ignored.  This covers both unparsable 'hello' and 'rpc'
	  messages.
	</desc>
      </elem>
      <elem name="inBadHellos" type="xs:unsignedInt">
	<desc>
	  The total number of sessions silently dropped because an
	  invalid 'hello' message was received.  This includes hello
	  messages with a 'session-id' attribute, bad namespace, and
	  bad capability declarations.
	</desc>
      </elem>
      <elem name="inRpcs" type="xs:unsignedInt">
	<desc>
	  The total number of rpc requests received.
	</desc>
      </elem>
      <elem name="inBadRpcs" type="xs:unsignedInt">
	<desc>
	  The total number of rpcs which were parsed correctly, but
	  couldn't be serviced because they contained non-conformant
XML.
	</desc>
      </elem>
      <elem name="inNotSupportedRpcs" type="xs:unsignedInt">
	<desc>
	  The total number of rpcs which were parsed correctly, but
	  couldn't be serviced because they were not supported by the
	  agent.
	</desc>
      </elem>
      <elem name="outRpcReplies" type="xs:unsignedInt">
	<desc>
	  The total number of 'rpc-reply' messages sent.
	</desc>
      </elem>
      <elem name="outRpcErrors" type="xs:unsignedInt">
	<desc>
	  The total number of 'rpc-reply' messages with 'rpc-error'
	  sent.
	</desc>
      </elem>
      <elem name="outNotification" type="xs:unsignedInt">
	<desc>
	  The total number of 'notification' messages sent.
	</desc>
      </elem>
    </elem>

  <xs:simpleType name="protocolType">
    <xs:restriction base="xs:string">
      <xs:enumeration value="console"/>
      <xs:enumeration value="tcp"/>
      <xs:enumeration value="ssh"/>
      <xs:enumeration value="ssl"/>
    </xs:restriction>
  </xs:simpleType>

  <xs:simpleType name="contextType">
    <xs:restriction base="xs:string">
      <xs:enumeration value="cli"/>
      <xs:enumeration value="netconf"/>
      <xs:enumeration value="webui"/>
    </xs:restriction>
  </xs:simpleType>

_______________________________________________
NGO mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ngo

_______________________________________________
NGO mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ngo

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 20 Nov 2007 12:27:32 +0100 (CET)

Hi,

> implementation specific.  Items such as "inBadHellos" and

If we make a model which is used to montior NETCONF, it seems strange
to remove things which are too NETCONF specific...

*If* we want to do this, we should change the scope of this model to
monitor all northbound management sessions.   But I'm not sure that's
a good idea.

> and only made available when querying a specific session, or session

I think global statistics makes more sense, but Andy also had
session-specific stats.  Here's one way to handle this (in YANG),
using your current definitions:

  container sessions {
    description
      "List of NETCONF sessions currently
       active on this device.";
    list session {
      key sessionId;
      leaf sessionId   { type SessionId; }
      leaf transport   { type transportType; }
      leaf protocol    { type sessionType; }
      leaf username    { type string; }
      leaf src-adreess { type inet:ip-address; }  // handles ipv4 and ipv6
      leaf loginTime   { type yang:date-and-time; }
    }
  }

The we can add:

  // reusable statistics objects
  grouping netconf-stats {
    leaf inXMLParseErrors   { type yang:zero-based-counter32; }
    leaf inRpcs             { type yang:zero-based-counter32; }
    leaf inBadRpcs          { type yang:zero-based-counter32; }
    leaf inNotSupportedRpcs { type yang:zero-based-counter32; }
    leaf outRpcReplies      { type yang:zero-based-counter32; }
    leaf outRpcErrors       { type yang:zero-based-counter32; }
  }

  // augment all netconf sessions with netconf specific counters
  augment /netconfState/sessions/session {
    when "protocol == 'netconf'";
    // common counters
    uses netconf-stats;
  }

  container global-netconf-stats {
    // common counters
    uses netconf-stats;
    // extra global-only counters
    leaf inSessions       { type yang:zero-based-counter32; }
    leaf inBadHellos      { type yang:zero-based-counter32; }
  }

(FYI, if you haven't seen it, I translated your data model into YANG
here: 
http://www.yang-central.org/twiki/bin/view/Main/YangExamplesNetconfStateHtml)

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 20 Nov 2007 12:45:07 +0100

> > 'sessionType' agnostic (which maps to your 'contextType' below).  

Actually, here is what the abstract says:

   This document defines Netconf content via XML Schema to be used to
   monitor the Netconf protocol.  It provides information about Netconf
   sessions and subscriptions.  The proposed schema is intended to
   provide sufficient information to facilitate Netconf session and
   subscription management.  It does not intend to provide detailed
   statistical session information.  Though primarily intended to
   monitor Netconf protocol the schema also provides information for
   non-Netconf sessions.  In particular for session management purposes.

This reads like the focus is clearly on netconf sessions. I am
actually wondering what those "non-netconf sessions" are you want to
monitor. This is nowhere specified or described; at least some
examples should be given and I like to know how I can actually find
out what type of session I have in front of me...

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: "Sharon Chisholm" <[email-masked]>
Date: Tue, 20 Nov 2007 09:16:23 -0500

Hi

This is interesting. I had been thinking mainly about monitoring NETCONF
itself, but Mark raises an interesting requirement. Are there things we
need to know about CLI sessions for example? What if session with the
configuration lock is not NETCONF, but rather CLI? How does this show
up? If we just show NETCONF then it appears that no session has the
lock.

Sharon 

-----Original Message-----
From: Juergen Schoenwaelder
[mailto:[email-masked]] 
Sent: Tuesday, November 20, 2007 6:45 AM
To: Scott, Mark (CAR:2Y00)
Cc: [email-masked]
Subject: Re: [NGO] NETCONF monitoring schema

This reads like the focus is clearly on netconf sessions. I am actually
wondering what those "non-netconf sessions" are you want to monitor.
This is nowhere specified or described; at least some examples should be
given and I like to know how I can actually find out what type of
session I have in front of me...

/js

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 20 Nov 2007 16:11:49 +0100

> up? If we just show NETCONF then it appears that no session has the

If the CLI is layered on top of NETCONF, the problem does not really
exist (since there will be a NETCONF session for the CLI). If there
are things outside NETCONF that can hold NETCONF locks, we can try to
find agreement on all the other things outside that may hold a lock or
we just report that the lock is taken and we are done.

/js

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 20 Nov 2007 16:26:46 +0100

Hello,
I think we must show all locks even if you do not / can not identify the session.

Also you could have a number of entities/interfaces/protocols locking the datastore: Netconf, 
CLI, GUI, SNMP, LDAP, internal locking, backup, reload etc. There are so many options that 
better make this generic. (stealing from Martin)

Username can be internal, as well as a real one.
Protocol can be: internal, GUI, CLI, ...
Maybe add a reason: explicit-lock, backup, reload, etc.

   container sessions {
     description

Balazs

-- 
Balazs Lengyel                       Ericsson Hungary Ltd.
TSP System Manager
ECN: 831 7320                        Fax: +36 1 4377792
Tel: +36-1-437-7320     email: [email-masked]

---

From: "Mark Scott" <[email-masked]>
Date: Tue, 20 Nov 2007 11:46:24 -0500

Hello,

Yes this was my thinking as well - that netconf is only one of a number
of potential protocols accessing/monitoring/etc the datastore.
Additionally I didn't consider a lock to be specifically a 'NETONF lock'
since other interfaces can access, modify, and lock the same data.

The example I eluded to in the abstact was for "session management"
purposes.  For example, if an application (using netconf) needs to
terminate other sessions (perhaps to clear a lock, or to perform some
function requiring exclusive access) then they would require information
on the non-netconf sessions as well.

So I figured better to include these in the NETCONF monitoring as well,
rather than having to potentially use multiple protocols to get a device
wide view of the sessions, lock states, etc.

Mark

-----Original Message-----
From: Balazs Lengyel [mailto:[email-masked]] 
Sent: Tuesday, November 20, 2007 10:27 AM
To: Chisholm, Sharon (CAR:ZZ00)
Cc: [email-masked]; Scott, Mark (CAR:2Y00);
[email-masked]

Hello,
I think we must show all locks even if you do not / can not identify the
session.

Also you could have a number of entities/interfaces/protocols locking
the datastore: Netconf, CLI, GUI, SNMP, LDAP, internal locking, backup,
reload etc. There are so many options that better make this generic.
(stealing from Martin)

   container sessions {
     description
       leaf src-adreess { type inet:ip-address; }  // handles ipv4 and
ipv6

Balazs

Sharon Chisholm wrote:
> Hi
> 
> This is interesting. I had been thinking mainly about monitoring 
> NETCONF itself, but Mark raises an interesting requirement. Are there 
> things we need to know about CLI sessions for example? What if session

> with the configuration lock is not NETCONF, but rather CLI? How does 
> this show up? If we just show NETCONF then it appears that no session 
> has the lock.
> 
> Sharon
> 
monitor.
> This is nowhere specified or described; at least some examples should 
> be given and I like to know how I can actually find out what type of

---

From: Andy Bierman <[email-masked]>
Date: Tue, 20 Nov 2007 10:24:23 -0800

Hi,

I attached the YANG version of the 'sessions' module that I have implemented.
At the end is a cut-and-paste <rpc-reply> for <get> (in NCX conf file format
instead of XML) to provide an example to help understand the data model.

IMO, this stuff is a low priority to standardize.
This is monitoring and the WG charter is configuration.

Andy

module sessions {

  // XXX namespace to be allocated by IANA

  namespace "urn:ietf:params:xml:ns:yang:sessions";

  prefix "ses";

  organization
    "The YANG Gang";

  contact
    "Andy Bierman <[email-masked]>";

  description
    "This module contains objects for monitoring NETCONF
     session statistics and state info.";

  revision 2007-11-20 {
    description 
       "Initial revision, converted from sessions.ncx.";
  }

  import yang-types { prefix "yang"; }

  // Types

  typedef SessionId {
    description "NETCONF Session ID number";
    type uint32 { 
      range "1..max";
  }

  typedef UserName {
    description "NETCONF Session User name string";
    type string { 
      length "1..1023";
      pattern = '[a-z,A-Z][a-z,A-Z,0-9,\-,_]*';
  }

  // Groupings

  grouping SessionStats {
    description 
      "The set of session statistics collected by the agent";

    leaf inBytes {
      description
        "The total number of bytes contained in NETCONF PDUs
         received on a particular session.";
      type yang:zero-based-counter64;
      config false;
    }

    leaf inDropBytes {
      description
        "The total number of bytes dropped (and therefore not received)
         on a particular session.";
      type yang:zero-based-counter64;
      config false;
    }

    leaf inMsgs {
      description
        "The total number of NETCONF PDUs received
         on a particular session without errors.";
      type yang:zero-based-counter64;
      config false;
    }

    leaf inErrMsgs {
      description
        "The total number of NETCONF PDUs received
         on a particular session with errors";
      type yang:zero-based-counter64;
      config false;
    }
        
    leaf outBytes {
      description
        "The total number of bytes contained in NETCONF PDUs
         transmitted on a particular session.";
      type yang:zero-based-counter64;
      config false;
    }

    leaf outMsgs {
      description
        "The total number of NETCONF PDUs transmitted
         on a particular session which did not contain any 
         <rpc-error> elements.";
      type yang:zero-based-counter64;
      config false;
    }

    leaf outErrMsgs {
      description
        "The total number of NETCONF PDUs transmitted
         on a particular session which contained one or more
         <rpc-error> elements.";
      type yang:zero-based-counter64;
      config false;
    }
  }

  // Data objects
       
  container netconfd {
    description
      "The sessionInfo container is implemented as a child
       of the top-level element <netconfd>.";

    container sessionInfo {
      description
        "Read-only Session Statistics and Information";
      config false;
      status current;
      
      leaf activeSessions {
        description
          "Current number of open NETCONF sessions.";
        type yang:zero-based-counter32;
      }

      leaf closedSessions {
        description
          "Total number of closed NETCONF sessions.";
        type yang:zero-based-counter32;
      }

      leaf failedSessions {
        description
          "Total number of failed NETCONF sessions, which
           failed to be established for any reason.";
        type yang:zero-based-counter32;
      }

      container sessionTotals {
        description 
          "Running global statistics totals aggregating
           all open and closed session statistics totals";
        uses SessionStats;
      }

      container sessions {
        description 
          "Container for active session information and statistics.
           There is one entry for each session which is active 
           or starting up.";

        list session {
          description "Per-Session Information Block";

          key "id";

          leaf id { 
            description 
             "The session ID assigned by the agent for the session.";
            type SessionId; 
          }

          leaf startTime {
            description
              "The system time when this session was first opened."
            type yang:date-and-time;
          }

          leaf userName {
            description
              "The user name associated with this session.";             
            type UserName;
          }
  
          leaf state {
            description
              "A textual representation of the current session state.";
            type string;
          }

          container locks {
            description
              "Container for the list of names of the locked 
               configurations";

            // currently implemented as xsd:list instead
            // but YANG does not support the xsd:list type

            leaf-list config {
              description
                "Each <config> leaf contains the name of a 
                 configuration that is currently locked by 
                 this session.";
              type string;
              min-elements 0;
            }
          }

          container stats {
            description
              "The set of statistics collected for this session."
            uses SessionStats;
          }
        }  // end list session
      }    // end container sessions
    }      // end container sessionInfo
  }        // end container netconfd

}
  netconfd {
   sessionInfo {
    activeSessions 3
    closedSessions 0
    failedSessions 0
    sessionTotals {
     inBytes 1405
     inDropBytes 0
     inMsgs 7
     inErrMsgs 0
     outBytes 5665
     outMsgs 3
     outErrMsgs 0
    }
    sessions {
     session 1 {
      startTime 2007-11-20T09:49:25.000-08:00
      userName andy
      state idle
      locks { }
      stats {
       inBytes 425
       inDropBytes 0
       inMsgs 2
       inErrMsgs 0
       outBytes 1264
       outMsgs 1
       outErrMsgs 0
      }
     }
     session 2 {
      startTime 2007-11-20T09:49:46.000-08:00
      userName andy
      state idle
      locks { }
      stats {
       inBytes 425
       inDropBytes 0
       inMsgs 2
       inErrMsgs 0
       outBytes 1264
       outMsgs 1
       outErrMsgs 0
      }
     }
     session 3 {
      startTime 2007-11-20T09:49:54.000-08:00
      userName andy
      state in-msg
      locks { }
      stats {
       inBytes 555
       inDropBytes 0
       inMsgs 3
       inErrMsgs 0
       outBytes 3420
       outMsgs 1
       outErrMsgs 0
      }
     }
    }
   }
  }
_______________________________________________
NGO mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ngo