Subject: Re: [MIB-DOCTORS] Early MIB Dr. Review of draft-ietf-idr-bgp4-mibv2-06.txt

From: "Joan Cucchiara" <[email-masked]>
Date: Fri, 4 Jul 2008 20:52:24 -0400

Hi Jeff,

First want to apologize for the delay in responding.
I have deleted the parts of the email where we have agreed.
See inline below for more discussion.

Thanks,
  Joan

----- Original Message ----- 
From: "Jeffrey Haas" <[email-masked]>
To: "Joan Cucchiara" <[email-masked]>
Cc: "Jeffrey Haas" <[email-masked]>; "Dan Romascanu (E-mail)" 
<[email-masked]>; "David

Ward" <[email-masked]>; <[email-masked]>; "MIB Doctors (E-mail)" 
<[email-masked]>
Sent: Tuesday, June 10, 2008 10:29 PM
Subject: Re: Early MIB Dr. Review of draft-ietf-idr-bgp4-mibv2-06.txt

>>>> E: f(BGP4-MIB), (1228,5) Item "bgpNlriPrefixType" has invalid
>>>> value for MAX-ACCESS
>>
>> This is not listed in the INDEX clause and it has a MAX-ACCESS of
>> "not-accessible".    Is this supposed to be part of the INDEX?  If not,
>> then it should have a different type of MAX-ACCESS.
>
> I've made this read-only and added it to the bgpAfPathAttributesGroup
> OBJECT-GROUP.
>
>>>> W: f(BGP4-MIB), (2689,5) Table "bgpRcvdPathAttrTable" and
>>>> Row "bgpPathAttrEntry" should have same name prefix
>>>
>>> This issue was previously existing for earlier versions of this MIB and
>>> cannot be changed.
>>>
>>
>> I would encourage the working group to fix these sorts of obvious 
>> mistakes
>> rather than carry them forward for a few reasons:
>>
>> 1) A great deal of this MIB is changing already so the working group may
>> want
>> to reconsider fixing this mistake,
>> 2) MIB tools which generate code "might" find errors here and so 
>> developers
>> are left dealing with fixing these sorts of bugs by going in and editing
>> files that are generated (usually not what we want to do).
>> 3)  If the intention is to make this MIB one that is the basis for other
>> BGP
>> MIBs then having a solid foundation to build upon is important IMHO, so
>> again
>> would request that the working group consider this.  Certainly, I will
>> abide
>> by their decision.
>
> This portion of the MIB is widely deployed.  I'm going to recommend
> against changing this.  If you feel strongly otherwise, let's grab Bert
> and/or the relevant ops-nm folk and go offlist for the discussion.

I think it's best to keep these sorts of discussions on
the list since (as you point out) the previous version(s) of
this MIB are widely deployed.  If folks have an opinion
about this issue or other parts of the MIB draft,
this is an opportunity for them to give it.

Speaking more broadly than just this issue for a moment:
I think there is some misunderstandings here with what
typically happens when MIBs
are revised.  Vendors support both (or even several)
versions of a MIB, leaving it to the customer to decide
what they want to deploy.  Sometimes customers have supported
different versions of the same MIB, and when customers
do (eventually) upgrade to the latest and greatest
MIB, this process is done with much testing and usually
involves testing the entirety of the MIB (not just parts that changed).
While there are benefits to keeping parts of the MIB the same,
those benefits are more limited than one might think.

According to http://www.ietf.org/ID-nits.html
  All MIB modules SHOULD have correct SYNTAX, so they should compile cleanly 
using:
     smilint -m -s -l 6 -i namelength-32 <module>

(disregarding only those warnings pertaining to names longer than 32
characters)

Getting back to this specific SMI mistake, you state that the MIB
is widely deployed, but since this is an SMI mistake, and
since many folks probably fix this so that the MIB can
be downloaded and/or compiled by MIB tools, what exactly
is being deployed?  Is it the fixed syntax, or is it this MIB as written?

>
>>>> W: f(BGP4-MIB), (3144,19) Variable "bgpPeerRemoteAddr" in notification
>>>> "bgpEstablishedNotification" is an index for a table
>>>>
>>>> W: f(BGP4-MIB), (3158,19) Variable "bgpPeerRemoteAddr" in notification
>>>> "bgpBackwardTransNotification" is an index for a table
>>>
>>> These issues are the result of the SMIv1 to SMIv2 porting issues that
>>> were largely addressed by RFC 4273.  Consensus at the time was to not
>>> alter the MAX-ACCESS of those objects.
>>>
>>
>> Would again suggest fixing mistakes likes these for the reasons stated
>> above
>> wrt the other error.
>
> Same as above
>
>>> One of the most contentious portions of feedback to prior structure
>>> attempts for this MIB was configuration objects or read-write objects
>>> vs. no-config and read-only.  The working group consensus was that the
>>> complexity of BGP configuration, especially with regards to policy, was
>>> so complicated and vendor-specific that trying to represent it in a MIB
>>> was inappropriate.  The remaining vendor-common functionality, such as
>>> configuring BGP peering sessions, was the MIB would interfere with
>>> required vendor-specific knobs.
>>>
>>
>> Please point me to the relevant emails in the archive.  I would like to
>> understand this because the motivation seems contrary to what the IETF is
>> trying to do
>> with Standards.
>
> Attempting to locate the relevant mails was one of biggest reasons for
> the delay in my latest response.
>
> As best I can tell, most of this discussion took place either in
> hallways discussions with likely implementors or via e-mail that doesn't
> seem to have survived a server transition.  The only idr specific mail I
> can find on this particular topic is:
>
> Date: Wed, 23 Aug 2006 16:30:13 -0400
> From: Jeffrey Haas <[email-masked]>
> [...]
> I've factored the complex functionality out, removed configuration
> objects[...]
>
> Here is my best memory of the concerns from that time:
> 1. Many operators were *very* uncomfortable with the idea that any of
> their BGP configuration could be affected by SNMP, even while protected
> via SNMPv3.  This opinion was based upon feedback garnered from
> inquiries to the NANOG mailing list and hallway conversation during at
> least one NANOG conference.
>

Yes, I agree, but this issue is not just with the BGP MIB.  Many operators
are not comfortable with using SNMP for configuration, and many vendors
provide a way to disable configuring with SNMP.

Additionally, most MIBs being developed by the IETF today have read-only
conformance statements so that the vendors are compliant to the standard
MIB.

> 2. Several likely implementors had issue with the fact that the
> configuration of the BGP peering session given the MIB structure was
> incomplete enough as to represent an obstacle to configuration.  Given
> that the goal of the time to to represent only components of the BGP
> protocol that were going through standardization, the lack of
> configuration for components that were operationally necessary meant
> that the MIB was at best useless and at worst dangerous when being used
> with various extensions.  For example, since the MIB did not support
> configuration of BGP MPLS VPNs, configuration changes via this MIB could
> potentially disrupt the relevant configuration for that kind of network.
>
> This implied that additional vendor-specific enterprise mibs would be
> required to flesh out configuration, even for IETF standard track features
>
> 3. The majority of the configuration represented in this MIB would only
> affect peering session configuration.  Most of BGP is operationally
> based upon "policy".  Policy, while abstractly referred to in the BGP
> base RFC as the Policy Information Base, is highly vendor specific.
> Without policy, it's not very useful to configure BGP via SNMP.  Also,
> given the vendor specific nature of a number of policy elements,
> attempting to standardize policy in SNMP seemed a somewhat futile and
> highly contentious effort.

Granted there are complexities when other MIB modules
are involved, but how does that differ with respect to CLI?  There will
be certain CLI commands issued before others and this has to be carefully
thought out and executed.

The upshot is that I believe this could be done using SNMP and carefully
crafted commands, similar to what is done for the CLI.

>
> If it's still your belief that this MIB should have additional
> configuration objects given the above, let's fork another e-mail thread
> to gather working group consensus.  I would welcome the working group's
> feedback on this topic.

Seeing as this seems to have taken place almost 2 years ago, I think that
writing an email to ask for current opinion would be appropriate.
I would specifically like to hear from any large enterprises that may
be using SNMP for configuring BGP.  So as we discussed, can work on
this email.

>
> [MIB Structure]
>> Yes, but there are other tables, for example timers, which are grouped
>> based on the type of data (i.e. timer) that it is, when some of this is
>> really related to the bgp router information or the bgp peer connection.
>> An analogy might be that some of MIB constructs are approached as
>> object-oriented and some is classic C data structs.   (Modeling a bgpPeer
>> and bgpPeer's connection
>> which is OO, and Timer data which is C.)
>>
>> Sometimes this type of mixing of two styles is not at all a concern, but 
>> I
>> think there is a better organization that could be done with this MIB. 
>> So,
>> for
>> example, a table which contains info on this Bgp's Instance (so some of 
>> the
>> timer objects
>> and other objects  that were  previously configurable would likely be in
>> this table),
>> the current bgpAfPeerTable, and
>> then possibly a third table to contain information regarding the actual
>> session
>> information (i.e. counters which change when the FSM to established).
>
> I've spent some time thinking about this issue and I believe I
> understand your concern at this point.
>
> The bgpPeerAfTable represents the core elements of what define the
> transport end points of a peering session and the session status.  I
> believe this to be inseparable data and wouldn't recommending factoring
> this information into another table.  (Note that I've already
> reorganized the columns in this table to group the indices as you've
> previously suggested.)
>
> With regard to the per-peer timers, I agree that they could be
> potentially better organized.  bgpPeerAfEventTimesTable,
> bgpPeerAfConfiguredTimersTable and bgpPeerAfNegotiatedTimesTable could
> be merged into a single table.  Would this satisfy part of your concern?
> Their layout was somewhat an artifact of previous structure of this MIB,
> some of it unpublished.
>

What I was thinking actually was to not separate Timer objects into
specific "timer" tables, but to group these with their respective categories
(i.e. bgp manager instance, bgp peer, session).

However, I say this assuming that there will be more separation than
currently exists. For example, if the bgpPeerAfTable
is observed, you have comments, Local, Remote, Session Status in one table.
The Local refers to (I think?) what you call the "bgp manager instance",
remote is the "peer" and session status is the "session".
Timers could also be grouped in these categories.

Taking this all a step farther, would suggest to create a
Local (bgp manager instance table) and a remote table (which may or may not
also include session objects).

Does this seem like a reasonable suggestion?  (My 2 cents is that the 
structure
of this MIB has not changed too much since the 1994 MIB.  Yes, objects and
tables have been added, but the original structure was flat and this 
structure
is also flat, one table and having all the other tables AUGMENT from it).

DISCONTINUITY DISCUSSION:

>> I think this is related to the issue of the counters Discontinuity also.
>> Some counters
>> are related to the Peer and some are related to the established session.
>
> Upon further consideration of likely discontinuity events, the only
> major discontinuity event that requires a discontinuity object IMO is
> one that represents when this BGP management instance was last started.
> This would cover all arbitrary abstract indexes that could change
> meaning after a reboot.  Per-peer values that depend on the state of a
> BGP session are already documented as such.  These similarly would reset
> when the BGP management instance restarts.

So is the BGP management instance, refer the "entity" on the router which
starts and maintains the BGP protocol?

>
> Would a single discontinuity object satisfy your concerns?  If so, I
> would propose the following:
>
>    --
>    -- Discontinuity
>    --
>    bgpDiscontinuityTime OBJECT-TYPE
>        SYNTAX TimeStamp
>        MAX-ACCESS read-only
>        STATUS current
>        DESCRIPTION
>            "The value of sysUpTime at the most recent occasion at which
>             this BGP management instance has suffered a discontinuity.
>
>             In particular, tables with abstract indexes such as
>             bgpAfPathAttrIndex, bgpAsPathIndex and
>             bgpAfPathAttrUnknownIndex are not guaranteed to contain the
>             same data across discontinuities."
>         ::= { bgp 13 }
>

If the value of the indices change
and can be different from the objects, then what is the point of
having the objects?

I would like to break this down into a specific discussion
prior to agreeing on one Discontinuity Object (or more than one).

discontinuity scenarios in BGP4 Protocol:

* SNMP agent restarts (sysUpTime is affected) which is default behavior
* bgp entity restarts (what you are calling the BGP Management instance?)
   (assume this is separate from SNMP agent)
* peer restarts/lost connection
* specific session goes from established state to some other state

Are there any other scenarios?

The following (I think?) would be affected by SNMP Agent Restarts, bgp 
Entity
Restarts, peer restarts, and FSM establish state transition.  Is this
correct?

bgpPeerAfInUpdates
bgpPeerAfOutUpdates
bgpPeerAfFsmEstablishedTransitions
bgpPeerAfInTotalMessages
bgpPeerAfOutTotalMessages

[So more of a question for the WG,
Are the bgpPeerAfInTotalMessages and bgpPeerAfOutTotalMessages
counters useful? ]

[As an aside, the bgpPrefixCountersTable should be renamed to
remove the word counter since these are actually Gauge32.]

bgpAfPathAttrCounter - Typically, these values are not
included in MIBs and are calculated by the NMS.  However, if
this needs to be in the MIB, then
think that it probably should be a Gauge32, rather than a
Counter32.  (You could make this a counter, but then would need
to consider this object when a discontinuity occurs and
include that information in the DESCRIPTION).

To quote from RFC2578:

"7.1.6.  Counter32

   The Counter32 type represents a non-negative integer which
   monotonically increases until it reaches a maximum value of 2^32-1
   (4294967295 decimal), when it wraps around and starts increasing
   again from zero.

   Counters have no defined "initial" value, and thus, a single value of
   a Counter has (in general) no information content.  Discontinuities
   in the monotonically increasing value normally occur at re-
   initialization of the management system, and at other times as
   specified in the description of an object-type using this ASN.1 type.
   If such other times can occur, for example, the creation of an object
   instance at times other than re-initialization, then a corresponding
   object should be defined, with an appropriate SYNTAX clause, to
   indicate the last discontinuity.  Examples of appropriate SYNTAX
   clause include:  TimeStamp (a textual convention defined in [3]),
   DateAndTime (another textual convention from [3]) or TimeTicks.

   The value of the MAX-ACCESS clause for objects with a SYNTAX clause
   value of Counter32 is either "read-only" or "accessible-for-notify".

   A DEFVAL clause is not allowed for objects with a SYNTAX clause value
   of Counter32.

7.1.7.  Gauge32

   The Gauge32 type represents a non-negative integer, which may
   increase or decrease, but shall never exceed a maximum value, nor
   fall below a minimum value.  The maximum value can not be greater
   than 2^32-1 (4294967295 decimal), and the minimum value can not be
   smaller than 0.  The value of a Gauge32 has its maximum value
   whenever the information being modeled is greater than or equal to
   its maximum value, and has its minimum value whenever the information
   being modeled is smaller than or equal to its minimum value.  If the
   information being modeled subsequently decreases below (increases
   above) the maximum (minimum) value, the Gauge32 also decreases
   (increases).  (Note that despite of the use of the term "latched" in
   the original definition of this type, it does not become "stuck" at
   its maximum or minimum value.)"

>
>>> I believe operationally that keeping this information in a single table
>>> for walk purposes would be consistent with operator expectations based
>>> on common CLI output.
>>
>> Yes, I agree and would like to hear from operators on this issue also.
>> I am confused when you talk about MIB walks and CLI output.  Although 
>> many
>> vendors do provide a way to dump MIB data to the CLI and page through it,
>> I wouldn't necessarily consider that common CLI output since most of this
>> (at least in my experience) is dictated by the CLI request (i.e. which
>> columns to show).
>
> What I was attempting to say was that operators tend to view this
> operational data via the output of one CLI command.  The MIB structure
> attempts to group data that is commonly available via the CLI together.
>

I don't want to be restricted by how information is displayed on
a GUI or Terminal Screen.  The MIB structure should not be dictated by
this, and since this MIB is very old, what was appropriate years ago,
may not suit the needs today.   Also, CLI commands can be fairly easily
written to display data as wanted across tables or in the same table.

>
>>>> 5.6 Extensions
>>>> -----------------
>>>> How do you plan to enforce using bgpExtensions for
>>>> other MIB Modules?
>>>
>>> Please see draft-jhaas-idr-bgp4-mibv2-community-00 as an example.
>>>
>>
>> Okay, thanks that gives me more information.  I think you should 
>> formalize
>> this via a Standards Action as specified in RFC2434.  Please see the
>> IANA Considerations sections in the MPLS WG (RFC3811-RFC3815) and
>> CCAMP WG (RFC4801-RFC4803) for some examples.
>
> The following text occurred in draft-ietf-idr-bgp4-mibv2-06:
>>
> "9. IANA Considerations
>
>   This document includes an OID, bgpExtensions, which defines a name
>   space for future BGP extensions.  IANA is requested to create a new
>   registry for new OIDs under bgpExtensions that will define the root
>   OID of future MIB modules for bgp extensions.  The assignment OIDs
>   should be done based upon IDR working group consensus."
>
> What further text would you suggest?
>

Since the BGP4-TC-MIB Module needs a branch to go under, I suggest
renaming bgpExtensions to some more generic name like bgpMIB, bgpStdMIB,
bgp4MIB, or bgp4StdMIB).

Specific text (using bgp4MIB and BGP4-TC-MIB as an example):
NB I have also suggested a specific sub-id,  15, since
bgp under transmission is 15 (and the sub-ids under bgp
are at about 12/13).

9.  IANA Considerations

   IANA is requested to make a MIB OID assignment under the bgp
   branch, that is, assign the bgp4MIB under { bgp 15 }.
   This sub-id is requested because 15 is bgp's value under
   the transmission branch and because the next available
   sub-id under bgp is close to 15.

   In the future, BGP4 related standards track MIB modules should be
   rooted under the bgp4MIB subtree.  The IANA is requested to manage
   that namespace.  New assignments can only be made via a Standards
   Action as specified in [RFC2434].

   This document also requests IANA to assign { bgp4MIB 1 } to the
   BGP4-TC-MIB specified in this document.

-Joan

_______________________________________________
MIB-DOCTORS mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mib-doctors

---

From: Jeffrey Haas <[email-masked]>
Date: Sun, 6 Jul 2008 22:33:01 -0400

Joan,

I'm forking this particular thread regarding issues propagated from 
RFC 1657 so we can track it separately.

If someone could get Bert Wijnen and Sharon Chisholm's attention for
this thread it will help drive it to completion quicker.

Please excuse the long quoted sections ahead.

> is being deployed?  Is it the fixed syntax, or is it this MIB as written?

The syntax of the portions of the MIB propagated from RFC 1657 that
generate errors fall into three classes:

 - Violations of variable naming conventions.
 - INDEX objects that are read-only instead of not-accessible.
 - Inclusion of those INDEX objects in Notifications.

Naming convention violations:

As best I am able to tell, while the violation of the naming convention
is not completely at odds with SMIv2 (RFC 2578).  Even smilint considers
this only a warning.

When this issue was raised during the work on RFC 4273, the consensus
within the OPS group review was that it was better to leave these
entries alone since they had been long deployed and renaming them
wouldn't provide sufficient benefit.

Could you please bounce this issue to OPS and request additional
feedback, especially among those involved at the time?  

INDEX MAX-ACCESS issues:

I was similarly recommended to leave this alone.  This fell under the
following from RFC 2578:

:    Objects which are both specified in the INDEX clause of a conceptual
:    row and also columnar objects of the same conceptual row are termed
:    auxiliary objects.  The MAX-ACCESS clause for auxiliary objects is
:    "not-accessible", except in the following circumstances:
: 
: (1)  within a MIB module originally written to conform to SMIv1, and
:      later converted to conform to SMIv2; or

Inclusion of INDEX object in Notifications:

This is similarly a warning and shouldn't result in an on-the-wire
error.  The restriction in SMIv2 (RFC 2578, Section 8.1) is:

: 8.1. Mapping of the OBJECTS clause
: 
: [...]
: 
: An object type specified in this clause must not have an MAX-ACCESS
: clause of "not-accessible".

The notifications generating the error, bgpEstablishNotification and
bgpBackwardTransNotification, were specifically made to address issues
with bgpEstablished and bgpBackwardTransition while providing parity
with their OBJECTS clauses.  

The consensus at the time of last call was that this was not an ideal
"fix" but was the best available one at the time.  

All new objects in the BGP MIBv2 based off the Af tables do the Right
Thing while at the same time deprecating the older objects that are at
issue.

-----

Most of the above was based off of a desire to minimally disrupt a
widely deployed MIB.  Additionally this represents OPS WG consensus 
for dealing with these conversion errors.  If you can get OPS WG
consensus on making your desired changes, I'm happy to put them in the
MIB.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Sun, 6 Jul 2008 22:54:50 -0400

Joan,

Forking this from the main thread for tracking:

> also include session objects).

I think you may be having a small amount of confusion regarding the
objects that constitute the indices of the peer table.

A BGP peer session is defined by a TCP connection between two BGP
speakers.  It is possible for a BGP speaker to have multiple parallel
BGP peering sessions with the same remote BGP speaker so long as the TCP
tuple is different.  See RFC 4271, Section 6.8 for the definition of a
session collision which occurs when two parallel sessions occur with the
same TCP IP address endpoints.

bgpPeerAfInstance serves a somewhat similar purpose as the ENTITY-MIB to
permit multiple BGP sessions.  Since it possible, especially in certain
VPN scenarios, for a given router to permit multiple peering sessions
via differing IP instances, this object was necessary to permit a single
MIB the capability to mange those instances where an end-point might be
the same.

Thus the bgpPeerAfTable's indices are somewhat similar to TcpConnEntry
and should be kept together.  The bgpPeerAfInstance should similarly be
kept with them.

The objects under "Session status" could be factored into an AUGMENTS
table if this will address your concerns.  However, this is the most
factoring I would suggest here.

Overall, I'm not sure what the structural win for this would be.  While
it gains some "purity" of table contents, the similarity to the previous
bgpPeerTable was intentional as an aid to implementors who have to
implement bgpPeerTable.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Sun, 6 Jul 2008 23:24:51 -0400

> and can be different from the objects, then what is the point of

I'm not sure I understand your question.

Fundamentally, the problem is the inability of SMIv2 to represent
complex structured information.  If SMI had retained ASN.1's ability to
have stacked data structures, all of this information would be present
in a single, albeit verbose, table.  This verbosity is notable since BGP
implementations usually implement mechanisms to factor and re-use data
that is present more than once.  For example, while multiple route sets
may all have the same identical set of BGP Communities, it is typically
stored once and reference counted.

Since we are limited to using SMI, the result is that an abstract index
must be used to glue the tables one to another.  This also allows a
given table row to have a many to one relationship to shared data.  E.g.
BgpNlriEntry's bgpAfPathAttrIndex.

>   (assume this is separate from SNMP agent)

It is possible that a BGP instance will share the fate of its SNMP agent
and vice versa but this varies wildly by implementation.

> * specific session goes from established state to some other state

[These are fundamentally the same.]

While this is a discontinuity of sorts, the behavior of this is well
defined by the BGP specifications.  The base behavior is defined by the
FSM in RFC 4271 and potentially altered by Graceful Restart, RFC 4274.

> bgpPeerAfOutTotalMessages

These are affected by SNMP agent restarts and potentially by BGP
Instance restarts.  It is typical for the SNMP agent to fetch these
values from the BGP instance.  While it is possible for the BGP
instance upon restart to maintain their values, this is completely an
implementation detail and may not happen.

> bgpPeerAfFsmEstablishedTransitions

This may be affected by the SNMP agent restarting, the BGP instance
restarting and is altered as a result of BGP peer transitions/restarts.

> Are the bgpPeerAfInTotalMessages and bgpPeerAfOutTotalMessages

Not speaking for the whole of the working group but some operators use
these values as a metric of BGP "chatter".  This is helpful in
pinpointing specific BGP instances and peering sessions that are "close
to" route flapping.

> remove the word counter since these are actually Gauge32.]

I have renamed these to be bgpPrefixGauges*

> Counter32.

I have made this a Gauge32 and renamed it to bgpAfPathAttrGauge.

Feedback over the years on the MIB had stressed on the ability to
monitor the general size of the routing table.  The prefix gauges
provide one level of monitoring this while the Path Attribute gauge
provides another.  These tables are typically *VERY LARGE* and a full
walk simply to generate this count (which is often available from the
management instance as a scalar courtesy of the BGP instance) puts undue
load on the BGP Instance.

-- Jeff

---

From: "Joan Cucchiara" <[email-masked]>
Date: Tue, 15 Jul 2008 09:45:02 -0400

Hi Jeff,

See inline below.

----- Original Message ----- 
From: "Jeffrey Haas" <[email-masked]>
To: "Joan Cucchiara" <[email-masked]>
Cc: "Dan Romascanu (E-mail)" <[email-masked]>; "David Ward" 
<[email-masked]>; <[email-masked]>; "MIB Doctors (E-mail)" 
<[email-masked]>
Sent: Sunday, July 06, 2008 11:24 PM
Subject: BGP MIBv2 discontinuity objects

> BgpNlriEntry's bgpAfPathAttrIndex.

So want to make sure we agree that the value of the bgpAfPathAttrIndex
OBJECT can change if the peer goes away and comes back (or changes from an
Established state to some other state and then back to Established), do you 
agree?

Further, assuming that the object's value changes (due to reasons as stated 
in the previous paragraph),
the value of the bgpAfPathAttrIndex INDEX  does NOT change, do you agree?

So to restate my question, when the value of the object
changes, the object and the index contain different values,
what purpose does this object serve at this point?

In other words, there might be a better way to show a relationship
between these two rows (i.e the row containing the OBJECT and the row using
the OBJECT's initial value as an INDEX), but first I need to understand what 
the
intent is supposed to be because the MIB as written is unclear wrt intent.

> and vice versa but this varies wildly by implementation.

Agreed.

> FSM in RFC 4271 and potentially altered by Graceful Restart, RFC 4274.

If counters can miss information such that they are NOT incrementing when 
they
normally would,  then this is a discontinuity.

> implementation detail and may not happen.

I'm not concerned with whether or not implementations reset counters.
I'm trying to isolate situations where counters miss counting/incrementing.

So, for a "BGP instance" restart,  InUpdates and InTotalMessages are
affected.  So, the peer is sending, but the router is likely missing these 
messages
since it is busy restarting BGP.  Another way to think of this is that the 
related
Out counters on the peer would be incrementing, but these are not able to be
processed by this router as In counts.

> pinpointing specific BGP instances and peering sessions that are "close

Since there are only Counters for UPDATE Messages, could you specify what
is included in the Total Counters?  I am wondering how a developer would 
know
what to include here.  Are the messages counted by this counter
going  to change as more BGP extensions are developed?

If these counters are specific for Route Flapping, then maybe that should be
the focus.

> I have renamed these to be bgpPrefixGauges*

Thanks.

> load on the BGP Instance.

Okay.

   -Joan

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 15 Jul 2008 10:54:55 -0400

Joan,

> Established state to some other state and then back to Established), do you 

Not exactly.  To quote from the MIB:
:              Note well that this index is used to distinguish unique
:              sets of Path Attributes sent as part of BGP NLRI.  The
:              implementor is thus encouraged to make this index unique
:              per set of all received path attributes.  This value may
:              be used to index tables in extension MIBs that share the
:              property of belonging to the same received Path Attribute
:              tuple.

The key words here are "unique sets of Path Attributes".  If, by some
feat of chance you receive that set of path attributes from more than
one peer, it is possible that one peer going down may not cause this set
of path attributes to go away.

Thus, this is not only an index that may have a many to one relationship
from a given set of routes from a given peer, it's possible it may be
many to one for multiple peers for a given set of routes from those
peers.  

This does not preclude an implementation from creating a single entry in
the tables and having a one-to-one mapping with a distinct
bgpAfPathAttrIndex per route.  While such an implementation is possible,
it is not scalable.

> the value of the bgpAfPathAttrIndex INDEX  does NOT change, do you agree?

In order for the contents of the bgpAfPathAttrTable or other tables
indexed by bgpAfPathAttrIndex to change, one of two things would have to
happen:
- All routes sharing these path attributes will have also changed to
  have a new set of shared path attributes.  (Possible, but very
  unlikely.)
- That set of path attributes ceases to be used for long enough for the
  index to be re-used.

In the event of a BGP Instance restart, the indexes *may* (and likely will)
be re-used.  Avoiding this particular scenario is perhaps worthy of some
code but may be very problematic for some implementations due to the
lack of persistance of this index in the data structures.  And again,
BGP graceful restart means a peer may restart but the "resources" allocated
by that peer may stay around.

> what purpose does this object serve at this point?

Simply a pointer to related information in another table.

> the OBJECT's initial value as an INDEX), but first I need to understand 
> what the

I suspect the intent is clear to those who have implemented BGP but
that's clearly not the only audience for this document.  Did the above
explanation help and if so, what changes to these objects would you
recommend?

Please note that while merging the contents of the bgpAfPathAttrTable into
bgpNlriTable would likely make this abundantly clear it is a terribly
unscalable solution for MIB walks.

Throughout the course of your MIB evaluation, please keep in mind that the
typical number of route instances received from a eBGP peer is on the order
of 250,000 routes.  This corresponds to 250K entries in the bgpNlriTable,
some smaller number of entries in the bgpAfPathAttrTable (based on shared
attributes across at least the same peer) and a smaller number of entries in
the bgpAfPathAttrUnknownTable.

This is per-*peer*.  Many BGP routers have hundreds of peers.

This means many of the structural design choices in this MIB are meant to
accomodate the required scalability.

> normally would,  then this is a discontinuity.

But not one expected to affect counters.

> affected.

*May* be affected.  This depends on implementation.

> So, the peer is sending, but the router is likely missing these 
> messages

Sure.  Dropped packets happen.  It's a matter of course that a user of TCP
may not be aware that some data has not been received even though it has
sent it.  If this was a matter of concern some other protocol than TCP would
be used.

(As a matter of fact, this is one of the classes of concern that lead to the
BFD protocol.  TCP liveness is not good enough sometimes.)

> going  to change as more BGP extensions are developed?

    bgpPeerInTotalMessages OBJECT-TYPE
        SYNTAX     Counter32
        MAX-ACCESS read-only
        STATUS     deprecated
        DESCRIPTION
                "The total number of messages received
                 from the remote peer on this connection.

                 This object has been replaced with
                 bgpPeerAfInTotalMessages."
        REFERENCE
                "RFC 4271, Section 4."
        ::= { bgpPeerEntry 12 }

I see that the bgpPeerAf* versions of these counters do not contain the
REFERENCE section.  I'll correct that.

In the context of the reference, the meaning of message is clear.  Yes, as
new extensions are defined they are counted since they are in the format of
a BGP message.  Consider RFC 2918 - a new message type.  It's still a
message.

> If these counters are specific for Route Flapping, then maybe that should 

That would be the focus of another MIB - perhaps one covering RFC 2439.
(Although realistically not.  RFC 2439 RFD is not what is deployed.)

-- Jeff

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 16 Jul 2008 11:27:06 -0700

Hi -

> Cc: "David Ward" <[email-masked]>; "MIB Doctors (E-mail)" <[email-masked]>; <[email-masked]>
> Sent: Sunday, July 06, 2008 7:33 PM
> Subject: [MIB-DOCTORS] RFC 1657 MIB errors/corrections
...
> consensus on making your desired changes, I'm happy to put them in the
...

In case the silence hasn't made it obvious, the resons you give for *not*
making those changes are all good ones.  The costs of making those
changes would outweigh any conceivable benefit.

Randy

---

From: "Joan Cucchiara" <[email-masked]>
Date: Wed, 16 Jul 2008 15:27:30 -0400

Jeff and I talked on the phone about this one, sorry but there was
some misunderstandings which were best clarified by an actual conversation.

I'll try to recap as best as I can, and Jeff can correct me if I'm wrong, 
but
the upshot is the bgpAfPathAttrIndex OBJECT (contained in the bgpNlriTable)
and the bgpAfPathAttrIndex which serves as a single index for 
bgpAfPathAttrTable
should originally contain the same value to show a relationship between
the row in the bgpNlriTable and the bgpAfPathAttrTable.  After a 
discontinuity
the value of the bgpAfPathAttrIndex (used as an index in the 
bgpAfPathAttrTable) does
not change, BUT the bgpAfIndex OBJECT in the bgpNlriTable could change.

For those implementations where the bgpAfPathAttrIndex OBJECT in the 
bgpNlriTable
changes after a discontinuity, the relationship
between the NLRI  (Network Layer Reachability Information)
and the related Path Attribute information is lost.
This means that the bgpAfPathAttrIndex OBJECT's
new value should not be the same value as any bgpAfPathAttrIndex in the 
bgpAfPathAttrTable, so
if the intention is that the NLRI and the Path Attributes are not related, 
checking needs to be
done.   This will be stated more clearly.

Additionally, if implementations want to keep the relationship between the 
NLRI and the
Path Attributes, then implementations should  "save" the value of
the bgpAfPathAttrIndex OBJECT with NLRI information (this is not saved in 
the MIB but in
some data structure) and in the event that
the same NLRI comes back, the original value of the bgpAfPathAttrIndex 
OBJECT can be used
so that the relationship between the NLRI and bgpAfPathAttrEntry is 
preserved.

   -Joan

----- Original Message ----- 
From: "Jeffrey Haas" <[email-masked]>
To: "Joan Cucchiara" <[email-masked]>
Cc: "Jeffrey Haas" <[email-masked]>; "Dan Romascanu (E-mail)" 
<[email-masked]>; "David Ward" <[email-masked]>; <[email-masked]>; "MIB 
Doctors (E-mail)" <[email-masked]>
Sent: Tuesday, July 15, 2008 10:54 AM
Subject: Re: BGP MIBv2 discontinuity objects

---

From: "David B Harrington" <[email-masked]>
Date: Wed, 16 Jul 2008 16:20:31 -0400

Hi,

In response to Randy's comment about what the silence means, I would
like to say that I am too busy at this time to research the issues
involved, and have not commented for that reason, not because I think
a stated position is correct or not.

In general, redesigning a MIB is bad because it hurts
interoperability. But some MIB modules deserve to be redesigned.

I have some concerns if different implementations do not represent
things in a consistent manner, and reflect the changing conditions in
a consistent manner. I am concerned that the design of these tables
and a lack of standardized behavior at discontinuity might lead to
misinformation being presented to an operator. "Implementations vary
widely" raises a red flag for me about any supposed **standard**.

I also get concerned when a MIB designer talks about the limitations
of SMI at using complex modeling stsructures. MIB modules are
effectively relational database models, and in my experience
relational database models typically do not use deeply nested
structures. And attempts to force SMI to represent deeply nested
structures usually result in poor MIB designs. "If only SMI supported
ASN.1 deeply nested structures" raises a red flag for me.

But I do not have time to study these tables now to determine whether
these are valid concerns.

Please do not assume my silence means consent.

dbh

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Joan Cucchiara
> Sent: Wednesday, July 16, 2008 3:28 PM
> To: Jeffrey Haas
> Cc: Jeffrey Haas; David Ward; MIB Doctors (E-mail); [email-masked]
> Subject: Re: [MIB-DOCTORS] BGP MIBv2 discontinuity objects
> 
> 

> >> OBJECT can change if the peer goes away and comes back (or 
> changes from 
> >> an
> >> Established state to some other state and then back to 
> Established), do 
> >> you
> >> agree?
> >
> >> Further, assuming that the object's value changes (due to 
> reasons as 
> >> stated
> > In the event of a BGP Instance restart, the indexes *may* 
> (and likely 
> > will)
> >> between these two rows (i.e the row containing the OBJECT 
> and the row 
> >> using
> >> intent is supposed to be because the MIB as written is unclear
wrt 
> >> intent.
> > typical number of route instances received from a eBGP peer 
> is on the 
> > order
> > attributes across at least the same peer) and a smaller 
> number of entries 
> > in
> >> since it is busy restarting BGP.  Another way to think of 
> this is that 
> >> the
> >> related
> >> Out counters on the peer would be incrementing, but these 
> are not able to
> >> be
> > sent it.  If this was a matter of concern some other 
> protocol than TCP 
> > would
> > be used.
> >
> > (As a matter of fact, this is one of the classes of concern 
> that lead to 
> > the
> > new extensions are defined they are counted since they are 
> in the format 
> > of
> >> If these counters are specific for Route Flapping, then 
> maybe that should
> >> be
> >> the focus.
> >

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 16 Jul 2008 13:31:11 -0700

Hi -

> From: "David B Harrington" <[email-masked]>
> To: "'Joan Cucchiara'" <[email-masked]>; "'Jeffrey Haas'" <[email-masked]>
> Cc: "'Jeffrey Haas'" <[email-masked]>; "'MIB Doctors (E-mail)'" <[email-masked]>; "'David Ward'" <[email-masked]>;
<[email-masked]>
> Sent: Wednesday, July 16, 2008 1:20 PM
...
...

PLEASE note that my comment was on the thread entitled
"RFC 1657 MIB errors/corrections", NOT this one!

I've stayed away from this thread for the same reasons as DavidH
has.  My sense from the conversation so far is that the design is
a bit baroque, but there are serious motivations for it being the way it is.

If a design works and has been deployed, we shouldn't be talking about
changing it unless there is an alternative proposal that would provide
overwhelming operational benefits.

Randy

---

From: "David B Harrington" <[email-masked]>
Date: Wed, 16 Jul 2008 16:34:21 -0400

sorry if I mixed threads.

dbh 

> [mailto:[email-masked]] On Behalf Of Randy Presuhn
> Sent: Wednesday, July 16, 2008 4:31 PM
> Cc: 'MIB Doctors (E-mail)'; [email-masked]
> > Cc: "'Jeffrey Haas'" <[email-masked]>; "'MIB Doctors 
> (E-mail)'" <[email-masked]>; "'David Ward'" <[email-masked]>;
> <[email-masked]>

---

From: "David B Harrington" <[email-masked]>
Date: Wed, 16 Jul 2008 16:38:42 -0400

Can somebodsy tell me the URL for the MIB Doctors archive?

dbh

---

From: "David B Harrington" <[email-masked]>
Date: Wed, 16 Jul 2008 19:10:36 -0400

Hi Jeff, 

> 1. If you interpret this response as "snippy", please don't take
> personal offense. 

I never take personal offense when a response seems "snippy" after I
have told an author "maybe your design just sucks" ;-)

[..]
> We're not here to tell people that the MIB is meant to constrain how
> they write their code.  We're here to say that the MIB models the
data
> in a clear and operationally useful fashion.  

We're here to say that the MIB models the data in a clear and
operationally useful ***and standardized*** fashion.

Standards should not be written to make sure that any implementation
can be declared compliant. 
These are implementation decisions, not implementation issues.
A standard MIB should standardize the representation of the underlying
state.
It is then an implementation decision whether to be compliant to the
standard or not.

> I'd be lax in 
> my duties if
> I didn't point out where the model and reality might have problems
> meshing.

My concern is that you may be lax in your duties if you do not define
an interoperable, standardized representation for this situation.

[...]
> >  And attempts to force SMI to represent deeply nested
> Excellent.  You obviously have an opinion.  
OK. Now you're getting "snippy" ;-)

> If you don't have time to
> study the whole MIB, at least study the bgpAfPathAttrTable and its
> relationsip to the bgpNlriTable.
> 
I spent an hour looking over these two tables. Without a knowledge of
BGP, that study was useless.
I do not have time at this point to do the research necessary to
understand the BGP relationships and the modeling relationships.

I did spot one thing I would recommend changing. You have objects that
"will be absent" under certain conditions, such as the preference
objects. I think it better to implement the object with special values
for "no preference indicated"; then an NMS knows whether there was no
preference indicated, or the implementer simply chose not to implement
the object. In addition, having "holes" in tables makes MIB Walk
presentation more complex and easier for operators to misconstrue.

> Over the past 5 years of working on this MIB, I've received
> contradictory advice from several "mib experts".
> 
> 1. Flatten the tables.  This removes the "deeply nested" 
> concern.  This
> makes a MIB walk full of redundant information.
> 2. Factor the tables.  This removes redundant information at 
> the risk of
> introducing relational integrity and discontinuity issues.
> 
> Given that BGP implementations have data structures that are
factored
> and that size of MIB walks was a concern I was repeatedly 
> presented with, I
> chose 2.  This was after a lot of hallway discussion over 
> several IETFs.
> 
> If this is a problem, *speak now*.  

As I said, I do not have time for a detailed review of these issues
now. 
I will just point out the red flags that I spotted in the discussions.
A big part of what the MIB Doctors do is raise concerns so the ADs
(and other MIB Doctors) can consider those concerns.
If you have considered these concerns and reached a suitable
engineering tradeoff, great.
If you can convince the ADs that you have done due diligence in
considering these issues, great.
My goal is to help ensure the ADs consider whether you considered
these issues.

> Given the amount of churn 
> over both
> views over the last several years, if we can't even get 
> consensus among
> the MIB doctors 

Engineering is all about tradeoffs. I am not surprised when there is
not a clear consensus among MIB Doctors or any other group of
"experts" on a particular design.

The MIB Doctors have talked about, but never produced, a "MIB Design
Guidelines" document.

> we'd be best off driving a stake into this spec and
> handing the resultant dust to Sue and Yakov for careful burial as a
> warning to the future to not even bother trying.

Ah, now you're getting snippy again ;-)

---

From: "David B Harrington" <[email-masked]>
Date: Wed, 16 Jul 2008 20:58:57 -0400

> The problem is the protocol permits the full range of values for an
> Integer32 to be used.  I will revert to a previous strategy of using
a
> related TruthValue object to document whether the object contains a
> useful value.

Ah, I womndered if all the values were useable.
Let's ask the other MIB Doctors what they think about this; which is
the better approach? absent objects or two objects, one of which says
"that other object is meaningless"?

> I believe you'd find a lot of support for this effort.

But would we ever reach MIB Doctor consensus on the guidelines?
Would they just be a bunch of CLRs?
Should we just point to Dave Perkins' "SNMP MIBs" book?

> Thanks for your input.

my pleasure.
dbh

---

From: "Randy Presuhn" <[email-masked]>
Date: Wed, 16 Jul 2008 19:48:08 -0700

Hi -

> To: "'Jeffrey Haas'" <[email-masked]>
> Cc: "'MIB Doctors (E-mail)'" <[email-masked]>; "'David Ward'" <[email-masked]>; <[email-masked]>
> Sent: Wednesday, July 16, 2008 5:58 PM
...
...

It may be a merely hypothetical concern, but "absent" objects are
indistinguishable from "excluded by access control" during a get
or a mib walk.  It's nice to have a sentinel value for the purpose,
but sometimes, as apparently in this case, it's not possible.

Randy

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 16 Jul 2008 17:13:37 -0400

David,

1. If you interpret this response as "snippy", please don't take
personal offense. 
2.  This is  exactly the sort of feedback that this MIB has
been requiring.

> widely" raises a red flag for me about any supposed **standard**.

Just like most IETF protocols, BGP-4's specifications are largely about
on-the-wire behavior and abstract data modeling.  The MIB is there to
give people a canonical way to get at operational state and the abstract
databases.

The IETF isn't here to tell you how to write your code.

If your implementation doesn't have persistent storage, you're not going
to have a place to store the last-used INDEX for a table in the case of
a discontinuity.  That's an implementation issue.

If your implementation chooses to load your path attribute set from a
local cache before you even bring up a single BGP peering session and
maintain consistent indices across reboots, that's an implementation
issue.

We're not here to tell people that the MIB is meant to constrain how
they write their code.  We're here to say that the MIB models the data
in a clear and operationally useful fashion.  I'd be lax in my duties if
I didn't point out where the model and reality might have problems
meshing.

> structures.

But BGP does.  Honestly, most TLV based protocols do.  Perhaps the only
novel thing about BGP (and probably not even novel) is that path
attributes are shared amongst prefixes.  And thus have a one-to-many
relationship.

> ASN.1 deeply nested structures" raises a red flag for me.

Excellent.  You obviously have an opinion.  If you don't have time to
study the whole MIB, at least study the bgpAfPathAttrTable and its
relationsip to the bgpNlriTable.

1. Flatten the tables.  This removes the "deeply nested" concern.  This
makes a MIB walk full of redundant information.
2. Factor the tables.  This removes redundant information at the risk of
introducing relational integrity and discontinuity issues.

Given that BGP implementations have data structures that are factored
and that size of MIB walks was a concern I was repeatedly presented with, I
chose 2.  This was after a lot of hallway discussion over several IETFs.

If this is a problem, *speak now*.  Given the amount of churn over both
views over the last several years, if we can't even get consensus among
the MIB doctors we'd be best off driving a stake into this spec and
handing the resultant dust to Sue and Yakov for careful burial as a
warning to the future to not even bother trying.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 16 Jul 2008 20:00:59 -0400

> have told an author "maybe your design just sucks" ;-)

Constructive criticism is good, even when it hurts.  It's better than a
few unnecessary trips on the I-D-go-round.

> operationally useful ***and standardized*** fashion.

In the case of the Path Attributes tables and the indexing, I don't have
great concerns about standardization.  The language clearly needs to be
expanded so that it's clear what it's for and the issues you can run
into for both usage and implementation.

> an interoperable, standardized representation for this situation.

I'd like to think I have enough IDR cred at this point for people to
know that I'm a big fan of clear, interoperable standards. :-)

> understand the BGP relationships and the modeling relationships.

Fair enough.  The general issue is that the data represents a
many-to-one relationship of prefixes to path attributes; more than one
prefix can point to the same path attribute row.  From an SNMP modeling
perspective, it's irrelevant whether a set of path attributes are
present once or multiple times within the path attribute table.  This
means it is possible to model the prefix row to path attribute row
relationship as one to one if that is the implementor's inclination.

It is even possible for a path attribute row to exist without being
referred to by a prefix.

Documenting this is clearly the challenge.

The problem is the protocol permits the full range of values for an
Integer32 to be used.  I will revert to a previous strategy of using a
related TruthValue object to document whether the object contains a
useful value.

> engineering tradeoff, great.

It's my belief that I've achieved rough consensus and Joan (and
previously Sharon) has been great about ironing out issues that would be
of concern within the OPS WG.  Once we're over the remaining bumps,
we'll hopefully be to running code.

Thanks for your input.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 16 Jul 2008 21:37:16 -0400

Forked to just mib-doctors.

> Should we just point to Dave Perkins' "SNMP MIBs" book?

During my initial SNMP education, I found the Perkins book to be a useful
overview of SNMP and to a lesser extent MIB design.  It was very useful
from a "reading MIBs" standpoint.

The things I could have used help on in my earlier days included some of
the following:
- The MIB review guidelines used by the OPS group make a good reading
  list of things you should know before you start.  Some of these things
  are accepted conventions that are almost more tradition than words in
  an RFC.
- Here's a list of common MIBs and textual conventions - places to go so
  you're not reinventing the wheel.
- SNMP persistence and configuration.  What a RowStatus and StorageType
  are and why your head shouldn't hurt when trying to understand them
  from the RFC.
- Common methods of representing things in a protocol:
  - Global configuration
  - Per-session configuration
  - Protocol/protocol instance state machines
  - Protocol data
    - Mapping 1:1, 1:Many, Many:1.
    - Mapping tables to go from one index set to another table.
    - Relational integrity when dealing with tables with abstract
      indices.
  - Pointers to example MIBs that implement each of these.

Much of the above can be found from repeated deep readings of RFCs.  I
dare say after a certain point you *must* do that reading.  However,
reading those RFCs as your initial introduction to writing MIBs within
the IETF is intimidating at best, even for protocol engineers.

In any case, thanks for the help the mib-doctors has been able to
provide. 

-- Jeff

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Thu, 17 Jul 2008 15:21:02 +0200

http://www.ietf.org/mail-archive/web/mib-doctors/current/maillist.html

Dan

> [mailto:[email-masked]] On Behalf Of David B Harrington
> Sent: Wednesday, July 16, 2008 11:39 PM
> To: 'MIB Doctors (E-mail)'
> Subject: [MIB-DOCTORS] MIB Doctors archive
>