Subject: Excessive data staleness for ifMIB and interfaces get-responses

From: [email-masked]
Date: Wed, 14 Jul 1999 07:40:56 -500

Hi,

(This is a second attempt - the first did not show
up on the if-mib mailing list.)

I want to bring attention to a get-response data staleness problem
that may have implications for if-mib (and possibly other) MIB
specifications.

This problem has been seen recently for ifTable and ifMIB OIDs in
several router HW/SW bases from the same major vendor.  Data
illustrating the problem is provided at the end of this message.  The
level of data staleness seen in this data breaks with many years of
experience.

In summary, ifTable get-responses can contain data that is sometimes
at least 10 seconds old with respect to the sysUpTime.0 returned in
the same response.  In addition, there is a time-granularity effect on
the order of 20-30 seconds for ifHC counters in the ifMIB.  The exact
syndrome seems to be subject to some variation based on the HW/SW
version and options in use.

One consequence is that, in order to keep relative error effects to
desirable levels, rate computations require counter differences taken
over (unreasonably) long time periods.

For instance, in order to bound the relative error of computed In or
Out Octets/S to no more than +/- 1%, based on 32-bit counter
differences, 1000 second time periods are required.  But this exceeds
the time to wrap 32-bit Octet counters on T3 and higher-speed
interfaces.  Going directly to 64-bit ifMIB HC counters is even worse,
likely requiring 2000-3000 second time periods to insure that relative
error effects are < 1%.

These effects greatly interfere with using SNMP data for supporting a
variety of management, operations and provisioning practices in a
manner that is consistent with past practices.  Even data returned
using 60 second polling periods are very noisy.  Furthermore, special
studies that would use high-frequency SNMP polling (for instance,
using snmpdelta with the default 1-second polling period) are rendered
essentially useless.

Ideally, to meet all of these needs, the time-accuracy of returned
data should be at least as good as sysUpTime.0 (one timetick or 10
mS).  Even a one-second granularity can be a problem, as this would
require 100-second periods for keeping computed rate errors under 1%.
One hundred seconds exceeds the ifOctet counter wrap time on OC12s
and remains a very long time for special study purposes.

These recent SNMP data staleness problems raise some issues that may
reflect on the usefulness of implementations of the interfaces MIB
and the ifMIB, and hence may refect on these (and other) MIBs:

1.  Is there anything technically infeasible about implementing the
    existing ifTable and ifMIB for higher speed interfaces that calls
    into question the usefulness of these MIBS as we go forward?

2.  What does the SNMP standards community expect with respect to data
    accuracy in relationship to the time the get-responses are generated?

3.  Do existing standards provide enough guidance regarding the
    desired/required accuracy of data with respect to the time that
    get-responses are generated?

I couldn't find any text in the RFCs I examined that explicitly sets
data timeliness expectations, or that defines procedures which define
unacceptable data staleness for get-responses.  It is not even clear,
from existing text, that sequential get-responses for the same object
should yield values corresponding to non-decreasing times (although
going backwards in time would seem to violate a generic 'usefulness'
criterion).

Is there any IETF work that directly addresses data staleness in
get-responses, particularly with respect to interface statistics?

Thanks,

Carl Holmberg
GTE Internetworking

####################################################################
#  SAMPLE DATA demonstrating 10-30 second time granularity effects
#  (and hence corresponding minimum staleness) in get-response data.
####################################################################

The data below provides an example of the staleness problem.

The ifInOctet and ifOutOctet counter movements exhibit underlying
8-10-second periodicities.  They seem to have multiple feeds (since
they continue to move every second or so).

The 64-bit ifMIB HC counters are clearly being updated from the 32-bit
counters about once every 20 seconds, with no movement between
updates.  Detailed observations show that when the 64-bit counter
jumps, its lower 32 bit portion is very close to, but may lag slightly
behind, the corresponding 32-bit counter.

The movements of ifTable and corresponding ifMIB HC counter are not
exactly synchronized since the number of Counter32 jumps that are
included in each Counter64 jump can vary.  (See record numbers 65 and
127 for cases where three ifInOctet Counter32 jumps are included in
the corresponding Counter64 jump instead of the typical two jumps.)

This data is not an artifact of measurement tools - various methods
show a similar problem.  These data staleness problems first came to
attention when standard tools began reporting excessive interval
utilizations.

# All observations are for the same ifIndex, same device, taken
# sequentially July 8.  Link capacity is approximately 75M Octets/S,
# although individual interval observations may exceed that figure.
#
# The polling period for the data below averages at about one poll per
# 1.8 seconds.  All related counters for the same interface were
# bundled together in single get-requests, including bracketing
# sysUpTime.0 requests (a few packet counters are omitted from the
# data below).  The responses to the first and last sysUpTime.0
# request never differed by more than 1 timetick.  The RTT (including
# polling host service time, using the polling host's 1-second clock)
# averaged < 1 second, and was never more than 3 seconds.
#
#  LEGEND:       (all differences are for this-record less previous)
#
#         Cnt   - Record sequence number
#        ESec   - (Elapsed target sysUpTime from first record)/100.0
#      DTicks   - Difference in target sysUpTime.0
#    DifInOct   - Difference between ifInOctets
#  DifHCInOct   - Difference between ifHCInOctets
#   DifOutOct   - Difference between ifOutOctets
# DifHCOutOct   - Difference between ifHCOutOctets

# Cnt    ESec DTicks  DifInOct   DifHCInOct  DifOutOct  DifHCOutOct
   2     2.44    244  85437296            0  116558817            0
   3     4.91    247      4982            0      59270            0
   4     7.19    228      3007            0       2837            0
   5     9.29    210      1702            0       2500            0
   6    12.50    321  85035162    259200948  115654171    232498433
   7    14.44    194       687            0        767            0
   8    16.80    236      1752            0       1790            0
   9    19.55    275      2383            0  104709036            0
  10    21.77    222  92029508            0     326483            0
  11    24.26    249      2691            0      49600            0
  12    26.92    266      4010            0       9795            0
  13    29.42    250  84921258            0  121995986            0
  14    31.93    251      4073            0     212681            0
  15    34.38    245      3393    176968357      46253    227351435
  16    37.38    300      3143            0       3795            0
  17    39.32    194  85085457            0  107392416            0
  18    41.80    248      2029            0     220953            0
  19    43.27    147      2436            0       2920            0
  20    45.47    220       529            0        696            0
  21    47.59    212  90503438            0   79928249            0
  22    49.96    237      4355            0   37761942            0
  23    52.48    252      6598    175611041      10993    225324748
  24    54.70    222      6266            0      10202            0
  25    56.39    169  85704443            0   77449148            0
  26    58.78    239      2160            0   36586369            0
  27    61.51    273      4196            0      43960            0
  28    63.40    189      5172            0       1593            0
  29    66.28    288 100311296            0   86589212            0
  30    68.14    186     11644            0      52532            0
  31    70.21    207      8689            0   37110899            0
  32    72.19    198     10195    186063819      10824    237854531
  33    74.10    191     12004            0      12338            0
  34    76.22    212  91630849            0  110869966            0
  35    78.03    181      2291            0     280069            0
  36    80.01    198      2341            0       1744            0
  37    81.81    180      2647            0       3088            0
  38    83.40    159      1848            0       1554            0
  39    86.11    271  99181955            0  117320701            0
  40    88.30    219      7298            0     346207            0
  41    90.23    193      2517            0       2777            0
  42    92.00    177      5855            0       2891            0
  43    93.86    186      1950    190850083   68094828    228841842
  44    95.34    148  97447716            0   36521668            0
  45    96.94    160      1614            0     203957            0
  46    98.84    190       874            0       1505            0
  47   100.43    159       806            0       1441            0
  48   102.01    158      6095            0   76610750            0
  49   103.70    169  96765818            0   35070207            0
  50   105.50    180      2746            0     248562            0
  51   107.54    204      2989            0       3215            0
  52   109.20    166      1411            0       1690            0
  53   111.12    192      2226            0       2441            0
  54   112.99    187  90409626    194234452  121980121    338739866
  55   114.68    169      3153            0     349114            0
  56   116.54    186      3537            0      37359            0
  57   118.24    170      2190            0       3083            0
  58   119.83    159      5461            0       5240            0
  59   121.66    183  94402806            0   79246119            0
  60   123.55    189       815            0   36374863            0
  61   125.34    179      1474            0      47267            0
  62   127.40    206      2733            0       3329            0
  63   128.99    159      3629            0       3745            0
  64   130.59    160       962            0  118431836            0
  65   132.74    215  96177384    281013108     229105    234503743
  66   134.23    149       320            0      37457            0
  67   135.77    154      1098            0       1545            0
  68   137.17    140      4157            0       7551            0
  69   138.66    149       749            0        851            0
  70   140.12    146      1119            0  114943117            0
  71   141.70    158 100325589            0       1343            0
  72   143.31    161      1256            0     322583            0
  73   144.78    147      8288            0       1031            0
  74   146.75    197      2838            0      19236            0
  75   148.41    166      3879            0  115058964            0
  76   149.99    158      1795            0       1938            0
  77   151.56    157  97302340            0     249951            0
  78   153.05    149      1125    197654602       1161    230874042
  79   154.58    153      1448            0       1424            0
  80   155.98    140      1244            0       1492            0
  81   157.48    150      2791            0  111263333            0
  82   158.90    142       825            0        535            0
  83   160.30    140  96558808            0       1667            0
  84   161.86    156       422            0     217464            0
  85   163.42    156     13150            0       4032            0
  86   164.86    144     10219            0      19935            0
  87   166.36    150      2999            0  121332176            0
  88   168.01    165     18147            0      11349            0
  89   169.67    166  93169487            0     191008            0
  90   171.30    163      3246            0      33860            0
  91   173.02    172      1231    189783756       1131    233079045
  92   174.42    140      2017            0       2098            0
  93   175.88    146      2620            0  110537943            0
  94   177.38    150  98115993            0       1181            0
  95   179.14    176      2582            0       2719            0
  96   180.89    175      2700            0     259126            0
  97   182.54    165      3755            0       2828            0
  98   184.29    175      2927            0  122625366            0
  99   186.51    222  86854665            0       6494            0
 100   188.34    183      3105            0      17534            0
 101   190.36    202      2670            0     682148            0
 102   191.93    157      1944            0       2050            0
 103   193.40    147      2813    184995853   36074394    234140370
 104   194.94    154      1985            0   90957206            0
 105   196.35    141  96209791            0        702            0
 106   197.77    142      2388            0     592704            0
 107   199.19    142       890            0        921            0
 108   200.94    175      1756            0       1792            0
 109   202.58    164      2817            0       1879            0
 110   204.09    151      1077            0  123095940            0
 111   206.46    237  83485766            0      39570            0
 112   208.44    198      4602            0     540079            0
 113   209.95    151      5148            0       5532            0
 114   211.50    155      2546            0       2758            0
 115   213.08    158      1875    179722379  118856631    370169677
 116   214.72    164  88197567            0      51563            0
 117   216.31    159      2926            0     830304            0
 118   218.22    191      1020            0       1120            0
 119   219.86    164       789            0        565            0
 120   221.42    156       661            0  113695354            0
 121   223.06    164  87165089            0       2209            0
 122   224.56    150       529            0      43557            0
 123   226.15    159      2336            0     851661            0
 124   227.89    174       696            0        733            0
 125   229.71    182       534            0        871            0
 126   231.63    192  95477966            0  117173867            0
 127   233.44    181      1854    270851302      34286    232652395
 128   235.26    182      6310            0     605218            0
 129   236.89    163      3643            0       3073            0
 130   238.70    181      6263            0       1497            0
 131   240.57    187      4690            0  117734972            0
 132   242.19    162 105011921            0       3777            0
 133   244.01    182      2495            0     620288            0
 134   245.72    171      1791            0       4408            0
 135   247.50    178      2730            0       2219            0
 136   249.33    183      3665            0  115229432            0
 137   251.27    194 106923176            0       7698            0
 138   252.98    171      2478    211970434     604878    234248030
 139   254.46    148       878            0       2490            0
 140   256.24    178      1698            0       2993            0
 141   258.26    202      2351            0   81365388            0
 142   260.22    196      1748            0   36698037            0
 143   262.14    192  97213174            0     519684            0
 144   263.95    181      3730            0       3404            0
 145   265.66    171      2505            0   75034576            0
 146   267.36    170      2188            0       2433            0
 147   269.15    179  89330910            0   33440217            0
 148   271.08    193      5229            0       5857            0
 149   272.86    178      2109    186565192     658239    228330775