Subject: [ippm] draft ietf58 ippm minutes

From: "Henk Uijterwaal (RIPE-NCC)" <[email-masked]>
Date: Thu, 18 Dec 2003 11:51:07 +0100 (CET)

Comments to Matt before Friday 19, 17:00h GMT please.

Henk

-=-=-
IETF IP Performance Metrics WG (ippm)
Thursday, November 13, 2003 at 13:00 to 15:00
=============================================

The meeting was moderated by the working group chairs, Matt Zekauskas
and Henk Uijterwaal.  Al Morton and Ruediger Geib took notes, which
were edited into these minutes by the chairs.

AGENDA
1. Agenda Bashing
2. Packet Reordering
3. Reordering Density
4. IPPM-MIB
5. OWAMP
6. Status, Milestones & Futures

1. Agenda Bashing
   -- The chairs

There was no bashing, but Matt welcomed Henk Uijterwaal as the new
co-chair, replacing Merike Kaeo who is now focusing on security work.

2. Packet Reordering
   -- Al Morton

Al Morton presented the changes since the last meeting to the
reordering draft (see slides).  The major changes are (1) continued
clarifications started in the last version, including changing the
sense of the base metric so it reports "true" when a packet has been
reordered, moving alternate methods of detecting reordering (time or
bytes instead of sequence number) to a separate section, (2) adding
the concept of steady-state versus transient reordering, and (3)
adding text to cover packet fragmentation.  The fragmentation text
needs work, and will likely have the details moved to a separate
section.  The draft is still missing an appendix on applicability
promised by Greg Wright.  Matt asked which appendices, if any,
were going to be normative (and we should so state).

Jerry Perser asked if we could drop IPPM from the title, since in
his view it was more general (that is, there was some applicability
to BMWG as well).  There was no objection from the assembled group.

Matt (chair hat off) asked if adding fragmentation added too much
complexity; none of the other metrics look at individual fragments.
Jerry felt that either we should specify the metric so fragmentation
does not occur, or we should check whether fragments are reordered.
Al Morton noted that he has seen systems "in the wild" that don't
handle reordered fragments well.  Stanislav Shalunov noted that some
versions of Linux actually send fragments in the reverse order because
they get a slight performance improvement during reassembly; a note
should be added.

3. Reordering Density
   -- Anura Jayasumana

The next presentation was by Anura Jayasumana on reordering density (an
individual submission).  This is the second presentation we had on this
topic, but the first by an author; there was a presentation at the last
IETF meeting by Jerry McCollum from Agilent, who is familiar with the work.
One can think of the reorder density function like a normalized histogram
of buffer occupancy.

The major problem that the group has with this work is that lost
packets show up in the reorder density, so you cannot completely
separate losses from reordering events using this technique.

An example showing a lost packet was given.  Stanislav Shalunov noted
that it appears that loss is counted as reordering.  Anura stated that
it was correct.  Stanislav asked if a sequence had no reordering but
just loss would the result be non-zero?  Anura said that was correct,
that there is a problem with packet loss, the current definition was a
compromise, and ensured the computation could be done in order n time
and constant space.

One interesting claim made by the author is that they have been able
to characterize the nature of loss based on distributions of reorder
density. It could also be a basis for TCP adjusting it's sensitivity
to reordering on the fly, and the distribution could be used when
modeling a network.  (See also the "Properties" slides.)

One major change from the previous proposal was the addition of
"late" and "early" reordering density.  Together with the original
definition, Anura felt that this characterization might also be
useful in solving concatenation of partial path reordering metrics
to give a reordering metric for the full path.

This led back to a discussion of packet loss.  Jerry Perser asked if
lost packets were early or late?  Anura said that was recognized with
the help of a place label.  Al Morton again brought up that this was
the biggest issue in this draft; the current working group draft makes
reordering orthogonal with loss, and there is a clear definition of
"early" and "late".  The definitions of in and out of order and early
and late need to merge to add this to the draft.  Anura said that it
depends on what you are trying to achieve with reordering measurements:
perform TCP or network diagnosis?  Predict network behavior?  The use
of density would add value; a simple number describing reordering doesn't
provide enough information.  The definition of loss would have to be agreed.

Stanislav asserted that n-reordering makes the definition of loss
orthogonal to reordering. n-reordering metric doesn't change in the
presence of loss.  Not every router sends a stream of numbers with
monotonically increasing sequence numbers.  Anura thought there was a
problem with n-reordering and fast routers. Stanislav said that a
linear buffer would be sufficient.  Anura said that n squared would
hit. Stanislav said not for n-reordering.  Anura said that then
duplicates are an issue. Al said that you either save information on
packets which have arrived or those missing, usually the latter.
Anura thought that a moving window is better than keeping information
on packets.  Jerry Perser noted that test instruments can't track
millions of duplicate packets.  Anura said that with reordering
density you only need to keep track for a fixed window.  Stanislav
wanted to get back to loss.  Anura said they recognize loss within
window.  Jerry wondered why it isn't called loss and reordering
density.  Anura said it isn't a loss density.  Jerry said that he had
a problem claiming reordered packets when only one is lost.  For
example, look at slide 6; Jerry said packet 4 is in sequence.  Anura
said that according to his definition packet 4 is reordered [3 was
lost].  Anura wanted to know about packets arriving much later;
suddenly "2" comes in and your whole view changes.  Is this
appropriate?  Applications would presumably treat them as lost.  Are
they lost or reordered?  Stanislav said the definition of reordering
is only based on a sequence of arrivals.

Jerry wanted to know if the authors checked the resources you require to
implement reordering density (say for an oc192 link) -- what was the fastest
speed tested?   Anura said that results are forthcoming; they have software
on Linux for testing, and the fastest speed tested so far was fast Ethernet.

The authors would like this draft used as an alternative
characterization metric in the current draft or published as a
separate informational draft.  Henk asked if we should merge drafts
or proceed separately?  Matt said that his sense was that the current
authors seem to be disinclined to merge with loss mixed with reordering.
Al Morton said that was true, and there were no other comments.

4. IPPM-MIB
   -- Emile Stephan

Next, Emile Stephan presented the latest version of the MIB.  He
wanted to know the status of the registry -- it will be presented to
the ADs as soon as the (apparently now expired) version is refreshed
by the I-D editor after the meeting.  RMON would like the OIDs for the
metrics.  Bob Cole asked what happens for new metrics?  Matt said OIDs
should be included in those drafts.

As far as the MIB goes, Emile presented some recent (minor) changes,
and then tried to walk the room through a sample setup showing how all
the tables were organized and how views of the data are consumed (see
slides).  He tried to do a demo showing the MIB presenting results
from a commercial measurement system, but the demo hiccupped in the
middle.  (He later showed it to the chairs.)  He noted that
Andy Bierman had sent a comment to the list today, and that he hasn't
had time to consider it in detail, but he will respond.

Matt tried to start a discussion based on Andy's initial paragraph to
the list, which Matt summarized as that the MIB was "too complex".
Henk asked who had read the draft and if anyone was interested in
implementing it or had an alternative.  Only the chairs and Andy
admitted to reading it.  There was therefore no point in an extended
discussion, but Andy noted that there was a higher-level question:
what should be in an SNMP agent.  There are roles for an SNMP agent,
and roles for the management station.  In his mind, there are too many
features, and a lot in this draft is out of scope.  The draft ideas
are basically two years old, and there are no other implementations.
If no one is going to implement this (or even read it) then there must
be some action at the WG-level...  Emile noted that an operator would
not like to get flooded by "unsolicited" measurement results; filters
are required close to the controller of the measurement system, not in
the back office.

The feedback from Henk's customers is that the MIB should be simple,
light-weight, and give users the freedom to do their own aggregation
and statistics computation.  The draft seems to be too France Telecom
specific.  Emile noted that this desire was in contradiction with a
previous concern that the buffer for all results might be too big and
therefore some aggregation might be required.  There is a
contradiction between a desire to report only raw data and
a need for aggregation.  Henk wondered if perhaps the draft could
be limited to just the two tables it needs to provide raw data and
remove everything else?  Emile noted that the report tables are optional.
Bob Cole wondered if the compliance section was clear that only
two tables were required.  Al said that he would read a document
limited to core functionality.  Andy felt the document itself was too
complex?  Does it want to specify a MIB for a single aggregation box
or 1000 agents doing measurements?  Matt said that he would like to
see comments on this issue on the mailing list.

5. OWAMP
   -- Stanislav Shalunov

Finally Stanislav Shalunov presented the updates to OWAMP.  The
approval of the requirements document was announced to the group.
Updates to the protocol document are based on a sample open source
implementation, and include test vectors for the random number
generator used in the generation of packet sending schedules, so that
the receiver knows accurately when the sender was supposed to send
packets (which allows losses to be detected at the receiver without
having to correlate sent packets with the sender).

Matt asked how close Stanislav felt this document was to be forwarded
to the IESG.  One big issue is that the security has not been reviewed
by experts. That needs to be done.  Matt: What about a comparison with
IPMP? Stanislav said that there are two differences: OWAMP includes
control, and OWAMP tries to be "sneaky" and be not recognizable by
routers.  Matt noted that there was an ongoing discussion on IPMP on
the IRTF/IMRG mailing list if any working group members were
interested in following IPMP.

6. Milestones and Futures
   -- The Chairs

Matt Zekauskas closed the meeting noting the milestones that
have been met, those that are in process, and those that will
be dropped if no one steps up to work on them.  (see slides)
These were largely a re-iteration of the conclusions from the
Vienna meeting; they need to be run through the mailing list.
Matt and Henk will send email to the list to solicit input.

_______________________________________________
ippm mailing list
[email-masked] 
https://www1.ietf.org/mailman/listinfo/ippm

---

From: Matthew J Zekauskas <[email-masked]>
Date: Thu, 18 Dec 2003 11:09:21 -0500

The slides are also available currently at
http://people.internet2.edu/~matt/IPPM/Meetings/ietf58/

--Matt

---

From: Matthew J Zekauskas <[email-masked]>
Date: Thu, 18 Dec 2003 11:09:21 -0500

--Matt