Subject: [PMOL] SIP performance metrics draft

From: Al Morton <[email-masked]>
Date: Tue, 29 Jan 2008 19:54:43 -0500

Hi Daryl,

Here are a few comments on the SIP draft.

I'm raising some metric development related points,

and trying to harmonize the draft with the pmol framework.

Some of the points imply additional development of

the pmol framework, and are comments for *both* drafts.

(I've marked these with "FRMWK")

I will have a few more comments soon, but this is

a good start.

regards,

Al

===========================================================

Intro

It's useful to have a Scope, and the current statements of scope

can be found in the Introduction section.  I suggest re-titling

the section "1.0  Introduction and Scope", or splitting
into 2 sections.

Active and/or Passive in scope of draft:

I think this question came up along the way:

Is the draft applicable to Active/Passive or both?

I think the answer is both, and suggest to add this:

The metrics defined in this document are applicable in scenarios
where the SIP messages launched (into a network under test) are dedicated
messages for testing purposes, or where the messages are user-initiated
and a portion of the live traffic present.  These two scenarios are
sometimes referred to as active and passive measurement, respectively.
FRMWK: I'm not sure where to mention this consideration in the
text.

Section 3.1 of the framework draft highlights the need to

identify the audience, or users of the metrics

(FRMWK: need to flesh this section out).

Here's a suggestion, it could be combined with the paragraph

that begins, "the scope":

The intended audience for this document can be found among network
operators, who often collect information on the responsiveness of the
network to customer requests for services.

Section 3 candidate material to add

SIP Configuration:  Various timers influence performance, so
the

draft must mention them (at least).  Ideally, the supporting
info

for any measurement report would include a list of relevant

configuration variables and their settings (in the originating UA).

T1 looks like a good candidate:

(from

http://tools.ietf.org/html/rfc3261#section-17.1.1.1 )

17.1.1.1 Overview of INVITE Transaction

The INVITE transaction consists of a three-way
handshake.  The client

transaction sends an INVITE, the server transaction sends
responses,

and the client transaction sends an ACK.  For
unreliable transports

(such as UDP), the client transaction retransmits requests
at an

interval that starts at T1 seconds and doubles after
every

retransmission.  T1 is an estimate of the round-trip
time (RTT), and

it defaults to 500 ms.  Nearly all of the transaction
timers

described here scale with T1, and changing T1 adjusts their
values.

Elsewhere, the default value of T2 is 4 seconds. We usually list all

the configuration parameters in the BMWG drafts, so that everyone

reports the same ones.

Clock Accuracy and Error:  Since most of the SIP metrics are
time

intervals measured at a single UA, time sync between two remote

points is not an issue. Time interval accuracy for the single

clock should be identified as a source of error, however.

Section 10 of RFC 2330

http://tools.ietf.org/html/rfc2330#section-10

discusses clock issues in detail (reference in FRMWK).

"Wire Time": This is another issue described well in

section 10.2 of RFC 2330

http://tools.ietf.org/html/rfc2330#section-10.2

The basic problem is that there is a difference between the time

when a host sees a packet for time-stamping, and the time that the

packet actually appears on the "wire".

_______________________________________________
PMOL mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pmol

---

From: Al Morton <[email-masked]>
Date: Thu, 31 Jan 2008 10:23:00 -0500

Hi Daryl,

A few more points:

At 07:54 PM 1/29/2008, Al Morton wrote:
>Clock Accuracy and Error:  Since most of the SIP metrics are time

It will help to designate one of the two time stamps (TB or TS)
as "the" time each individual test was conducted.  Clearly,
TB is the most suitable for this use. TB is the time that is
controled in an Active test scenario. I suggest adding this
to the current definition of TB:

... TB represents the time at which each request-response test begins,
and SHALL be used to designate the time when a particular test was conducted
(e.g., The Session Request Delay at "TB" and <some specific UA interface>
was measured to be X ms.)

OR

we could add a Date-Time stamp format, and let TB and TS exist
in a locally-defined format, as long as their difference produces
sufficient resolution. There are plenty of date-time formats
that we could specify:
RFC 2330 specifies UTC format for time in section 6.1, when discussing
measurement units.
The IETF reference for time formats appears to be
http://tools.ietf.org/html/rfc1123#section-5.2.14,
but there are others such as ISO 8601.
NTP format has more resolution, but it is expressed
relative to 0h Jan 1 1900.

Session Setup: and Session Establishment:
The presence of these two definitions begs for a general definition of
"Session Failure", especially when so many metrics have "failure" versions.
The definition should probably include a reference to the IANA list
of Response codes (and thereby, the Reference for each code)
http://www.iana.org/assignments/sip-parameters

Singletons, Samples, Statistics (as defined in RFC 2330):
http://tools.ietf.org/html/rfc2330#section-11
Some one-line definitions for these terms are:
Singleton:  a single instance of a metric, a single result.
Sample: a number of single instances, or singletons.
Statistic: a metric derived from a sample by computing some statistic.

I think it will help to introduce these (or similar) definitions
into the SIP metrics draft (and the FRMWK). Each of the
message flows illustrate a Singleton metric, while equations like
the Average Registration Request Delay in section 3.1 are examples
of Statistical metrics.

The Data Collection Section 5.4 touches on the topic of Samples
in the last paragraph. Would it be useful to include discussion of how
the Sample should be collected for summarization with a Statistic?
Perhaps this could be addressed in a short section on Methodology?
(This is where it helps to associate a date-time with each
request-response singleton, so that the time coverage of the sample
and resulting statistic are known.)

Al