Subject: [PMOL] Re: DISCUSS: draft-ietf-pmol-sip-perf-metrics

From: Daryl Malas <[email-masked]>
Date: Thu, 3 Jun 2010 08:49:35 -0600

Hi Robert,

Is it your opinion I switch all of the metrics back to their original names
or just PDD?

Regards,

Daryl

> 
> ======= NEW ISSUES with -05 =======
> 
> 27 I think relabeling SRD as PDD is a mistake. At the very least it is
>    a change of such magnitude that it should be re-reviewed by the working
>    group. SRD as defined by -04 is similar, but not an exact reflection of
>    PDD. Saying it is "like" Post-Dial-Delay as defined in the PSTN is risky
>    enough. Using the same name makes it even more likely that someone will
>    come to the conclusion that it is measuring exactly the same thing. It
>    does not. For example, it fails to capture any delay from when
>    the user "finishes dialing" to when the INVITE is generated due to
>    DNS processing. It would be possible to engineer highly constrained
>    networks (that didn't use DNS or allow redirection or forking for example)
>    where the metric might behave very much like PDD in the PSTN, but that
>    will not be true in the general case.
> 
>

---

From: Robert Sparks <[email-masked]>
Date: Thu, 3 Jun 2010 09:56:45 -0500

PDD is the one I feel strongly about. 

If any of the others risk the same concerns, it would make sense to switch them back too.

RjS

---

From: Marian Delkinov <[email-masked]>
Date: Fri, 4 Jun 2010 14:09:01 +0200

Regarding the new issues with -05:

I have similar concerns about relabeling the following metrics:

Session Duration Time (SDT) --> Call Hold Time (CHT)

Although this metric corresponds to Call Holding Time in the telephony, this may lead to confusion that it's exactly the same thing. Still, here we refer to sessions that may not always relate to calls. So, I would prefer keeping the term "Session". 

Session Establishment Ratio (SER) --> Answer Seizure Ratio (ASR)

The term "Seizure" is used in the telephony signalling. In SIP we have "Invite". If in the telephony we use "Answer to Seizure Ratio" to identify the % of answered vs all initiated calls, I strongly feel we should avoid using the same terms "Answer" and "Seizure" in SIP. If there is any real need to change the metric label, it would make more sense to use "Invite Success Rate" or similar. 

Session Establishment Effectiveness Ratio (SEER) --> Network Effectiveness Ratio (NER)
The same thoughts: If we use NER, perhaps many will start comparing the measurement results with the TDM telephony, while here we have larger scope and different results.

In earlier Comments Robert wrote:
19 The ratio metrics don't define (or convey) the interval that totals
   are taken over. Are these supposed to be "# requests received since
   this instance was manufactured' or "since last reboot" or "since last
   reset of statistics" or something else? What is the implementation
   supposed to report when the denominator of a ratio is 0?

I propose a text to be inserted for all ratio metrics (may be valid for time metrics as well) specifying recommended measurements granularity of a 15 min interval. That interval can be used for benchmarking. All other granularities can also be supported, e.g. 1 min (for real time measurements), 3 min, 5 min, 30 min, 60 min etc. Thus various reports can later be generated based on these granularities. The same granularity intervals are widely used for the similar metrics in the TDM Telephony. How the counters are organized can be left to the manufacturers, but I also thought about the option to consider a standard MIB (like MIB II) for the counters that SIP end-to-end performance metrics are derived from.

At the end of chapter 4.7:
"Reference the example flow is Section 4.7."
Looks like the intention was to refer to Section 4.6?

Regards!
Mario.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Sparks
Sent: Thursday, 03 June, 2010 16:57
To: Daryl Malas
Cc: [email-masked]
Subject: Re: [PMOL] DISCUSS: draft-ietf-pmol-sip-perf-metrics

RjS

On Jun 3, 2010, at 9:49 AM, Daryl Malas wrote:

> Hi Robert,
> 
> Is it your opinion I switch all of the metrics back to their original 
> names or just PDD?
> 
> Regards,
> 
> Daryl
> 
>> 

_______________________________________________
PMOL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pmol

---

From: Daryl Malas <[email-masked]>
Date: Fri, 4 Jun 2010 09:06:40 -0600

Mario,

Thanks for the feedback.  It appears I may have gone to far to the "basic
telephony" side, and I will change the metric names back to their -04 names.

Regards,

Daryl

On 6/4/10 6:09 AM, "Marian Delkinov" <[email-masked]> wrote:

> Although this metric corresponds to Call Holding Time in the telephony, this
> may lead to confusion that it's exactly the same thing. Still, here we refer
> to sessions that may not always relate to calls. So, I would prefer keeping
> the term "Session".
> 
> The term "Seizure" is used in the telephony signalling. In SIP we have
> "Invite". If in the telephony we use "Answer to Seizure Ratio" to identify the
> % of answered vs all initiated calls, I strongly feel we should avoid using
> the same terms "Answer" and "Seizure" in SIP. If there is any real need to
> change the metric label, it would make more sense to use "Invite Success Rate"
> or similar. 
> 
> The same thoughts: If we use NER, perhaps many will start comparing the
> measurement results with the TDM telephony, while here we have larger scope
> and different results.
> 
> I propose a text to be inserted for all ratio metrics (may be valid for time
> metrics as well) specifying recommended measurements granularity of a 15 min
> interval. That interval can be used for benchmarking. All other granularities
> can also be supported, e.g. 1 min (for real time measurements), 3 min, 5 min,
> 30 min, 60 min etc. Thus various reports can later be generated based on these
> granularities. The same granularity intervals are widely used for the similar
> metrics in the TDM Telephony. How the counters are organized can be left to
> the manufacturers, but I also thought about the option to consider a standard
> MIB (like MIB II) for the counters that SIP end-to-end performance metrics are
> derived from.
>