Subject: RE: [Sipping] Comments on draft-malas-performance-metrics-05.txt

From: "Michael Hammer \(mhammer\)" <[email-masked]>
Date: Tue, 7 Nov 2006 13:31:37 -0500

Daryl,

The only metric that might be useful (throwing out for consideration) is
the differential between when the 200 OK passes some signaling point,
and when the forward or backward media passes the border.  This may give
some sense of whether clipping is occurring.  

Do we care if there is significant difference between when media flow
arrives at the caller and 200 OK arrives, or vice versa?

Do we care if there is a delay from 200 OK received and forward path
flow?

Note this may require some timing relationship if the border element is
decomposed.

Note, the 3 cases we seem to have is signaling-only interactions,
media-only, and signaling-media interactions.  I am wondering about the
latter case.

Mike

> -----Original Message-----
> From: Malas, Daryl [mailto:[email-masked]] 
> Sent: Tuesday, October 31, 2006 12:33 PM
> To: Dolly, Martin C, ALABS; Michael Hammer (mhammer); Gonzalo 
> Camarillo; sipping
> Subject: RE: [Sipping] Comments on 
> draft-malas-performance-metrics-05.txt
> 
> All of the metrics in this draft deal with signaling 
> indicators to TB and TS.  They have nothing to do with media. 
>  The "session" deals with signaling not media.  The 
> correction Gonzalo mentioned is to an example call flow in 
> section 3.9.1.
> 
> Do we need a metric that measures the actual receipt and 
> sending of media as related to the UAC and UAS?  If that is 
> the case, then I think there are two types of media that 
> should be considered here.  First, do we care about 
> "early-media" as related to a 183 w/ SDP?  Second, 
> "session-media" as related to media generated by the UAC and 
> UAS after a 200 OK and the reception of the ACK by the UAS.
> 
> I personally cannot think of any metrics we should consider 
> that deal with the media portion, which aren't dealt with in 
> RFC 3550 and 3611 (or elsewhere such as ITU-T P.862, etc.).  
> Mike, you seemed to begin to mention some.  Thoughts?
> 
> --Daryl
> 
> > -----Original Message-----
> > From: Dolly, Martin C, ALABS [mailto:[email-masked]]
> > Sent: Tuesday, October 31, 2006 6:59 AM
> > To: Michael Hammer (mhammer); Gonzalo Camarillo; sipping
> > Cc: Malas, Daryl
> > Subject: RE: [Sipping] Comments on
> draft-malas-performance-metrics-05.txt
> > 
> > The timing of cut-thru is extremely important, as it 
> relates to tones 
> > and announcements that occur before answer.
> > 
> > -----Original Message-----
> > From: Michael Hammer (mhammer) [mailto:[email-masked]]
> > Sent: Tuesday, October 31, 2006 8:51 AM
> > To: Gonzalo Camarillo; sipping
> > Cc: [email-masked]
> > Subject: RE: [Sipping] Comments on
> > draft-malas-performance-metrics-05.txt
> > 
> > Is there an issue associated with when "cut through" really occurs?
> > 
> > In other words, in some cases each media path may be available at 
> > different times.  So, perhaps the setup times should be relative to
> each
> > unidirectional path.
> > 
> > Also, is the focus of "session" the media or signaling?  Just trying
> to
> > be sure it is clear what we intend to measure.
> > 
> > Mike
> > 
> > 
> > > -----Original Message-----
> > > From: Gonzalo Camarillo [mailto:[email-masked]]
> > > Sent: Tuesday, October 31, 2006 3:00 AM
> > > To: sipping
> > > Cc: [email-masked]
> > > Subject: [Sipping] Comments on
> draft-malas-performance-metrics-05.txt
> > >
> > > Hi,
> > >
> > > a few comments on draft-malas-performance-metrics-05.txt.
> > >
> > > The Section "Conventions used in this document", which currently 
> > > appears between the Abstract and the TOC, is usually part of the 
> > > Terminology Section instead.
> > >
> > > Expand acronyms (e.g., UAC, UAS) on their first use.
> > >
> > > Section 3.1.2: RRD is defined as:
> > >
> > > "the interval is defined from the initial REGISTER 
> request and the 
> > > final response indicating a failure received from the destination 
> > > registrar or interim proxies."
> > >
> > > However, in the example, the UAC receives an error response, it 
> > > sends a new REGISTER, and then gives up. RRD is 
> calculated until the 
> > > UAC gives up. Therefore, the description needs to clarify which 
> > > final responses are taking into account to calculate RRD 
> (i.e., when 
> > > the UAC gives up) and which ones are not (e.g., authentication 
> > > challenges).
> > >
> > > Section 3.2 talks about the Session Request Delay (SSD 
> and says that 
> > > is telephony applications of SIP is known as Post Dial Delay.
> > >
> > > The ITU defines the post-selection delay as:
> > >
> > > "Post-selection delay (en bloc sending) is defined as the time 
> > > interval from the instant the first bit of the initial 
> SETUP message 
> > > containing all the selection digits is passed by the calling 
> > > terminal to the access signalling system until the last 
> bit of the 
> > > first message indicating call disposition is received by 
> the calling 
> > > terminal (ALERTING message in case of successful call)."
> > >
> > > However, the draft defines Time Begin as:
> > >
> > >    "TB occurs when the designated request has been 
> processed by the
> > >     application and last bit of the request packet has been sent 
> > > from the
> > >     proxy or UA (and is externally observable at some physical
> > >     interface)."
> > >
> > > Therefore, while for the ITU, TB relates to the first bit of the 
> > > message, for the draft TB relates to the last bit of the 
> message. We 
> > > should either use the first bit of the message as well of clarify 
> > > that the Post Dial Delay is not the same thing as
> SRD.
> > >
> > > Section 3.4 defines the Session Duration Time as an interval 
> > > starting with a 200 OK response. However, session establishments 
> > > involving an answer in the ACK cannot be considered to be 
> > > established until the ACK arrives to the UAS. These 
> sessions can be 
> > > considered to be "accepted", though. Section 3.4 should 
> clarify this 
> > > point.
> > >
> > > The example in Section 4 does not seem to relate to any 
> of the ways 
> > > to provide finer granularity, which are described right 
> before the 
> > > example.
> > >
> > > Cheers,
> > >
> > > Gonzalo
> > >
> > > _______________________________________________
> > > Sipping mailing list  
> https://www1.ietf.org/mailman/listinfo/sipping
> > > This list is for NEW development of the application of SIP Use 
> > > [email-masked] for questions on current sip Use 
> > > [email-masked] for new developments of core SIP
> > >
> > 
> > _______________________________________________
> > Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> > This list is for NEW development of the application of SIP Use 
> > [email-masked] for questions on current sip Use 
> > [email-masked] for new developments of core SIP
> 

_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: "Fardid, Reza" <[email-masked]>
Date: Thu, 16 Nov 2006 18:24:24 -0800

Comments are inline.

Thanks,
-Reza

-----Original Message-----
From: Daryl Malas [mailto:[email-masked]] 
Sent: Thursday, November 16, 2006 10:11 AM
To: Fardid, Reza
Cc: [email-masked]
Subject: Re: [Sipping] Comments on

On Sat, 2006-11-11 at 15:25 -0800, Fardid, Reza wrote:
>  
> 
> Daryl,
> 
>  
> 
> Here are a few comments on this draft, based on a first pass through
> it:
> 
>  
> 
> 1. Introduction
> 
>  
> 
> I would add User Behavior as another factor that can affect
> measurements of some of the metrics.
> 
I'll have to think about this further.  Can you please give an example
of how user behavior can effect signaling and therefore impact a metric?
Thanks...
>  
> 
>> ASR (==SER) measurement is affected by UA2 busy or unavailable
signals, >> User Behavior in this context means not caused by the
network, excluding >> overload conditions, which are covered with the
ISA metric.
>
> 3. SIP Performance Metrics
> 
>  
> 
> If User Behavior MAY affect measurements of a metric, then it MUST be
> noted for each defined metric.
> 
See above.
>  
> 
> 3.5 Session Establishment Ratio (SER)
> 
>  
> 
> I suggest referencing ITU-T E.411 for the telephony definition of ASR,
> unless GR-512-CORE includes it.
I will review and reference as necessary. Thanks...
> 
> 3.11 (or 3.5.1) Session Establishment Efficiency Ratio (SEER):
> proposed as a new metric, unless it can be derived from currently
> defined metrics
> 
>  
> 
> This metric is used to provide a better representation of network
> performance by eliminating user behavior from  Session Established
> Ratio (SER). It is known as Network Efficiency Ratio (NER) in
> telephony applications, and was adopted from the ITU-T E.411
> Amendment.
> 
>  
> 
> The following response codes provide a guideline for this metric:
> 
>  
> 
> -        480  Temporarily Unavailable
> 
> -        486  Busy Here
> 
> -        600  Busy Everywhere
> 
>  
> 
> SEER % = (# of  INVITEs w/ associated 200OK + # of 480s + # of 486s +
> # of 600s)/(Total # of INVITEs)
> 
There are a number of metrics people have suggested.  All of them are
valid and good metrics.  I will ask the working group to review them,
and comment.  I don't mind adding them if the mailing list finds
significant value in them.  Thanks...
>  
> 
>> Sounds good. I think NER (==SEER) is more useful for both the 
>> customer (SLA) and service provider (performance) tracking purposes 
>> than ASR, which is contaminated.
>
> Regards,
> 
> Reza Fardid
> 
>  
> 
>

---

From: "Venna, Nagarjuna" <[email-masked]>
Date: Tue, 21 Nov 2006 10:44:23 -0500

Daryl,
 
I agree with Reza on NER (=SEER). We have found that it is a useful
metric to set thresholds on for SLA monitoring and conformance.
 
Regards,
Nagarjuna

>> than ASR, which is contaminated.

-----------------------------------------------------------------------------------------------------------------------------------------------
Service Quality Matters. Test the performance and quality of your VoIP or IP video service at: 
http://www.TestYourVoIP.com
http://www.TestYourIPVideo.com