Subject: RE: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics

From: [email-masked]
Date: Sun, 25 Jun 2006 08:07:35 -0400

The authors of this work have been in regular communication from Erich M.
Nahum of SPEC.  Here is a snip from one of the emails that well summmarizes
the relationship:

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Friday, May 26, 2006 2:51 PM
> To: Vijay K. Gurbani
> Cc: Malas, Daryl; [email-masked]; [email-masked]; [email-masked]
> Subject: [bmwg] SPEC SIP Benchmarking Effort
>
> [snip]
>
> I have been organizing the SPEC SIP benchmarking effort, and would
> love to get participation from IETF folks.  It seems like the two
> activities could be very complementary.  
> 
> Based on what I've seen, it seems like the IETF BMWG and Daryl Malas' 
> draft posted on SIPPING a few days ago are more oriented towards defining 
> the proper methods and metrics to measure (e.g., call success rate,
defects 
> per million, etc.).  
> 
> In contrast, SPEC wants to develop and distribute common code for 
> benchmarking, as is done with SPECWeb and SPECJAppServer.  That code can 
> and should use the standardized peformance definitions agreed to by 
> SIPPING and/or BMWG.  

Scott

-----Original Message-----
From: Romascanu, Dan (Dan) [mailto:[email-masked]]
Sent: Sunday, June 25, 2006 6:00 AM
To: Burger, Eric; Malas, Daryl
Cc: [email-masked]; [email-masked]
Subject: RE: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics

Eric,

>From the bmwg charter: 

> The major goal of the Benchmarking Methodology Working Group is to
make
a series of recommendations concerning the measurement of the
performance characteristics of various internetworking technologies;

To the extend that we consider SIP an internetworking technology, I
believe that the scope of the 'SIP Performance Metrics' draft is within
the scope of what bmwg is doing for a while, quite successfully, some
say. 

On a more 'philosophical plan', there is nothing that says that the IETF
work must strictly deal with defining the bits in the Internet Protocols
- see http://www.ietf.org/internet-drafts/draft-hoffman-taobis-08.txt.
And in any case, measuring how a protocol or a device implementing a
protocol behaves can be considered also 'DIRECTLY related to protocol
development'.

Dan

PS - This does not mean that I do not have my own questions about the
I-D :-)

PPS - If we end by deciding that we do this work in the IETF we should
certainly coordinate with other organizations developing standards,
performance measurement and test methodologies in this space. 

> -----Original Message-----
> From: Burger, Eric [mailto:[email-masked]] 
> Sent: Friday, June 23, 2006 11:18 PM
> To: Malas, Daryl
> Cc: [email-masked]; [email-masked]
> Subject: [bmwg] RE: [Sipping] Draft: SIP Performance Metrics
> 
> More to the point, is this work appropriate for the IETF at 
> all?  I would offer that if we needed additions to SIP to 
> instrument it for monitoring, then sure.  However, I think 
> this would much better be done in another venue such as the 
> SIP Forum or IMS Forum or even SPEC.  If it turns out that 
> there need to be some SIP changes, then we could bring that 
> to the IETF.
> 
> We have way too much on our plate as it is.
> 
> 
> 
> Disclaimer: I'm on the boards of both the SIP Forum and IMS Forum.
> However, this isn't make-work for those groups.  Rather, this 
> is a plea to NOT make more work in the IETF that is not 
> DIRECTLY related to protocol development.
> 
> -----Original Message-----
> From: Henning Schulzrinne [mailto:[email-masked]]
> Sent: Wednesday, May 24, 2006 2:44 PM
> To: Vijay K. Gurbani
> Cc: Malas, Daryl; [email-masked]; [email-masked]; [email-masked]
> Subject: Re: [Sipping] Draft: SIP Performance Metrics
> 
> A working group has just been formed (this week) in the SPEC 
> consortium.
> 
> Vijay K. Gurbani wrote:
> > Daryl: There is work starting in the BMWG along SIP performance 
> > metrics, being worked by Scott Poretsky and me.  At the 
> very least, we 
> > should co-ordinate your draft with the thoughts there, and then can 
> > figure out with the help of chairs and ADs as to where the 
> work should 
> > proceed.
> > 
> > I have also heard about some performance metrics work in TISPAN and 
> > SPEC, but I do not have any more details about these.
> > 
> 
> _______________________________________________
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
> This list is for NEW development of the application of SIP 
> Use [email-masked] for questions on current 
> sip Use [email-masked] for new developments of core SIP
> 
> _______________________________________________
> bmwg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/bmwg
> 

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

_______________________________________________
bmwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/bmwg

---

From: [email-masked]
Date: Sun, 25 Jun 2006 17:23:07 -0400

> Nahum of SPEC.  Here is a snip from one of the emails that well summmarizes

Actually, I'm from IBM, not SPEC.  SPEC is just a consortium of its member
companies/organizations.  Sort of like IETF.

-Erich

> Use [email-masked] for questions on current sip
> Use [email-masked] for new developments of core SIP

-- 
Erich M. Nahum                  IBM T.J. Watson Research Center
Research Staff Member           P.O. Box 704
[email-masked]            Yorktown Heights NY 10598

---

From: David Newman <[email-masked]>
Date: Sun, 25 Jun 2006 15:04:00 -0700

> companies/organizations.  Sort of like IETF.

Unlike SPEC, the IETF is *supposed* to be a collection of individuals,
not representatives of vendors or other member organizations.

When those in the operator community gripe, with some reason, that it's
devolved into the IVTF, that's when we most need to remind ourselves
that we represent ourselves and ourselves only.

See The Tao of IETF, guiding principle P4:

http://www.ietf.org/tao.html

dn

---

From: David Newman <[email-masked]>
Date: Mon, 26 Jun 2006 10:18:14 -0700

Rick Jones wrote:

> FWIW - SPEC isn't quite as officially enlightened as the IETF, but as a
> "SPECcie" from a vendor I'm supposed to sport a bit of a split
> personality, being able to wear as times warrant, either my HP hat or my
> SPEC hat.  So, it is indeed OK to say that someone is "from SPEC."
> 
> rick jones
> http://www.netperf.org/

Fair enough. Of course that's different than saying "from Vendor X"
on an IETF list or at a meeting. Expected behavior at IEEE or SPEC, not
so at the IETF. We can all say "from IETF" -- as individuals.

dn

---

From: [email-masked]
Date: Mon, 26 Jun 2006 15:05:42 -0400

From: [email-masked]

   Actually, I'm from IBM, not SPEC.  SPEC is just a consortium of its
   member companies/organizations.  Sort of like IETF.

IETF is a consortium of its member *individuals*, which makes a great
deal of difference.

Dale

---

From: Rick Jones <[email-masked]>
Date: Mon, 26 Jun 2006 09:50:22 -0700

David Newman wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 

rick jones
http://www.netperf.org/