Subject: Re: [ippm] WG interest in Multicast performance monitoring drafts

From: Yaakov Stein <[email-masked]>
Date: Mon, 2 Aug 2010 09:29:37 +0300

I see two distinct issues here :
  1) passive monitoring 
  2) multicast performance

If IPPM is interested in entering into passive monitoring methods,
then perhaps it should define these for unicast flows first.
(As I mentioned at the meeting, MEF has developed a lot of this for layer-2.)

Multicast performance is indeed an important topic to be studied.

Y(J)S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Henk Uijterwaal
Sent: Thursday, July 29, 2010 14:53
To: Tempia Bonda Alberto
Cc: [email-masked]; IETF IPPM WG
Subject: [ippm] WG interest in Multicast performance monitoring drafts

IPPM Group,

Last Monday, Alberto and others presented this set of drafts:

> > at the upcoming IPPM meeting in Maastricht we will present a draft about
> > multicast performance monitoring
> > (draft-bipi-mboned-ip-multicast-pm-requirement-01).
> > Requirement draft:
> > http://tools.ietf.org/html/draft-bipi-mboned-ip-multicast-pm-requirement-01
> > Solution drafts:
> > http://tools.ietf.org/html/draft-liu-mboned-multicast-realstream-monitor-02
> > http://tools.ietf.org/html/draft-cociglio-mboned-multicast-pm-00

We would like to know if people consider this topic appropriate for
the IPPM WG, and if so, if there is interest to work on this topic.

The drafts discuss passive measurements of actual traffic streams, using
the metrics developed in IPPM.  In the current charter, this work is
out of scope, so picking it up would require a charter change.  Obviously,
we only want to do this if there is sufficient interest in the work.

Please respond to the list or the chairs in private by September 13.

Matt & Henk

-- 
------------------------------------------------------------------------------
Henk Uijterwaal                           Email: henk.uijterwaal(at)ripe.net
RIPE Network Coordination Centre          http://www.xs4all.nl/~henku
P.O.Box 10096          Singel 258         Phone: +31.20.5354414
1001 EB Amsterdam      1016 AB Amsterdam  Fax: +31.20.5354445
The Netherlands        The Netherlands    Mobile: +31.6.55861746
------------------------------------------------------------------------------

I confirm today what I denied yesterday.            Anonymous Politician.
_______________________________________________
ippm mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ippm

---

From: Yaakov Stein <[email-masked]>
Date: Mon, 2 Aug 2010 10:03:38 +0300

I am working my way through draft-ietf-ippm-tcp-throughput-tm-04 .

The draft is not defined with the accuracy and precision I have come to expect 
in IPPM documents.

The efficiency measure is well-defined as the number of bytes NOT retransmitted.
As Al mentioned, the example thereafter then gives a different measure - 
the number of segments not retransmitted. Actually, both may be interesting,
but it needs to be clarified which is intended.

The second metric is not a well defined performance measure at all.
It is the amount of time needed to transfer some undefined amount of data using some number of connections,
and thus is highly dependent on the amount of data and the number of connections.
I understand that given the parameters you can come up with a number to expect
and compare what you measure to the expected, but if so, let's define this comparison.

I would propose normalizing by defining the ratio of the actual transfer time 
to the expected time (given the layer-3 1-way delay and bandwidth).
This is a pure TCP-layer measure.

Unfortunately, this measure is not completely divorced from the efficiency.
One reason that it takes longer to transfer at the TCP-layer than at the IP-layer
is because of the retransmissions!  
However, TCP's inherent flow control, and ECN if used, and the various algorithms
for determining the RTT, etc. all have their effects.
This is why the number of connections is important - using multiple connections
fools the transport into giving the user more weight.

Regarding the traffic shaping/policing section, I am confused. 
The traffic shaping and policing I am familiar with are done at layer-2,
and are thus layer-2 quantities, although they of course affect TCP performance.
(W)RED is of course a transport quantity - but what is it that we are trying to quantify here ? 
The number of dropped packets is already covered in the efficiency measure.

Y(J)S

---

From: <[email-masked]>
Date: Mon, 2 Aug 2010 09:20:18 +0100

Hi, guys, 
  I agree with Yaakov. All the existing group RFCs are based on active measurement. Passive measurement could be interesting but we need some foundation before calling it. 
  The performance of multicast is an important matter to study and it is within the group charter. 

Cheers,
Lei  

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yaakov Stein
Sent: 02 August 2010 07:30
To: Henk Uijterwaal; Tempia Bonda Alberto
Cc: [email-masked]; IETF IPPM WG
Subject: Re: [ippm] WG interest in Multicast performance monitoring drafts

If IPPM is interested in entering into passive monitoring methods, then perhaps it should define these for unicast flows first.
(As I mentioned at the meeting, MEF has developed a lot of this for layer-2.)

Y(J)S

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Henk Uijterwaal
Sent: Thursday, July 29, 2010 14:53
To: Tempia Bonda Alberto
Cc: [email-masked]; IETF IPPM WG
Subject: [ippm] WG interest in Multicast performance monitoring drafts

IPPM Group,

Last Monday, Alberto and others presented this set of drafts:

> > at the upcoming IPPM meeting in Maastricht we will present a draft 
> > about multicast performance monitoring 

We would like to know if people consider this topic appropriate for the IPPM WG, and if so, if there is interest to work on this topic.

The drafts discuss passive measurements of actual traffic streams, using the metrics developed in IPPM.  In the current charter, this work is out of scope, so picking it up would require a charter change.  Obviously, we only want to do this if there is sufficient interest in the work.

Matt & Henk

---

From: Barry Constantine <[email-masked]>
Date: Mon, 2 Aug 2010 06:15:10 -0700

Yaakov,

Thank you for taking the time to read the draft in detail, your comments are all very good.

Let me try to answer them here:

1. You and Al both caught a typo in the draft for the TCP Efficiency metric.  It is definitely in bytes and the example was shown in segments; the version 5 will correctly show the example in bytes.

2. As for TCP Transfer time, I like what you propose and we have considered this in the past "I would propose normalizing by defining the ratio of the actual transfer time to the expected time".  I think some network operators will like to see this as a single time (predicted versus measured) and other cases where a ratio is preferred.  I think a good compromise will be to discuss both techniques and provide examples in the next version of the draft.

3. "Unfortunately, this measure (TCP Transfer Time) is not completely divorced from the efficiency.  One reason that it takes longer to transfer at the TCP-layer than at the IP-layer is because of the retransmissions!"  - Also a very good point, we have considered spelling this out explicitly in the document and that TCP Efficiency per connection should be measured along with the overall TCP Transfer time.  This too will be added into draft-05.

4. Traffic shaping versus policing.  You are correct that traffic shaping / policing are done at Layer 2 and that traffic shaping is a technique commonly used to smooth the bursty nature of TCP. RED techniques are also Layer 2 oriented (buffer queue management), but are intended to directly affect TCP by causing "early" loss which in turn causes TCP to decrease its transmission rate. 

   Traffic shaping and RED could be tested at Layer 2, although it would require the emulation of TCP's burstiness, retransmission behavior, congestion control, etc.; the draft specifies a guideline to test for proper shaping / RED settings in the network using actual TCP layer traffic.  

   So long answer to the question: this traffic shaping / policing section shows how to test for proper traffic shaping configuration and RED by using multiple TCP connections; very useful technique which is not well known.

Again, your comments will be added to the next version 5 of the draft and thank you for taking the time to read in this detail!

Thanks,

Barry

Principal Member of Technical Staff

JDSU Communication Test (formerly Acterna)

Emerging Markets and Technology Research         

One Milestone Center Court                              

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Yaakov Stein
Sent: Monday, August 02, 2010 3:04 AM
To: IETF IPPM WG
Subject: [ippm] TCP-layer performance metrics

Y(J)S

---

From: Foldøy, Johan <[email-masked]>
Date: Thu, 5 Aug 2010 12:25:08 +0200

Hi all,

Reading draft-ietf-ippm-tcp-throughput-tm-04, there is a minor editorial point (which may or may not have been discussed already) that might strengthen the draft: it seems most of the example scenarios* describe issues relevant to an IPv4-only context? If so, maybe this could be clearly stated at the beginning of the document, or one could add text which explains how the same issues are handled in IPv6 implementations as well?          

*e.g section 3.1

Best Regards
 
Mr. Johan Foldoy
Senior Engineer
 
Terminals and Networks Department/Section for Networks
Norwegian Post and Telecommunications Authority

-----Opprinnelig melding-----
Fra: [email-masked] [mailto:[email-masked]] På vegne av Yaakov Stein
Sendt: 2. august 2010 09:04
Til: IETF IPPM WG
Emne: [ippm] TCP-layer performance metrics

Y(J)S

---

From: Barry Constantine <[email-masked]>
Date: Tue, 10 Aug 2010 08:43:56 -0700

Hello Johan,

As far as TCP's behavior is concerned, there is no difference if it is running over IPv4 or IPv6. Only difference is the TCP checksum calculation using either IPv4 pseudo header or IPv6 pseudo header.

Let me add a note to the introduction stating the fact that the methodology pertains to IPv4 and v6.

Thank you for the comment,
Barry

Germantown, MD 20876                                         

(W) 240-404-2227                                                

(C) 301-325-7069

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Foldøy, Johan
Sent: Thursday, August 05, 2010 6:25 AM
To: IETF IPPM WG
Subject: Re: [ippm] TCP-layer performance metrics

Hi all,

*e.g section 3.1

Best Regards
 
Mr. Johan Foldoy
Senior Engineer

-----Opprinnelig melding-----
Fra: [email-masked] [mailto:[email-masked]] På vegne av Yaakov Stein
Sendt: 2. august 2010 09:04
Til: IETF IPPM WG
Emne: [ippm] TCP-layer performance metrics

Y(J)S

---

From: <[email-masked]>
Date: Wed, 25 Aug 2010 08:46:16 +0200

Henk, 

I just had a look at

> > http://tools.ietf.org/html/draft-liu-mboned-multicast-realstream-monitor-02

The draft proposes to apply Y.1731 methodology to monitor multicast groups. While the loss measurement is active/passive, the delay measurement is clearly active. I'd suggest the authors to read RFC5644. I didn't compare details but would expect the following:
- differences in packet loss measurement between draft-liu and RFC5644.
- no serious difference in delay measurement methodology between draft-liu 
  and RFC5644 (meaning that the draft-liu delay measurement method is 
  identical with or a subset of RFC5644). 

Regards

Ruediger