Subject: [ippm] New Liaison Statement, "LS/r on current status of the draft Recommendation ITU-T Q.3961 (reply to SG11-LS31)"

From: Liaison Statement Management Tool <[email-masked]>
Date: Thu, 17 May 2018 10:40:35 -0700

Title: LS/r on current status of the draft Recommendation ITU-T Q.3961 (reply to SG11-LS31)
Submission Date: 2018-05-17
URL of the IETF Web page: https://datatracker.ietf.org/liaison/1575/

From: Judit Kiss <[email-masked]>
To: Brian Trammell <[email-masked]>,Bill Cerveny <[email-masked]>
Cc: Scott Mansfield <[email-masked]>,Bill Cerveny <[email-masked]>,Brian Trammell <[email-masked]>,IP Performance Measurement Discussion List <[email-masked]>,Mirja Kühlewind <[email-masked]>,Spencer Dawkins <[email-masked]>,[email-masked]
Response Contacts: A. C. Morton <[email-masked]>
Technical Contacts: 
Purpose: For information

Body: ITU-T Study Group 12 thanks you for your liaison, sharing your plan to put Draft Q.3961, “Testing methodologies of Internet related performance measurements including e2e bit rate within the fixed and mobile operator’s networks”, on hold, awaiting the progress of other standards. 

We are aware that text similar to Q.3961 is under development in ETSI INT and has received extensive comments during the attempted approval process, all of which remain un-addressed and unresolved after many months. 

We are also aware of similar text recently revised by ETSI STQ, which has been prepared for a specification with completely non-compatible scope to draft Q.3961, and which suffers from the same shortcomings raised in the ETSI INT approval process. In summary, there is no available text to support Draft Q.3961 moving forward at this time, in any form.

Study Group 12 has participated in the Joint meeting with SG11 and ETSI INT, and our plan is to prepare relevant normative specifications for IP-layer Capacity and Flow-related Parameters (throughput), and that SG11 make use of the new specifications when available, as expressed during the meeting by SG12 Chairman, Kwame Baah-Acheamfuor.  We were pleased that there were no objections to this cooperative plan, and will proceed as agreed.

At this time, we would like to inform you of our progress to develop the aforementioned normative specifications in Study Group 12. 

The IETF has approved and published RFC 8337, “Model-Based Metrics for Bulk Transport Capacity”, in March 2018. The Model-Based Metrics (MBM) work was the result of many years considering the problem of transport capacity measurement, primarily in the IETF IP Performance Metrics (IPPM) working group. The specification carefully describes the many issues and difficulties with repeatability when testing with standard-compliant TCP (section 4), and solves these problems principally by designing a method and a set of diagnostic tests where TCP’s flow-control is disabled. The method involves the evaluation of a target transport performance in terms of transmission rate and Round-trip time (RTT).

Study Group 12 now informs you of our plans to resolve the need for Recommendations in this area.  Study Group 12 will take the MBM work as a starting-point, prepare normative specifications for IP-layer Capacity and Flow-related Parameters (throughput), and evaluate multiple versions of candidate definitions for these parameters through a comprehensive evaluation plan. The approach is to compare measurements using implementations of candidate parameters with the “ground-truth” of baseline path bandwidth, characterized on a per-element basis. The testing campaign will be conducted over different access types and path conditions, evaluating the errors observed and determining where the candidates are sufficiently accurate, and where they are not.

Study Group 12 has provided a detailed explanation of the clause 6.12 requirements in Appendix IX of Rec. Y.1540, titled “Explanation of TCP-based measurement inadequacy to meet normative requirements”.  However, we have concluded that Capacity measurements described in RFC 8337 meet the requirements in clause 6.12 of ITU-T Recommendation Y.1540 (as described in SG12 C160, see attached). 

We invite all experts to participate in the evaluation plan’s development and to subsequently participate in the evaluations. SG11’s plans to develop the text of the draft Q.3961 should now move forward after the evaluation and approval of Study Group 12’s normative specifications are complete, according to the contribution-driven schedule.  Certainly, SG11 experts will be willing to contribute their expertise and resources to this plan, consistent with their desire to complete work on this topic as quickly as possible.

The current draft of revisions to Y.1540, and a new Annex A for the new parameter definitions now contains the current text of the evaluation plan for candidate definitions for the new normative parameters for IP-layer Capacity and Flow-related Parameters (throughput).

Study Group 12 would like to invite Study Group 11 to schedule co-located meetings in Geneva, as this would give us the additional opportunity to schedule a workshop adjacent to the meeting dates. We would like to use the workshop to share our experience, to make the work of overseeing our mandate and our coordination roles more efficient and effective. 

Attachments:
SG12-TD540R2 “Revisions to Y.1540 and text of new Annex A and Initial Evaluation Plan” (Editors of Y.1540 and new Annex A/Y.1540)
SG12-C160 “Development of Y.1540 Flow-related Parameters based on IETF RFC 8337” (AT&T, Inc.)
Attachments:

    oLS57-556R2
    https://www.ietf.org/lib/dt/documents/LIAISON/liaison-2018-05-17-itu-t-sg-12-ippm-lsr-on-current-status-of-the-draft-recommendation-itu-t-q3961-reply-to-sg11-ls31-attachment-1.docx

    oLS57att-td540R2
    https://www.ietf.org/lib/dt/documents/LIAISON/liaison-2018-05-17-itu-t-sg-12-ippm-lsr-on-current-status-of-the-draft-recommendation-itu-t-q3961-reply-to-sg11-ls31-attachment-2.docx

    oLS57att-C160
    https://www.ietf.org/lib/dt/documents/LIAISON/liaison-2018-05-17-itu-t-sg-12-ippm-lsr-on-current-status-of-the-draft-recommendation-itu-t-q3961-reply-to-sg11-ls31-attachment-3.docx