Subject: [ippm] Zaheduzzaman Sarker's No Objection on draft-ietf-ippm-capacity-metric-method-10: (with COMMENT)

From: Zaheduzzaman Sarker via Datatracker <[email-masked]>
Date: Fri, 28 May 2021 05:58:40 -0700

Zaheduzzaman Sarker has entered the following ballot position for
draft-ietf-ippm-capacity-metric-method-10: No Objection

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-ippm-capacity-metric-method/

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Thanks for the work on the document. I consider this document to be an
important one when it comes to IP layer capacity for access networks.

After my review and observing the IPPM working group discussions, I consider
the valid discuss points raised by Magnus Westerlund is now addressed in the
-10 version of this document.

please find my comments and observations below -

* Major Issue: I didn't find any restriction imposed by the memo on the number
of allowed concurrent tests between same Source and Destination. My
understanding is this memo should either strictly prohibit the concurrent tests
between same Source and Destination or provide a maximum limit of concurrent
tests where the algorithm has been tested to provide valid results. I am not
going to hold a discuss on this but I would like to be corrected if I am wrong
in the understanding.

* Minor Issues , nits:

   ** I found it a bit odd to not find Magnus Westerlund's name in the
   acknowledgement section as his review and comments has improved the
   algorithms and the document a lot. I would encourage the author's to include
   names of the individuals who has provided any sort of review (including the
   directorate reviews) that improved this document.

   ** Section 2:

      *** I am not getting the context of "local goal", what does it really
      mean?

      *** I kind of feel that the paragraph 2 and 3 should swap their order to
      focus the goal of "defining efficient test procedure" and then
      "harmonizing the metric and method across the industry" becomes "another
      goal". This is based on the current text which makes me feel there are
      number of goals and the ranks (if there is any) of the goals are not
      clear. If there is not rank among the goals then simply putting them in a
      bullet list would help the readability.

   ** Section 8.1:

      *** paragraph 1: s/ agressiveness / aggressiveness .

      *** paragraph 2: It will be helpful if it is mentioned who pre-builts the
      table and  who (and where) it will be used.

   ** Section 10 : I find 4, 5, 6 of the new security considerations introduced
   in this section not entirely security related rather general considerations
   on how to run the tests. I think those mentioned considerations are more
   suitable to put it in section 8 (in section 8.3). I understand the feeling
   the everyone should read the security considerations but there is no
   guarantee that it is the case specially while implementing the tests. Hence,
   putting then in section 8 likely will bring these to attention early.

---

From: "MORTON JR., AL" <[email-masked]>
Date: Sun, 30 May 2021 01:59:55 +0000

Hi Zahed,

Thanks for your review, please see replies below, [acm].

Al

> -----Original Message-----
> From: Zaheduzzaman Sarker via Datatracker <[email-masked]>
> Sent: Friday, May 28, 2021 8:59 AM
> To: The IESG <[email-masked]>
> Cc: [email-masked]; [email-masked];
> [email-masked]; Ian Swett <[email-masked]>; [email-masked];
> [email-masked]
> Subject: Zaheduzzaman Sarker's No Objection on draft-ietf-ippm-capacity-
> metric-method-10: (with COMMENT)
> 
...
[acm] 
I'm trying to understand your comment in order to address it properly.
Let's consider the existing requirements:

We have text to restrict concurrent non-measurement traffic, in the Scope:

	It is RECOMMENDED to discontinue non-measurement traffic that shares a 
	subscriber's dedicated resources while testing: measurements may not be 
	accurate and throughput of competing elastic traffic may be greatly reduced.

We allow a single test to use concurrent flows, and the default is one flow
(text below is near the end of section 8.3):

	In general, results depend on the sending stream characteristics; 
	the measurement community has known this for a long time, and needs to keep 
	it front of mind. Although the default is a single flow (F=1) for testing, 
	use of multiple flows may be advantageous for the following reasons:
	...

We don't use the term "concurrent" in the draft, and it is obviously
counter-productive to run more than one test independently attempting to 
measure the *maximum* capacity between the same Source and Destination.
The number of concurrent tests between same Source and Destination
that we have tested is one.

The maximum limit on concurrent tests *at an on-net server* is a function
of the server host characteristics: CPU power, network interface speed, and
the external Internet access capacity the host can expect to use.
None of these characteristics are the subject of standardization 
in the form of numerical limits. However, in Section 10 we already specified:

- Hosts MUST limit the number of simultaneous tests to avoid resource 
  exhaustion and inaccurate results.

Perhaps you have a better view of the current text now, and limits on testing 
that makes sense in the context of Maximum capacity measurement.
Was it your intent that we mandate what seems to be obvious (one test only)?
Or something else?

>    directorate reviews) that improved this document.
[acm] 
OK
> 
>    ** Section 2:
> 
>       *** I am not getting the context of "local goal", what does it
>       really mean?
[acm] 
Lars questioned this word choice too. We changed it to "secondary goal".

>       bullet list would help the readability.
[acm] 
The section reads differently with the wording changes related to replacement
of "local goal".
>       *** paragraph 1: s/ agressiveness / aggressiveness .
[acm] 
OK

> 
>       *** paragraph 2: It will be helpful if it is mentioned who pre- builts the
[acm] 
... (by the test initiator)...

>    putting then in section 8 likely will bring these to attention early.
[acm] 
We already make reference to these requirements much earlier than Section 8.3, 
in the text of Section 2, Scope:

- MUST only be used in circumstances consistent with Section 10, 
  Security Considerations

We can't make the reference much earlier than that!

Changes above have been implemented in the working text for version 11.