Subject: [bmwg] Flow Export Issue#2: Test Device Capacity

From: Al Morton <[email-masked]>
Date: Sat, 12 Nov 2011 22:18:09 -0500

At 01:23 PM 10/2/2011, Jan Novak (janovak) wrote:

>==========PJ ===============
>
>Section 4.1:
>
>     In all measurements, the export
>     interface MUST have enough bandwidth to transmit Flow Export data
>     without congestion. In other words, the export interface MUST NOT be
>     a bottleneck during the measurement.
>
>
>PJ: What if the receiving interface, or the collector itself, is a 
>bottleneck? Previously you said it doesn't matter, and I disagreed.
>
>- data could be lost at the receiving end, making the DUT look worse 
>than it really is.
>
>==========PJ ===============
>
>I answered this in the previous review already:
>http://www.ietf.org/mail-archive/web/bmwg/current/msg02413.html
>
>I believe this is all covered in the section 4.4 already
>

Jan,

I see that Collector packet dropping issues (due to resource limits)
are covered (in Sec 4.2 and 4.4), but I don't see anything specific
for the traffic receiver capacity. You can cite where you've already
covered this point, or try the DUT-less calibration described below,
possibly in Section 4.1:

The traffic receiver MUST have sufficient resources to measure all
test traffic transferred successfully by the DUT, and this may be
checked through measurements with and without the DUT.

Al

---

From: Al Morton <[email-masked]>
Date: Sat, 12 Nov 2011 22:40:19 -0500

At 01:23 PM 10/2/2011, Jan Novak (janovak) wrote:

>Section 4.2:
>     The DUT export interface (see Figure 2) MUST be configured with
>     sufficient output buffers to avoid dropping the Flow Export data due
>     to a simple lack of resources in the interface hardware.
>
>PJ: then it might not be configured as it would in real life 
>conditions. Wouldn't it be better to record the real-life results 
>than to tweak the settings just for testing purposes?
>
>- eg, consider if the DUT operates with insufficient buffers in the 
>real-world scenario, and therefore drops a lot more data than when 
>testing under "ideal" conditions in the lab.
>
>I would like to disagree here. We are not measuring real life
>scenario here. We are trying to measure the maximum what the
>DUT can do. In real life you would also configure the export interface
>(if there is one dedicated at all - see the discussion in Appendix
>B) not to drop the export packets.

The test designer has the freedom to run tests in multiple
configurations. It is therefore possible to run both ideal
and "real-world" configurations, according to the needs of the
tester.  All configurations MUST be fully documented.

Al

---

From: Al Morton <[email-masked]>
Date: Sat, 12 Nov 2011 23:02:58 -0500

At 01:23 PM 10/2/2011, Jan Novak (janovak) wrote:

>Section 4.3:
>     The DUT SHOULD support IPFIX [RFC5101]
>
>Say why?
>
>This has been changed several times as the result of the previous 
>reviews. I don't
>know what text you would like there. Please suggest something.
>
>The previous text was "The DUT SHOULD support IPFIX [RFC5101] to allow easier
>                        results comparison."

I suggest returning to this text, it answers the question
and SHOULD allows the tester to deviate if absolutely necessary.

Al

>and your/someone else's question was "What difference it would make 
>??" - sorry no idea what you want from me here, it just feels like 
>common sense to have same protocol for testing everywhere if possible ??

---

From: Al Morton <[email-masked]>
Date: Sat, 12 Nov 2011 23:37:46 -0500

At 01:23 PM 10/2/2011, Jan Novak (janovak) wrote:

>Section 4.3.2: add "of", remove "the", and move "instantly" :
>
>       The Cache's Inactive and Active Timeouts MUST be known and taken
>       into account when designing the measurement as specified in
>       section 5. If the Flow monitoring implementation allows only
>       timeouts of zero (e.g. immediate timeout or non-existent Cache) then
>       the measurement conditions in the section 5 are fulfilled
>       inherently without any additional configuration. The DUT simply
>       instantly exports instantly information about every single packet.
>
>PJ: Assuming that the cache works with timeouts. What if it uses 
>some other mechanism, eg number of packets in the flow?
>
>- state what impact other such mechanisms may have.
>
>Changed the text. I don't know about any such impact. Could you elaborate
>please ??

It seems that the essential requirement is to configure the DUT to
export information on every (or almost every?) packet
for the tests that follow in Section 5.

I suggest to say that:
The configuration of the metering process MUST be reported.

and then you can follow with the paragraph cited above,
possibly edited as follows:

       For example, when a DUT uses timeouts in its metering process,
       the Cache's Inactive and Active Timeouts MUST be known and taken
       into account when designing the measurement as specified in
       section 5. If the Flow monitoring implementation allows only
       timeouts equal zero (e.g. immediate timeout or non-existent Cache) then
       the measurement conditions in the section 5 are fulfilled
       inherently without any additional configuration. The DUT simply
       exports information about every packet immediately.

Al

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Mon, 14 Nov 2011 22:51:38 +0100

Hi Al,

Thanks for pointing that out - I was stuck with the
collector (as receiver ...) - thinking of boxes like
Ixia/Smartbit on the receiving side, losses did not
come to my mind - I will add the text you suggest.

Jan

The climate of Edinburgh is such that the weak succumb young .... 
and the strong envy them.
                                 Dr. Johnson

-----Original Message-----
From: Al Morton [mailto:[email-masked]] 
Sent: 13 November 2011 03:18
To: Jan Novak (janovak); Paul Aitken (paitken)
Cc: [email-masked]
Subject: Flow Export Issue#2: Test Device Capacity

Jan,

Al

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Mon, 14 Nov 2011 22:53:42 +0100

>>  All configurations MUST be fully documented.

[>] Ok, will check this statement is suitably 
[>] placed somewhere in the text.

Jan

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Mon, 14 Nov 2011 22:54:34 +0100

[>] I suggest returning to this text, it answers the question
[>] and SHOULD allows the tester to deviate if absolutely necessary.

Ok, will do.
 
Jan

---

From: "Jan Novak (janovak)" <[email-masked]>
Date: Mon, 14 Nov 2011 22:55:24 +0100

Hi again,

Thanks a lot Al for your mediation on this
- will amend the text.

Jan

-----Original Message-----
From: Al Morton [mailto:[email-masked]] 
Sent: 13 November 2011 04:38
To: Jan Novak (janovak); Paul Aitken (paitken)
Cc: [email-masked]
Subject: Flow Export Issue #5: Metering Configuration

I suggest to say that:

       timeouts equal zero (e.g. immediate timeout or non-existent
Cache) then

Al