Subject: [bmwg] Comments/review on draft-ietf-bmwg-bgp-basic-convergence

From: "Banks, Sarah" <[email-masked]>
Date: Tue, 8 Apr 2014 13:47:16 -0400

Hello draft-ietf-bmwg-bgp-basic-convergence authors,
	In prepping to be your document shepherd, I re-read the draft again with
fresh eyes (hah) and have a few comments; mostly editorial, but
nonetheless, here they come.

Section 1 and 1.1
	I'm not a fan of restating - say what you mean, and be clear and precise.
The document generally reads this way, so consider starting off that way
too. In particular, the first sentence of the first paragraph in Section
1.1, "Since benchmarking is a science of precision, let us restate the
purpose of this document in benchmarking terms." There's something about
this that rubs me the wrong way, as an editor. I'm also not 100% sure
benchmarking is a science of precision. :)  In any event, consider stating
once what the draft is about.

Section 1.1 
	What is "Basic BGP"? Is there "Advanced BGP" too? I jest - but you're
using a term before introducing it, instead, waiting to introduce it 2
paragraphs down. Consider adjusting this.

	You call out a test topology of 3 or 4 nodes - why? I don't think this
has to be a long paragraph, just a sentence, that covers why you chose 3/4
nodes, and not 2, or 200.

	Speaking of Basic BGP definitions - your definition says "as RFC 4271 ...
For IPv6". Your introduction states that this document covers
methodologies for both IPv4 and v6, yet here, you say Basic BGP for v6
only. Consider revising this sentence/paragraph.

Section 1.2
	A minor language nit. In your second sentence/first paragraph, you state,
"To maintain a reliable connectivity within...". Consider revising to, "To
maintain reliable connectivity..."

	Last sentence, "These simple tests... High-level check, of the ..." -
consider removing the ",". There's no need for a pause there.

	Last sentence - what is "multiple implementations"? Please consider
clarifying.

Section 1.4
	While I understand what you're trying to do here, I often tell customers
NOT to do this - that they SHOULD test with the timer settings, for
example, that they deploy in the network today. There are lots of reasons
why default timers don't work for real-life deployments, and test results
you get from default timers are often NOT the same as when, for example,
configured for aggressive timers. I think this skews the data in a way as
to make it useless if non-standard or non-defaults are deployed in the
wild. I'd prefer that if a customer wants to use aggressive timers, they
be configured the same way across each iteration of the test, and across
each vendor's gear, for apples to apples comparison.

Section 3
	You state, "These simple test nodes have 3 or 4 nodes with the following
configuration:" - but then you list what I think are 4 different
configurations. Consider revising the aforementioned sentence to read
something like, "These simple test setups have 3 or 4 nodes with one of
the following configurations:"

	BTW I'm not sure what use "simple" has in the original sentence; while
I'm not married to this idea, I'd nix the use of the word "simple" here,
even from the sample sentence I gave above. :)

	Section 3 is the first time the draft states that both iBGP and eBGP will
be covered. Consider adding this fact to your overview/introduction.

Section 4.2
	Second paragraph - "each test run must identify... Number of routes. This
route stream must be ... Reporting stream." Are these normative
references? :)

	Last paragraph, first sentence, "It is RECOMMENDED that the user may
consider..." Consider revising this to remove the "may" - "It is
RECOMMENDED that the user consider advertising..."

Section 4.3 
	why is "Minimal" with an "M"?

	Why are exact policy documentations a "should" - I think this is
normative anyhow, but why not MUST? If you don't document the policy
processes, so that the tests could be reproduced effectively?

Section 4.8
	Why should, and not MUST?

Section 5.1.1
	When you say "Stand Deviation" did you mean "Standard Deviation"?

Test repeatability:
	Some of the tests cases say that it's recommended to run the test case a
couple times - and not others.  I wonder if you meant this to be true for
all the cases. In any event, consider adding a section or adding to the
"test considerations" section a note on running the test cases multiple
times - and even further, consider taking a stand on how many times to run
them. :)

Section 5.8
	How does one trigger a GR event on the DUT? The document does a pretty
good job hand holding the tester - consider adding a sentence or two at
the start of the section on how to trigger the event. Do you expect the
DUT interface to flap or the test tool to cause the flap? Do you care?

	Why does the draft cover a test case for GR and not NSR?

Thanks
Sarah