Subject: [sacm] Review of draft-ietf-sacm-requirements-01

From: Chris Inacio <[email-masked]>
Date: Sat, 18 Oct 2014 03:18:05 +0000

I did a detailed reading of the draft and have a bunch of comments.  But I will start with high level impressions first.

The draft starts really strong in the requirements, but tails off towards the end - the requirements aren’t as well developed or clear.  Where it is clear that Nancy has spent some time capturing solid requirements, I would like to say kudos to her - she’s captured a lot of details in a relatively short amount of text and actually made me think about the overall architecture in some new ways.

I would ask for a more careful usage of MUST, SHOULD, MAY.  I would like to think of the requirements in terms of “absolutely have to have” => MUST and “desirable to have” => SHOULD or MAY, depending.  It isn’t clear to me, entirely, that everything isn’t intended to be a MUST, in which case, maybe this isn’t a useful comment.  But I think there might be some room in some of the requirements for leeway.

More detailed comments:

Section 2.1
	Should you create a 2.1  “Introduction” section for the text preceding requirement G-001?  While I don’t believe that the text in the list 1-5 are requirements, maybe we should be crystal clear on that.
	Then 2.2 would be the “General Requirements” section.

G-001 - I really want this split into 3 subsections of requirements, 
	* transport 
	* info model
	* query lang

	It’s beast of a requirement as it stands now.  Also, I’m not excited by the “allow support for non-standard and future extensions”.  I find it peculiar that a SDO would be requiring the development of a standard to allow for non-standards to be included in the standards.  (In the future, I will create a sentence with more uses of the word “standard”. :) )

G-003 - remove the final “defined” in the paragraph.

G-004 - reword:  “… protocols, transports are suitable to implementations are suitable to fit in …”  Maybe something like:  “that the data model, protocols, and transports are suitably implemented to fit into the different deployment models and scenarios.”

G-006 / GN-005:  “mandatory to implement data attributes” => needs to be changed to get rid of "data attributes” or we need to establish what this means in terms of the requirements doc.  I’m fairly certain I understand it, but if you ask me while/after building a sacm compliant implementation if that is met, I don’t think I’ll know.  We might really mean that the info model must have a set of IEs that will be mandatory to implement in all data models, and systems that support multiple data models must support the ability to transparently exchange all mandatory to implement IE’s across all DM’s.  (But we didn’t say that.  :) )

REQ-002:  Does this imply accepting multiple data models (& info models?) in the “SACM protocol”?  Does this imply how much of SACM is a metaprotocol, possibly?  Is SACM possibly a way to negotiate various disparate protocols (capabilities & versions) and maybe a query / response protocol?  I think this requirement creates FAR reaching implications. How does this requirement fit with G-006 / GN-005 which requires an explicit set of mandatory to implement data attributes?

REQ-006: “The query operation may be a synchronous or asynchronous request.”  First, an IETF comment, is that “may” a “MAY”?  From an implementors point of view: OUCH.  I would really prefer you pick one or both, but settle that THE PROTOCOL/STANDARD *MUST* support some set, but IMPLEMENTATIONS *MAY* choose and negotiate.  If you don’t choose, just remove the line from the requirements, it doesn’t help anyone in the future as it stands now.

REQ-009:  I actually don’t understand that requirement.

ARCH-001:  A little caution on the sizes used, if concrete numbers are really desired.  If sacm is deployed at Ft. Bragg in the US, it’s human population is 238,646 [http://www.army-technology.com/features/feature-largest-military-bases-world-united-states/].  If each of those folks has a cell phone, a desktop, and 1-2 other devices we’re in the millions.  If Ft. Bragg implements something like an IoT and manages it with sacm, then I think that number gets a whole lot higher.  I’m pretty sure that all the planes, helicopters, etc. probably have *at least* 1 IP address each.

ARCH-004:  “systems can utilize an already established mutually agreed upon native data format, which may be standard or implementation-specific”.  I understand why one might want that statement there, but it should be dropped.  (1) No one meeting in the sacm room could ever prevent this from happening if they wanted to (2) putting this in the requirements doesn’t help the design of the IM, DM, or any other protocols or operations.

IM-009:  Please expand the text.  (Lisa, I know you explained this, verbally, already, but it needs to be written down.)

IM-011:  remove “- for example”.  The follow on text is either a requirement or not.

IM-014:  Does this imply that using sacm protocols the sensors/agents can be configured via a sacm initiated request?

IM-015:  Does this imply something like a simple calculus that lets me logical / arithmetically / statistically define new “virtual” readings from actual readings?  Does this language have to be supported at every sensor/agent?  Also, we need a firm definition on the “point of collection”.

T-002: Reword please.  I take it to mean the following:  “Data Integrity: For protocols which do not provide for data integrity in the transport layer, a sufficient data integrity mechanism MUST be provided."

Security Considerations:  I really like this section, especially as someone who (in theory at least) does reviews for the security directorate.  BUT (you knew it was coming,) shouldn’t the last 3 paragraphs in the security considerations section actually be requirements?, say SEC-001, SEC-002, SEC-003?  :)

I hope this helps.  And I’ll see if I can plow through the info model this weekend.

Chris

---

From: "Nancy Cam-Winget (ncamwing)" <[email-masked]>
Date: Mon, 20 Oct 2014 18:02:27 +0000

Thanks Chris!

See below:

On 10/17/14, 8:18 PM, "Chris Inacio" <[email-masked]> wrote:

>
>
>I did a detailed reading of the draft and have a bunch of comments.  But
>I will start with high level impressions first.
>
>The draft starts really strong in the requirements, but tails off towards
>the end - the requirements aren¹t as well developed or clear.  Where it
>is clear that Nancy has spent some time capturing solid requirements, I
>would like to say kudos to her - she¹s captured a lot of details in a
>relatively short amount of text and actually made me think about the
>overall architecture in some new ways.
>
>I would ask for a more careful usage of MUST, SHOULD, MAY.  I would like
>to think of the requirements in terms of ³absolutely have to have² =>
>MUST and ³desirable to have² => SHOULD or MAY, depending.  It isn¹t clear
>to me, entirely, that everything isn¹t intended to be a MUST, in which
>case, maybe this isn¹t a useful comment.  But I think there might be some
>room in some of the requirements for leeway.
[NCW] Fair points and I¹ll try to bring better consistency to themŠ.but if
you see some you¹re passionate about getting the right language please
highlight them so I can ensure they¹re addressed.

>
>More detailed comments:
>
>Section 2.1
>	Should you create a 2.1  ³Introduction² section for the text preceding
>requirement G-001?  While I don¹t believe that the text in the list 1-5
>are requirements, maybe we should be crystal clear on that.
>	Then 2.2 would be the ³General Requirements² section.
[NCW] Right, is is meant to be preface before getting straight into the
general requirementsŠ.I¹ll see about breaking it into an overview or some
such subsectionŠ

>	It¹s beast of a requirement as it stands now.  Also, I¹m not excited by
>the ³allow support for non-standard and future extensions².  I find it
>peculiar that a SDO would be requiring the development of a standard to
>allow for non-standards to be included in the standards.  (In the future,
>I will create a sentence with more uses of the word ³standard². :) )
[NCW]  So, ³general² is meant to imply that it applies to all the
sections.  Do you suggest moving it or having it reasserted in each of the
other sections (e.g. Transport, info model..etc)?

The intent is to allow for future proofing the standards too as we have to
recognize that we will not be able to standardize ³every posture attribute
type and action².  So we have to allow for the extensibility for new types
to come inŠ.then it¹ll be up to the group whether these are managed
through IANA or notŠ..
I will try to rework the text so it¹s not explicitly stating allowance for
vendor specific extensions (but that will be very highly likely to happen
:-)

>
>G-003 - remove the final ³defined² in the paragraph.
[NCW] OK

>
>G-004 - reword:  ³Š protocols, transports are suitable to implementations
>are suitable to fit in Š²  Maybe something like:  ³that the data model,
>protocols, and transports are suitably implemented to fit into the
>different deployment models and scenarios.²
[NCW] OK

>
>G-006 / GN-005:  ³mandatory to implement data attributes² => needs to be
>changed to get rid of "data attributes² or we need to establish what this
>means in terms of the requirements doc.  I¹m fairly certain I understand
>it, but if you ask me while/after building a sacm compliant
>implementation if that is met, I don¹t think I¹ll know.  We might really
>mean that the info model must have a set of IEs that will be mandatory to
>implement in all data models, and systems that support multiple data
>models must support the ability to transparently exchange all mandatory
>to implement IE¹s across all DM¹s.  (But we didn¹t say that.  :) )
[NCW] HmmmŠ.so attributes are defined in the terminology draftŠ.would you
rather I remove ³data²?

>
>REQ-002:  Does this imply accepting multiple data models (& info models?)
>in the ³SACM protocol²?  Does this imply how much of SACM is a
>metaprotocol, possibly?  Is SACM possibly a way to negotiate various
>disparate protocols (capabilities & versions) and maybe a query /
>response protocol?  I think this requirement creates FAR reaching
>implications. How does this requirement fit with G-006 / GN-005 which
>requires an explicit set of mandatory to implement data attributes?
[NCW] I believe so (e.g. There may exist multiple data models)  so given
the existence of multiple, we have to put requirements of at least a set
of attributes (or perhaps a data model and the data model includes a MUST
set of attributes) that are mandatory to implement.  The negotiation is
part of the discovery in G-013

>
>REQ-006: ³The query operation may be a synchronous or asynchronous
>request.²  First, an IETF comment, is that ³may² a ³MAY²?  From an
>implementors point of view: OUCH.  I would really prefer you pick one or
>both, but settle that THE PROTOCOL/STANDARD *MUST* support some set, but
>IMPLEMENTATIONS *MAY* choose and negotiate.  If you don¹t choose, just
>remove the line from the requirements, it doesn¹t help anyone in the
>future as it stands now.
[NCW] This is why its left as an uncapped ³may² as imho SACM can MUST
support for both types of requests.
But the operation could be one or the other, or a task could use both.

>
>REQ-009:  I actually don¹t understand that requirement.
[NCW] This was the outcome of a WG discussion to ensure that we explicitly
stated that the task of collection and doing something with the collected
result could be separableŠ..don¹t recall the particular use case, but it
was agreed that it needed to be called out explicitly.

>
>ARCH-001:  A little caution on the sizes used, if concrete numbers are
>really desired.  If sacm is deployed at Ft. Bragg in the US, it¹s human
>population is 238,646
>[http://www.army-technology.com/features/feature-largest-military-bases-wo
>rld-united-states/].  If each of those folks has a cell phone, a desktop,
>and 1-2 other devices we¹re in the millions.  If Ft. Bragg implements
>something like an IoT and manages it with sacm, then I think that number
>gets a whole lot higher.  I¹m pretty sure that all the planes,
>helicopters, etc. probably have *at least* 1 IP address each.
[NCW] I¹m OK saying Œvery large¹, but somehow my notes from the
discussions said that I had to quantify.  But I agree that we could group
to the many millions.

>
>ARCH-004:  ³systems can utilize an already established mutually agreed
>upon native data format, which may be standard or
>implementation-specific².  I understand why one might want that statement
>there, but it should be dropped.  (1) No one meeting in the sacm room
>could ever prevent this from happening if they wanted to (2) putting this
>in the requirements doesn¹t help the design of the IM, DM, or any other
>protocols or operations.
[NCW] OK by me.

>
>IM-009:  Please expand the text.  (Lisa, I know you explained this,
>verbally, already, but it needs to be written down.)
[NCW] Yes, will likely need to expand on most of the 1 sentence reqmts in
the IM sectionŠ.

>
>IM-011:  remove ³- for example².  The follow on text is either a
>requirement or not.
{NCW] OK

>
>IM-014:  Does this imply that using sacm protocols the sensors/agents can
>be configured via a sacm initiated request?
[NCW] This again goes back to REQ-009 discussionsŠ.but I think its more
prescriptive of the ³collection task² which I¹m not sure we want to
explicitly defined? I¹m not sure it¹s really in scope for the charter, but
will have to have Dan, Adam and Kathleen weigh inŠ.

>
>IM-015:  Does this imply something like a simple calculus that lets me
>logical / arithmetically / statistically define new ³virtual² readings
>from actual readings?  Does this language have to be supported at every
>sensor/agent?  Also, we need a firm definition on the ³point of
>collection².
[NCW] My response is same as above.  The intent is to assert that a
request could be comprised of multiple requests, I¹m not sure we want to
be prescriptive of ³how² in the requirements draft.

>
>T-002: Reword please.  I take it to mean the following:  ³Data Integrity:
>For protocols which do not provide for data integrity in the transport
>layer, a sufficient data integrity mechanism MUST be provided."
[NCW] HmmmŠ.my bad.  It is to state that data integrity SHOULD be provided
at the transport layer.

>
>Security Considerations:  I really like this section, especially as
>someone who (in theory at least) does reviews for the security
>directorate.  BUT (you knew it was coming,) shouldn¹t the last 3
>paragraphs in the security considerations section actually be
>requirements?, say SEC-001, SEC-002, SEC-003?  :)
[NCW] Agreed. I¹m thinking that Requirements don¹t warrant
³considerations² but rather a set of requirements with appropriate
language, e.g. MUST, SHOULD, etc

>
>
>
>I hope this helps.  And I¹ll see if I can plow through the info model
>this weekend.
[NCW]  Many thanks for the thorough review!!!  Am working on incorporating
them this weekŠ

>
>Chris
>
>_______________________________________________
>sacm mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/sacm

---

From: Chris Inacio <[email-masked]>
Date: Wed, 22 Oct 2014 20:18:04 +0000

> other sections (e.g. Transport, info model..etc)?

[CMI]  I was thinking more of a G-001 that was more decomposed within G-001.  Just so that the overall point of the set of requirements was still maintained as a high-level adaptable architecture, but maybe break out the pieces (still under G-001) as separate (unnumbered) bullets.  Especially for posterity, and folks who aren’t necessarily intimating involved now.

> vendor specific extensions (but that will be very highly likely to happen
[cmi] I’m sure it will be likely to happen.  I believe the “ility” we can add to our work is extensibility, an escape hatch.  What *I* would clearly want in the protocol/IM/DM (someplace) is the ability to (1) mark a piece of data/protocol/information as vendor specific (2) ensure that even when adding vendor specific enhancements, those can be ignored by other implementations with the biggest loss being *only* those vendor specific pieces.  I have ideas how to do that at the IM / DM layer, but how one does that at the transport level is pretty far beyond my thoughts at the moment.

> [NCW] HmmmŠ.so attributes are defined in the terminology draftŠ.would you

[cmi] That’s my mistake for not finding the terminology;  I looked for “data attributes” not just attributes.  But I’m still stuck on if that applies more to IM or DM.  I’ll defer to you / the group.  I don’t have a solution for my discomfort with this requirement, so maybe its really a fault in me.  :)

> part of the discovery in G-013

[cmi] Okay, I understand what you wrote.  I just want to make sure that the whole group understands this requirement, and its implications.

> was agreed that it needed to be called out explicitly.
[cmi] I was the one who pushed for the 2 modes of analysis.  :)  My point, way back, was that collection should enable (well, really, the standard should not ``MUST'' enable, but instead, the standard MUST not prevent) storage and 2 modes of analytics against the system.  (1) Immediate alerting against a non-compliant attribute (2) retrospective analysis of the state of the universe.  Immediate alerting on something like ``firewall disabled’’ on endpoint seems obvious; but just as obvious to me is:  all machines were complaint 3 months ago, but we learn today that complaint software component X was zero-day targeted 3-months ago.  What machines had software X at version Y starting at 3-months ago to now?