Subject: Re: [sacm] FOR REVIEW: Vulnerability Assessment Scenario Issue #1 - Managing Terminology

From: Henk Birkholz <[email-masked]>
Date: Fri, 3 Jun 2016 21:15:11 +0200

Hello all,

some comments on the current definition of terms:

1) Supplemental Collection task: looks like a good way to move forward

Does this propose to define a separate collection task that focuses on 
on the acquisition of previously obtained endpoint attributes from a 
repository in the SACM domain? The endpoint attributes could be 
identical to the ones that can be acquired via the already defined 
collection task, but most certainly do not have to be! I would propose 
to "handle" the endpoint attributes in an identical way and to annotate 
where they come from, when they were acquired and similar metadata.

As a result, this could enable a single method to conduct both tasks and 
simply things, I think.

2) Vulnerability assessment: the proposed definition sounds fine

The current definition of Endpoint Assessment is: "Evaluate the actual 
values of the endpoint attributes against those expressed in the policy. 
(An evaluation result may become additional endpoint data)."

Implied - but yet not explicitly defined - in the definition of Endpoint 
Assessment is that it is a concatenated sequence of tasks defined by 
SACM. (Software) vulnerability assessment would be a specialization of 
that composition of tasks.

Background: i.e., the vulnerability assessment would process a sub-set 
of endpoint attributes that can be acquired by the:

- collection task (which is directly targeted at specific target 
endpoints) and the
- target endpoint discovery task (which is supposed to be acquiring any 
kind indicator information (i.e. also endpoint attributes) that can be 
used to detect and re-detect target endpoints in a SACM domain) as a 
basis for the collection task.

3) Endpoint management capability: seems to be a little bit vague, 
still. Especially the term "management".

This description seems to be more about a repository of expected 
(should) or possible (could) states that are based on configuration? Is 
an endpoint management capability (and its corresponding functions) also 
providing guidance(instructions)?

4) Vulnerability management capability: Here too, the term management 
seems to imply guidance (or management decision) of some sort.

The definition itself seems to describe a repository or a meta directory 
that provides information acquired, for example, by ROLIE?

5) Management Capability:

It should be made very clear what this capability actually provides; 
i.e., which category of endpoint attributes and/or guidance. This would 
resolve most of the ambiguity in a simple way, I think. Essential 
information elements of the content should also be included in the SACM 
IM, I think.

6) Endpoint Management Capability and Vulnerability Management Capability:

It seems - taking into account 2-4 - that Vulnerability Management 
Capability is a part of Endpoint Management Capability?

7) process & workflow:

A workflow might involve multiple processes with various dependences and 
interconnections, I think. But I am not sure if that distinction is 
really relevant to SACM. Workflow seems to be the more general term and 
is less overloaded.

Viele Grüße,

Henk

On 05/31/2016 10:41 PM, Haynes, Dan wrote:
> Did anyone else have any thoughts on this particular issue?
>
> If we want to take the approach of promoting a few terms as Adam has
> suggested, we might want do it as follows.
>
> Supplemental (previously targeted) collection: The task of collecting
> specific endpoint information from the target endpoint, that is not
> available from the endpoint management capability, in order to make a
> determination about that endpoint (vulnerability status, identification,
> etc.).
>
> This seems general enough to be promoted as-is.
>
> Vulnerability assessment: The process of determining whether a set of
> endpoints is vulnerable according to the information contained in the
> vulnerability description information.
>
> We would need to generalize it more to remove the reference to
> “vulnerability description information”. One approach might be to
> replace “vulnerability description information” with “information
> pertaining to the existence of a flaw or flaws in software, hardware,
> and/or firmware on an endpoint.”  It might also make sense to highlight
> this as a specialization of “assessment”.  As a side note, does anyone
> see a distinction between process and workflow (as described here
> http://blog.goodelearning.com/bpmn/business-process-vs-workflow/) and
> that assessment might be better described as a workflow? With that said,
> our definition comes from RFC5209 and we may not want to make any
> changes which is perfectly fine – just something to think about :).
>
> Endpoint management capability:  An enterprise IT capability managing
> endpoint identity, endpoint information, and associated metadata on an
> ongoing basis.
>
> Vulnerability management capability:  An enterprise IT capability
> managing endpoint vulnerabilities and associated metadata on an ongoing
> basis by ingesting vulnerability description information and
> vulnerability detection data, and performing a vulnerability assessment.
>
> Right now, we have definitions for “endpoint management capability” and
> “vulnerability management capability”. To support promoting these, we
> may want to create a more general “management capability” term and then
> include “endpoint management capability” and “vulnerability management
> capability” as specializations of that term.  A definition of
> “management capability” might read something like “An enterprise IT
> capability responsible for managing specific information about an
> endpoint in support of continuous monitoring processes which may or may
> not be driven by guidance”.
>
> Anyways, this is just a little bit more to think about :).  Let me know
> what you think.
>
> Thanks,
>
> Danny
>
> *From:*sacm [mailto:[email-masked]] *On Behalf Of *Haynes, Dan
> *Sent:* Tuesday, May 24, 2016 7:43 AM
> *To:* Adam Montville <[email-masked]>
> *Cc:* [email-masked]
> *Subject:* Re: [sacm] FOR REVIEW: Vulnerability Assessment Scenario
> Issue #1 - Managing Terminology
>
> Do others have thoughts regarding whether or not terminology from the
> vulnerability assessment scenario draft should be promoted ot the
> Terminology I-D?  If so, which terms make sense to promote?
>
> Thanks,
>
> Danny
>
> *From:*Adam Montville [mailto:[email-masked]]
> *Sent:* Thursday, May 19, 2016 8:04 PM
> *To:* Haynes, Dan <[email-masked] <mailto:[email-masked]>>
> *Cc:* [email-masked] <mailto:[email-masked]>
> *Subject:* Re: [sacm] FOR REVIEW: Vulnerability Assessment Scenario
> Issue #1 - Managing Terminology
>
> I’ll echo my statements from the virtual interim by pointing out that
> the vulnerability scenario, while narrowing our focus, does not change
> the scope of our charter.  From that perspective, I would like to see
> terms “promoted” into the terminology draft sooner rather than later.
>   Of the set of terms Danny described, I think endpoint management
> capability and targeted collection are prime candidates for such
> promotion.  I also believe that vulnerability management capability and
> vulnerability assessment are suitable for promotion. Vulnerability
> description information and vulnerability detection data could stay
> within the vulnerability draft.
>
> Adam
>
>     On May 17, 2016, at 1:15 PM, Haynes, Dan <[email-masked]
>     <mailto:[email-masked]>> wrote:
>
>     During today’s virtual interim meeting, we discussed various open
>     issues with respect to the Vulnerability Assessment Scenario [1] as
>     a result of feedback that we received on the draft which you can see
>     here [2][3].  The slide’s from today’s meeting can be found here [4].
>
>     The first issue was whether or not the following terms from Section
>     2 of the Vulnerability Assessment Scenario should be pulled into the
>     Terminology I-D [5] with the expectation that any terms pulled in
>     will be used outside of the Vulnerability Assessment Scenario.
>
>     •Vulnerability description information
>
>     •Vulnerability detection data
>
>     •Endpoint management capability
>
>     •Vulnerability management capability
>
>     •Vulnerability assessment
>
>     •Targeted collection
>
>     During the meeting, there were concerns that while some of these
>     terms leverage general terms already in the Terminology I-D, they
>     use them in specific ways with respect to the Vulnerability
>     Assessment Scenario.  It was also noted that some terms like
>     “endpoint management capability” and “vulnerability management
>     capability”, while specific, have the potential to become broader as
>     SACM eventually expands into other areas (e.g. configuration,
>     etc.).  As a result, the consensus on the call was to leave the
>     terms in Section 2 of the Vulnerability Assessment Scenario, for
>     now, with the expectation that we may reassess this decision in the
>     future.
>
>     If you have any thoughts on this issue, please provide feedback by
>     May 31^st .  We are planning to have an updated version of the
>     scenario for June 8^th .
>
>     Thanks,
>
>     Danny
>
>     [1]https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/
>
>     [2]https://github.com/sacmwg/vulnerability-scenario/pull/3
>
>     [3]https://www.ietf.org/mail-archive/web/sacm/current/msg03958.html
>
>     [4]https://datatracker.ietf.org/doc/slides-interim-2016-sacm-3-1/(looks
>     like the slides are not available just yet, but, I suspect they
>     should be soon)
>
>     [5]https://datatracker.ietf.org/doc/draft-ietf-sacm-terminology/
>
>     _______________________________________________
>     sacm mailing list
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/listinfo/sacm
>
>
>
> _______________________________________________
> sacm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sacm
>

---

From: Jerome Athias <[email-masked]>
Date: Sat, 4 Jun 2016 09:09:48 +0300

Hi,

while I do understand use and reference to rfc4949, etc. I don't see
it as an efficient strategic approach while it is not really aligned
with Frameworks used in real life.

Organis/zations would tend to use NIST Risk Management Framework or ISO 27001.

An approach like:
NIST Cybersecurity Framework
NIST Risk Management Framework
NIST SP 800-53
...
would benefit from a consolidated centralized (if not Ontology)
Lexicon/Glossary such as
https://niccs.us-cert.gov/glossary
at least for general terms.

Vulnerability Assessment and Management

Definition: In the NICE Workforce Framework, cybersecurity work where
a person: Conducts assessments of threats and vulnerabilities,
determines deviations from acceptable configurations, enterprise or
local policy, assesses the level of risk, and develops and/or
recommends appropriate mitigation countermeasures in operational and
non-operational situations.

From: NICE Workforce Framework

Vulnerability Assessment (NIST SP 800-53A Rev 4)
[CNSSI 4009, Adapted]
Systematic examination of an information system or product to
determine the adequacy of security and privacy measures, identify
security and privacy
deficiencies, provide data from which to predict the effectiveness of
proposed security and privacy measures, and confirm the adequacy of
such measures after implementation.

Then, it could be used "In the context of SACM, " or "Within SACM the
use of the term is expanded", etc.

So I would be in favor of having all terms in Terminology I-D

2016-06-03 22:15 GMT+03:00 Henk Birkholz <[email-masked]>:
> Does this propose to define a separate collection task that focuses on on
> the acquisition of previously obtained endpoint attributes from a repository
> in the SACM domain? The endpoint attributes could be identical to the ones
> that can be acquired via the already defined collection task, but most
> certainly do not have to be! I would propose to "handle" the endpoint
> attributes in an identical way and to annotate where they come from, when
> they were acquired and similar metadata.
>
> values of the endpoint attributes against those expressed in the policy. (An
> evaluation result may become additional endpoint data)."
>
> Assessment is that it is a concatenated sequence of tasks defined by SACM.
> (Software) vulnerability assessment would be a specialization of that
> composition of tasks.
>
> Background: i.e., the vulnerability assessment would process a sub-set of
> endpoint attributes that can be acquired by the:
>
> - collection task (which is directly targeted at specific target endpoints)
> and the
> - target endpoint discovery task (which is supposed to be acquiring any kind
> indicator information (i.e. also endpoint attributes) that can be used to
> detect and re-detect target endpoints in a SACM domain) as a basis for the
> collection task.
>
>
> 3) Endpoint management capability: seems to be a little bit vague, still.
> Especially the term "management".
>
> This description seems to be more about a repository of expected (should) or
> possible (could) states that are based on configuration? Is an endpoint
> management capability (and its corresponding functions) also providing
> guidance(instructions)?
>
>
> 4) Vulnerability management capability: Here too, the term management seems
> to imply guidance (or management decision) of some sort.
>
> It should be made very clear what this capability actually provides; i.e.,
> which category of endpoint attributes and/or guidance. This would resolve
> most of the ambiguity in a simple way, I think. Essential information
> elements of the content should also be included in the SACM IM, I think.
>
>
> interconnections, I think. But I am not sure if that distinction is really
> relevant to SACM. Workflow seems to be the more general term and is less
> overloaded.
>
>
> Viele Grüße,
>
> Henk
>
>
>
>
>
>

---

From: Tony Rutkowski <[email-masked]>
Date: Sat, 04 Jun 2016 02:33:16 -0400

More likely they would be using the Critical Security Controls framework.

---

From: Adam Montville <[email-masked]>
Date: Mon, 6 Jun 2016 07:22:23 -0500

A bunch of comments inline, as contributor.

> On Jun 3, 2016, at 2:15 PM, Henk Birkholz <[email-masked]> wrote:
> 
> Hello all,
> 
> Does this propose to define a separate collection task that focuses on on the acquisition of previously obtained endpoint attributes from a repository in the SACM domain? The endpoint attributes could be identical to the ones that can be acquired via the already defined collection task, but most certainly do not have to be! I would propose to "handle" the endpoint attributes in an identical way and to annotate where they come from, when they were acquired and similar metadata.
> 
> As a result, this could enable a single method to conduct both tasks and simply things, I think.
> 
> The current definition of Endpoint Assessment is: "Evaluate the actual values of the endpoint attributes against those expressed in the policy. (An evaluation result may become additional endpoint data)."
> 
> Implied - but yet not explicitly defined - in the definition of Endpoint Assessment is that it is a concatenated sequence of tasks defined by SACM. (Software) vulnerability assessment would be a specialization of that composition of tasks.
> 
> Background: i.e., the vulnerability assessment would process a sub-set of endpoint attributes that can be acquired by the:
> 
> - collection task (which is directly targeted at specific target endpoints) and the
> - target endpoint discovery task (which is supposed to be acquiring any kind indicator information (i.e. also endpoint attributes) that can be used to detect and re-detect target endpoints in a SACM domain) as a basis for the collection task.
> 
> 
> 3) Endpoint management capability: seems to be a little bit vague, still. Especially the term "management".
> 
> This description seems to be more about a repository of expected (should) or possible (could) states that are based on configuration? Is an endpoint management capability (and its corresponding functions) also providing guidance(instructions)?

“Endpoint management” is not something we are likely to get into, but it’s one way to talk about the processes and workflows used by a given enterprise to manage their endpoints.  We’re getting into the more abstract layers of a security program that are generally governed by chosen or required control frameworks, and it is important that we have continuity from that level to the level at which SACM operates.

> 
> 
> 4) Vulnerability management capability: Here too, the term management seems to imply guidance (or management decision) of some sort.
> 
> The definition itself seems to describe a repository or a meta directory that provides information acquired, for example, by ROLIE?

The same things I said for “endpoint management” apply for “vulnerability management”.  The definition may describe a repository or meta directory, but those things are facets to vulnerability management and provide vulnerability management capabilities—the things that are likely required by the processes and workflows derived from control frameworks.

> It should be made very clear what this capability actually provides; i.e., which category of endpoint attributes and/or guidance. This would resolve most of the ambiguity in a simple way, I think. Essential information elements of the content should also be included in the SACM IM, I think.

This is an abstract term from which specializations can be derived, and I believe the specialization is where we can be very clear.  We should be able tot describe, however, that a “management capability” in the context of SACM is one that derives from the level of a control framework.

> It seems - taking into account 2-4 - that Vulnerability Management Capability is a part of Endpoint Management Capability?

I would say no.  Endpoint management capability is part of asset management capability.  Vulnerability management capability will certainly rely upon endpoint management capabilities, but I would not view it as part of endpoint management, coming at this from a control framework perspective.  Most frameworks will say things like, "manage your endpoints so you know where everything is and what you have”, and then say things like, “for all your endpoints, make sure you’re addressing vulnerabilities”.

> 
> 
> 7) process & workflow:
> 
> A workflow might involve multiple processes with various dependences and interconnections, I think. But I am not sure if that distinction is really relevant to SACM. Workflow seems to be the more general term and is less overloaded.

I am partial to the term “workflow” over “process”.

>> *To:* Haynes, Dan <[email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>>
>> *Cc:* [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
>>    On May 17, 2016, at 1:15 PM, Haynes, Dan <[email-masked] <mailto:[email-masked]>
>>    <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>> 
>>    [1]https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/ <https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/>
>> 
>>    [2]https://github.com/sacmwg/vulnerability-scenario/pull/3 <https://github.com/sacmwg/vulnerability-scenario/pull/3>
>> 
>>    [3]https://www.ietf.org/mail-archive/web/sacm/current/msg03958.html <https://www.ietf.org/mail-archive/web/sacm/current/msg03958.html>
>> 
>>    [4]https://datatracker.ietf.org/doc/slides-interim-2016-sacm-3-1/ <https://datatracker.ietf.org/doc/slides-interim-2016-sacm-3-1/>(looks
>>    [5]https://datatracker.ietf.org/doc/draft-ietf-sacm-terminology/ <https://datatracker.ietf.org/doc/draft-ietf-sacm-terminology/>
>> 
>>    [email-masked] <mailto:[email-masked]> <mailto:[email-masked] <mailto:[email-masked]>>
>>    https://www.ietf.org/mailman/listinfo/sacm <https://www.ietf.org/mailman/listinfo/sacm>
>> 
>> 
>> 
>> https://www.ietf.org/mailman/listinfo/sacm <https://www.ietf.org/mailman/listinfo/sacm>
>> 
> 
> https://www.ietf.org/mailman/listinfo/sacm <https://www.ietf.org/mailman/listinfo/sacm>

---

From: Adam Montville <[email-masked]>
Date: Mon, 6 Jun 2016 07:23:28 -0500

Yes, some organizations may start with the CIS Critical Controls.

Full disclosure: My org manages development of those controls.

> On Jun 4, 2016, at 1:33 AM, Tony Rutkowski <[email-masked]> wrote:
>

---

From: "Haynes, Dan" <[email-masked]>
Date: Tue, 7 Jun 2016 01:59:24 +0000

> Does this propose to define a separate collection task that focuses on on the
> acquisition of previously obtained endpoint attributes from a repository in
> the SACM domain? The endpoint attributes could be identical to the ones

[danny]: No, I don't think this proposes a separate collection task.  It would just be the standard collection task already defined.  That is, once the vulnerability management capability realizes that it does not have certain information that it needs for an assessment (as specified in the vulnerability detection data), it would call out to a collector, via a collection task, with guidance on what information to collect. Once the task completes and the collector provides that information, the vulnerability management capability can continue with the assessment. 

> The current definition of Endpoint Assessment is: "Evaluate the actual values
> of the endpoint attributes against those expressed in the policy.

[danny]: I agree with this.

> (should) or possible (could) states that are based on configuration? Is an
> endpoint management capability (and its corresponding functions) also

[danny]: I was under the impression that it would just provide endpoint information and not guidance. With that said, I think an endpoint management capability could be controlled by guidance.

> The definition itself seems to describe a repository or a meta directory that
> provides information acquired, for example, by ROLIE?

[danny]: Yes, the vulnerability detection data is the guidance that drives how it carries out an assessment.

> information elements of the content should also be included in the SACM

[danny]: I left it particularly vague (i.e. information) because we have yet to define all the management capabilities we want :).  Based on the discussion during the last virtual interim meeting, I was under the impression that we would define new ones as we focus on additional areas (configuration, hardware, maybe something for guidance?, etc.). 

> Capability is a part of Endpoint Management Capability?

[danny]: I am not sure.  I suspect it is up to the implementer whether or not they want standalone capabilities or a monolithic one.

> really relevant to SACM. Workflow seems to be the more general term and

[danny]: So, is that a +1 for workflow :)?

Thanks,

Danny

> > "vulnerability description information". One approach might be to
> > replace "vulnerability description information" with "information
> > and/or firmware on an endpoint."  It might also make sense to highlight
> > this as a specialization of "assessment".  As a side note, does anyone
> > changes which is perfectly fine - just something to think about :).
> >
> > Right now, we have definitions for "endpoint management capability" and
> > "vulnerability management capability". To support promoting these, we
> > may want to create a more general "management capability" term and
> then
> > include "endpoint management capability" and "vulnerability management
> > capability" as specializations of that term.  A definition of
> > "management capability" might read something like "An enterprise IT
> > not be driven by guidance".
> >
> > I'll echo my statements from the virtual interim by pointing out that
> > terms "promoted" into the terminology draft sooner rather than later.
> >     During today's virtual interim meeting, we discussed various open
> >     here [2][3].  The slide's from today's meeting can be found here [4].
> >
> >     .Vulnerability description information
> >
> >     .Vulnerability detection data
> >
> >     .Endpoint management capability
> >
> >     .Vulnerability management capability
> >
> >     .Vulnerability assessment
> >
> >     .Targeted collection
> >
> >     "endpoint management capability" and "vulnerability management
> >     capability", while specific, have the potential to become broader as

---

From: Jerome Athias <[email-masked]>
Date: Mon, 13 Jun 2016 08:44:45 +0300

Hi,

7) "Workflow or process engine" would, IMHO, be ITIL compliant

"More likely they would be using the Critical Security Controls framework."
Right, I understand this statement in a maturity level context, etc.
(I appreciate a lot the CSC, and its usage in SACM)
I'm just thinking about providing metrics from SACM (to be aligned
with strategic levels)
But I don't want to add 'complexity'

Regards

---

From: "Haynes, Dan" <[email-masked]>
Date: Thu, 30 Jun 2016 20:16:28 +0000

I just wanted to follow on this thread.  Based on the 6/15 virtual interim meeting [1], there seemed to be consensus that we would leave the terms from the Vulnerability Assessment Scenario I-D [2] as well as include them in the Terminology I-D [3].  The reasoning behind this was:

 1) Terms would be included in the Vulnerability Assessment Scenario I-D for improved-readability (don't need to keep looking at Terminology I-D to know what the terms mean)
 2) Terms would be included in the Terminology I-D to capture the WG's thinking at a specific point in time as well as enable the use of the terms in other I-Ds where appropriate

Does this approach reasonable to others?  Are there any major objections?  Please let me know by next Wednesday July 6th.

Thanks,

Danny

[1] https://www.ietf.org/proceedings/interim-2016-sacm-04/minutes/minutes-interim-2016-sacm-4
[2] https://datatracker.ietf.org/doc/draft-ietf-sacm-vuln-scenario/
[3] https://datatracker.ietf.org/doc/draft-ietf-sacm-terminology/

> -----Original Message-----
> From: Jerome Athias [mailto:[email-masked]]
> Sent: Monday, June 13, 2016 1:45 AM
> To: Haynes, Dan <[email-masked]>
> Cc: [email-masked]
> Subject: Re: [sacm] FOR REVIEW: Vulnerability Assessment Scenario Issue #1
> - Managing Terminology
> 
> Hi,
> 
> (I appreciate a lot the CSC, and its usage in SACM) I'm just thinking about
> providing metrics from SACM (to be aligned with strategic levels) But I don't
> want to add 'complexity'
> 
> Regards