Subject: [sacm] Types of content

From: "Waltermire, David A." <[email-masked]>
Date: Wed, 5 Feb 2014 20:37:34 +0000

I'm working through the UC draft addressing discussion items from yesterday and the previous comments we have received on the list on -05.

Related to section 2.1.1, we have had many previous discussions around the notional idea of content. There has been general agreement that we need to be more specific. Please find below an attempt to put some "bounds" around different types of content.

Is this hitting the mark, or do we need to say something more/different?

The following is a snippet from a possible update to section 2.1.1 with some additional description:

I'd appreciate any thoughts, concerns, objections before I include the changes in the draft.

Thanks,
Dave

2.1.1.  Define, Publish, Query and Retrieve Content

   This use case describes the need for content to be defined and
   published to a data store, as well as queried and retrieved from the
   data store for the explicit use of posture collection and evaluation.

   The term "content" is a general concept that refers to any data
   expression that may be used as part of the process of collecting and
   evaluating endpoint posture.  Typical types of content include:

   o  Dictionaries of posture attributes (e.g. software installation,
      configuration, privileges) that may be collected from a given
      endpoint type that relate to installed hardware, installed
      software, software configurations, accounts, privileges, and other
      operational state.

   o  Expressions that provide instructions for using a specific API,
      management protocol, or other mechanisms to collect specific
      posture attributes from specific device types.

   o  Policies that define the selected posture attributes for which
      values must be collected to support one or more posture
      assessments.  This may include information that targets collection
      to a specific set of endpoints and instructions on the frequency
      and/or change events (e.g., configuration changes, software
      installation/uninstallation, account/privilege modifications) that
      will trigger collection of one or more posture attributes.

   o  Policies that define the logical criteria to evaluate collected
      posture attribute values to determine if they meet organizational
      requirements.  This may include information that targets
      evaluation to a specific set of endpoints.

---

From: Ira McDonald <[email-masked]>
Date: Wed, 5 Feb 2014 16:59:06 -0500

Hi Dave,

Sorry I couldn't join the SACM teleconference yesterday (stuck in an IEEE
call).

I would be much happier if SACM documents used a term like "SA content"
for what you defined below.

Using lowercase naked "content" is going to translate very poorly across
other
natural languages and become confusing to native English speakers (who don't
happen to be SACM geeks).

Cheers,
- Ira

Ira McDonald (Musician / Software Architect)
Co-Chair - TCG Trusted Mobility Solutions WG
Chair - Linux Foundation Open Printing WG
Secretary - IEEE-ISTO Printer Working Group
Co-Chair - IEEE-ISTO PWG Internet Printing Protocol WG
IETF Designated Expert - IPP & Printer MIB
Blue Roof Music / High North Inc
http://sites.google.com/site/blueroofmusic
http://sites.google.com/site/highnorthinc
mailto: [email-masked]
Winter  579 Park Place  Saline, MI  48176  734-944-0094
Summer  PO Box 221  Grand Marais, MI 49839  906-494-2434

On Wed, Feb 5, 2014 at 3:37 PM, Waltermire, David A. <
[email-masked]> wrote:

>  I'm working through the UC draft addressing discussion items from
> yesterday and the previous comments we have received on the list on -05.
>
>
>
> Related to section 2.1.1, we have had many previous discussions around the
> notional idea of content. There has been general agreement that we need to
> be more specific. Please find below an attempt to put some "bounds" around
> different types of content.
>
>
>
> _______________________________________________
> sacm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sacm
>
>

---

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Wed, 5 Feb 2014 22:24:55 +0000

Sounds like a good summary of what "content" could mean.
Panos

From: sacm [mailto:[email-masked]] On Behalf Of Waltermire, David A.
Sent: Wednesday, February 05, 2014 10:38 PM
To: [email-masked]
Subject: [sacm] Types of content

Thanks,
Dave

---

From: Stephen Hanna <[email-masked]>
Date: Wed, 5 Feb 2014 23:10:28 +0000

Would "dictionaries of posture attributes" include attribute values that have been collected from endpoints?

We want to have the ability to collect attribute values from endpoints and "persist" them, as described in section 2.1.3. So we'll need to have a repository for collected and persisted attribute values.

If collected endpoint attribute values are considered content, let's say so. If not, let's say that explicitly.

My personal view is that collected endpoint attribute values should be considered content. They may be a different kind of content from the others due to larger volume and higher change rate. For that reason, people may choose to put them into a different repository and maybe even use a different query protocol for that repository to munge the large quantities of data. But they're still content.

Thanks,

Steve

From: sacm [mailto:[email-masked]] On Behalf Of Panos Kampanakis (pkampana)
Sent: Wednesday, February 05, 2014 5:25 PM
To: Waltermire, David A.; [email-masked]
Subject: Re: [sacm] Types of content

From: sacm [mailto:[email-masked]] On Behalf Of Waltermire, David A.
Sent: Wednesday, February 05, 2014 10:38 PM
To: [email-masked]<mailto:[email-masked]>
Subject: [sacm] Types of content

Thanks,
Dave

---

From: "Waltermire, David A." <[email-masked]>
Date: Wed, 5 Feb 2014 23:33:08 +0000

Dictionaries of posture attributes was not intended to include attribute values. IETF examples of this concept might include attributes in a MIB or a NETMOD model.

I agree that we need to "persist" posture attribute values. I have always been on the fence personally about calling collection and evaluation results "content". I agree 100% that we need to be crystal clear about what we mean when we use the terms "content" and "content repository". If we choose to lump all of this together as "content", we will need to include some discussion in the use case document contrasting different types of content to drive discussion of content type-specific requirements.

I also personally like Ira's feedback on using a more specific term(s) instead of the more general "content".

I also share the same personal view around interacting with stores of collected posture attribute values and evaluation results. The larger data volumes and rate of change will likely require different approaches. I also believe that the temporal nature of this data, the need to persist historic information, will be more significant than other types of data that we may call "content".

Are there other opinions on this? Does anyone feel strongly that collected posture attribute values and evaluation results SHOULD NOT be considered content?

Thanks,
Dave

From: Stephen Hanna [mailto:[email-masked]]
Sent: Wednesday, February 05, 2014 6:10 PM
To: Panos Kampanakis (pkampana); Waltermire, David A.; [email-masked]
Subject: RE: Types of content

Thanks,

Steve

From: sacm [mailto:[email-masked]] On Behalf Of Panos Kampanakis (pkampana)
Sent: Wednesday, February 05, 2014 5:25 PM
To: Waltermire, David A.; [email-masked]<mailto:[email-masked]>
Subject: Re: [sacm] Types of content

Thanks,
Dave

---

From: Gunnar Engelbach <[email-masked]>
Date: Wed, 05 Feb 2014 19:37:14 -0500

I would prefer to reserve the term "content" for the instructional 
documentation.  That is, the documents that cause an action, be it 
legislation, prose guidance, or a set of machine instructions that 
implement that guidance.

We typically use "system characteristics" when referring to collected 
items, and variations on "endpoint attributes" has been used frequently 
on this list.  System characteristics are the product of applying 
content to an endpoint.

Collectively it's all data.  Is there a desire to have a general term 
for all data that could flow through a SACM implementation but is 
specific to SACM?

Also, while I agree with the need to allow the ability to persist 
collected data items (and the results of posture evaluations), I don't 
see that as a requirement.

On 2/5/2014 6:33 PM, Waltermire, David A. wrote:
> Dictionaries of posture attributes was not intended to include attribute
> values. IETF examples of this concept might include attributes in a MIB
> or a NETMOD model.
>
> I agree that we need to “persist” posture attribute values. I have
> always been on the fence personally about calling collection and
> evaluation results “content”. I agree 100% that we need to be crystal
> clear about what we mean when we use the terms “content” and “content
> repository”. If we choose to lump all of this together as “content”, we
> will need to include some discussion in the use case document
> contrasting different types of content to drive discussion of content
> type-specific requirements.
>
> I also personally like Ira’s feedback on using a more specific term(s)
> instead of the more general “content”.
>
> I also share the same personal view around interacting with stores of
> collected posture attribute values and evaluation results. The larger
> data volumes and rate of change will likely require different
> approaches. I also believe that the temporal nature of this data, the
> need to persist historic information, will be more significant than
> other types of data that we may call “content”.
>
> Are there other opinions on this? Does anyone feel strongly that
> collected posture attribute values and evaluation results SHOULD NOT be
> considered content?
>
> Thanks,
>
> Dave
>
> *From:*Stephen Hanna [mailto:[email-masked]]
> *Sent:* Wednesday, February 05, 2014 6:10 PM
> *To:* Panos Kampanakis (pkampana); Waltermire, David A.; [email-masked]
> *Subject:* RE: Types of content
>
> Would “dictionaries of posture attributes” include attribute values that
> have been collected from endpoints?
>
> We want to have the ability to collect attribute values from endpoints
> and “persist” them, as described in section 2.1.3. So we’ll need to have
> a repository for collected and persisted attribute values.
>
> If collected endpoint attribute values are considered content, let’s say
> so. If not, let’s say that explicitly.
>
> My personal view is that collected endpoint attribute values should be
> considered content. They may be a different kind of content from the
> others due to larger volume and higher change rate. For that reason,
> people may choose to put them into a different repository and maybe even
> use a different query protocol for that repository to munge the large
> quantities of data. But they’re still content.
>
> Thanks,
>
> Steve
>
> *From:*sacm [mailto:[email-masked]] *On Behalf Of *Panos
> Kampanakis (pkampana)
> *Sent:* Wednesday, February 05, 2014 5:25 PM
> *To:* Waltermire, David A.; [email-masked] <mailto:[email-masked]>
> *Subject:* Re: [sacm] Types of content
>
> Sounds like a good summary of what “content” could mean.
>
> Panos
>
> *From:*sacm [mailto:[email-masked]] *On Behalf Of *Waltermire,
> David A.
> *Sent:* Wednesday, February 05, 2014 10:38 PM
> *To:* [email-masked] <mailto:[email-masked]>
> *Subject:* [sacm] Types of content
>
> I’m working through the UC draft addressing discussion items from
> Related to section 2.1.1, we have had many previous discussions around
> the notional idea of content. There has been general agreement that we
> need to be more specific. Please find below an attempt to put some
> “bounds” around different types of content.
>
> I’d appreciate any thoughts, concerns, objections before I include the
> changes in the draft.
>
> Thanks,
>
> Dave
>

---

From: Stephen Hanna <[email-masked]>
Date: Thu, 6 Feb 2014 02:13:44 +0000

The terms "content" and "data" are just too general. "SA content"
isn't much better. It seems to include any kind of data related
to security automation. In the long run, that might include lots
of things like vulnerability reports and alerts that an intrusion
has been detected and remediation or mitigation instructions.
Also reports on endpoint compliance and attributes that have
been collected from endpoints and maybe persisted. It's just
not useful to use one term for all those kinds of things.

I think that we need a new, more specific term for the things
that Dave described below: attribute identifiers, lists of 
attributes that are to be collected and when and perhaps how,
and policies about what attribute values are acceptable for
different kinds or groups of endpoints. I suggest that we
use the collective term "guidance" for these things. This
indicates that these things (and maybe others to be added
later) guide the attribute collection and evaluation process.

We also need to have a term for attribute values that have
been collected from endpoints. I vote for "collected attributes".
These attributes might be persisted (stored) or maybe not.
Either way, they are attribute values that have been collected
from endpoints.

Thanks,

Steve

> -----Original Message-----
> From: sacm [mailto:[email-masked]] On Behalf Of Gunnar Engelbach
> Sent: Wednesday, February 05, 2014 7:37 PM
> To: Waltermire, David A.; Stephen Hanna; Panos Kampanakis (pkampana);
> [email-masked]
> > I agree that we need to "persist" posture attribute values. I have
> > evaluation results "content". I agree 100% that we need to be crystal
> > clear about what we mean when we use the terms "content" and "content
> > repository". If we choose to lump all of this together as "content",
> we
> > I also personally like Ira's feedback on using a more specific
> term(s)
> > instead of the more general "content".
> >
> > other types of data that we may call "content".
> >
> > Would "dictionaries of posture attributes" include attribute values
> that
> > and "persist" them, as described in section 2.1.3. So we'll need to
> have
> > If collected endpoint attribute values are considered content, let's
> say
> > so. If not, let's say that explicitly.
> >
> > quantities of data. But they're still content.
> >
> > Thanks,
> >
> > Steve
> >
> > "bounds" around different types of content.
> >
> > The following is a snippet from a possible update to section 2.1.1
> with
> > some additional description:
> >
> > I'd appreciate any thoughts, concerns, objections before I include
> the
> > changes in the draft.
> >
> > Thanks,
> >
> > Dave
> >

---

From: Gunnar Engelbach <[email-masked]>
Date: Wed, 05 Feb 2014 22:34:20 -0500

So "Guidance" is input and "collected attributes" is the immediate product?

That works.  And is line with common usage.

And for the final result of applying the collected attributes to the 
guidance, how about "assessment results?"

Is there a need for anything more granular?  Mention of other terms 
commonly used for the same thing?

What I frequently see:

Input:
   Guidance
   Content
   Checklist
   Questionnaire
   Data stream
   Benchmark
   Definitions

Product:
   Collected Attributes
   System Characteristics
   Assessment Results
   Posture

---

From: Stephen Hanna <[email-masked]>
Date: Thu, 6 Feb 2014 03:48:17 +0000

> So "Guidance" is input and "collected attributes" is the immediate

Right. At least, that's one way of describing it.
I'm thinking that guidance would include all the
things that Dave was previously calling "content".

> That works.  And is line with common usage.

Good! I'm glad to hear that it makes sense to you.

> guidance, how about "assessment results?"

Yes, I like that also.

> Is there a need for anything more granular?

Probably eventually. But there also seems to be
a need for broad terms since people want to be
able to put all the "guidance" into one repository
(perhaps) and have a different kind of repository
for "collected attributes" and maybe also
"assessment results".

> Mention of other terms commonly used for the
> same thing?

Maybe. Listing alternative terms in common usage
today can be confusing since different people may
use those terms differently.

Thanks,

Steve

> >>> collected posture attribute values and evaluation results SHOULD
> NOT
> >> be
> >>> considered content?
> >>>
> >>> Thanks,
> >>>
> >>> Dave
> >>>
> >>> yesterday and the previous comments we have received on the list on
> -
> >> 05.
> >>>
> >>>      published to a data store, as well as queried and retrieved
> from
> >> the
> >>>
> >>>      expression that may be used as part of the process of
> collecting
> >> and
> >>>
> >>>         software, software configurations, accounts, privileges,
> and
> >> other
> >>>
> >>>         operational state.
> >>>
> >>>         installation/uninstallation, account/privilege
> modifications)
> >> that
> >>>

---

From: "Waltermire, David A." <[email-masked]>
Date: Thu, 6 Feb 2014 15:29:34 +0000

This has been a great discussion. Some additional thoughts are below, along with some questions.

> Sent: Wednesday, February 05, 2014 10:48 PM
> To: Gunnar Engelbach; Waltermire, David A.; Panos Kampanakis (pkampana);
> [email-masked]
> Subject: RE: [sacm] Types of content
> 
> Gunnar Engelbach wrote:
> I'm thinking that guidance would include all the things that Dave was
> previously calling "content".

We have three basic processes defined in the use cases: Assessment Planning, Collection, and Evaluation. For each there are a set of instructions that may be used as input and a set of outputs that result from the process. The following listing of data inputs and outputs is derived from the use case/building block text. This represents a fairly comprehensive list of various forms of "content" that we will be dealing with.

The inputs for Assessment Planning include:
- Characterizations of endpoints (BB: Endpoint Characterization) - Endpoint metadata that describes the endpoint type, organizationally expected function/role, etc.
- Posture attributes values for HW/SW inventory and accounts/privileges (BB: Endpoint Component Inventory) - This provides context to identify what posture attributes should be collected as part of the assessment. 
- Assessment applicability statements (BB: Identify Endpoint Targets) - This supports the targeting concept allowing characterizations and collected posture attribute values to be evaluated to determine what set of endpoints should be assessed.
- Evaluation instructions (BB: Posture Attribute Identification) - Use to determine the set of posture attributes for which values are required for evaluation.

The outputs for the Assessment Planning include:
- An identified set of endpoints (BB: Identify Endpoint Targets) - The endpoints to assess.
- A set of identified posture attributes (BB: Posture Attribute Identification) - The required posture attributes needed to perform the evaluation.

At this point Collection will occur to collect the necessary posture attributes. This may have been done historically completely or in-part. If there is incomplete or stale information, this may be collected immediately or scheduled as appropriate.

The inputs for Collection include the following (provided by the outputs of assessment planning):
- An identified set of endpoints (BB: Identify Endpoint Targets) - The endpoints to assess.
- A set of identified posture attributes (BB: Posture Attribute Identification) - The required posture attributes needed to perform the evaluation.
- Collection instructions (BB: Collection Content Acquisition) - Directives on how the posture attributes should be collected. This may include: events to monitor, scheduling info, instructions for using specific models/protocols, etc.

The outputs for Collection include the following:
- A set of posture attributes and associated values (BB: Posture Attribute Value Collection) - The result of collection.

At this point evaluation can occur.

The inputs for Posture Evaluation include the following:
- A set of posture attributes and associated values (BB: Posture Attribute Value Query) - The previously collected posture attribute values. This may not require a query if the data is fed directly to an evaluator after collection.
- Evaluation instructions (BB: Evaluation Content Acquisition) - Directives on how to evaluate posture attribute values. This will likely include comparison operators and logical constructs. Additional instructions might indicate the required result output format.

The outputs for Posture Evaluation include the following:
- Assessment Results (BB: Posture Attribute Evaluation) - The resulting output generated by the evaluation process.

The big questions are:
1) Is this a complete list?
2) What of this will be stored or have the potential to be stored in a data repository?
3) What groupings of content types do we want to define?

Any thoughts are appreciated.

Thanks,
Dave

> Probably eventually. But there also seems to be a need for broad terms since
> people want to be able to put all the "guidance" into one repository
> (perhaps) and have a different kind of repository for "collected attributes"
> and maybe also "assessment results".
> 
> > Mention of other terms commonly used for the same thing?
> 
> Maybe. Listing alternative terms in common usage today can be confusing
> since different people may use those terms differently.
> 
> Thanks,
> 
> Steve
> 
> >
> >
> > What I frequently see:
> >
> > Input:
> >    Guidance
> >    Content
> >    Checklist
> >    Questionnaire
> >    Data stream
> >    Benchmark
> >    Definitions
> >
> >
> > Product:
> >    Collected Attributes
> >    System Characteristics
> >    Assessment Results
> >    Posture
> >
> >
> >
> >
> > > isn't much better. It seems to include any kind of data related to
> > > security automation. In the long run, that might include lots of
> > > things like vulnerability reports and alerts that an intrusion has
> > > been detected and remediation or mitigation instructions.
> > > Also reports on endpoint compliance and attributes that have been
> > > collected from endpoints and maybe persisted. It's just not useful
> > > to use one term for all those kinds of things.
> > >
> > > I think that we need a new, more specific term for the things that
> > > Dave described below: attribute identifiers, lists of attributes
> > > that are to be collected and when and perhaps how, and policies
> > > about what attribute values are acceptable for different kinds or
> > > groups of endpoints. I suggest that we use the collective term
> > > "guidance" for these things. This indicates that these things (and
> > > maybe others to be added
> > > We also need to have a term for attribute values that have been
> > > collected from endpoints. I vote for "collected attributes".
> > > Either way, they are attribute values that have been collected from
> > > endpoints.
> > >
> > > Thanks,
> > >
> > > Steve
> > >
> > >>> values. IETF examples of this concept might include attributes in
> > >>> a
> > >> MIB
> > >>> or a NETMOD model.
> > >>>
> > >>> need to persist historic information, will be more significant
> > >>> than other types of data that we may call "content".
> > >>>
> > >>> My personal view is that collected endpoint attribute values
> > >>> should
> > >> be
> > >>> people may choose to put them into a different repository and
> > >>> maybe
> > >> even
> > >>> the notional idea of content. There has been general agreement
> > >>> that
> > >> we

---

From: "Waltermire, David A." <[email-masked]>
Date: Wed, 12 Feb 2014 00:13:25 +0000

I'd like to get the discussion around this topic to a point where I have clear editorial direction on what text we want included in the updated use case draft for this Friday's submission cutoff deadline.

Based on the mailing list discussions to date, I'd like to propose the following categories of security automation data for inclusion in the use cases draft: Guidance, Endpoint/Asset Attribute Data, and Processing Artifacts.

The following sections roughly define each of the proposed categories. For each section some insights are provided into the how the data might be sourced and how often it may change. I have also included examples of the classes of information for each category with mappings to the use cases and building blocks they support to insure we have the necessary coverage.

To move forward I need answers to the following questions:

1) Do you like these groupings? If not, is there a better set of groupings?
2) Do you like the labels for each of the groups? If not, do you have a better label(s)?
3) Are there other types of content that should be included?
4) What level of detail should be included in the use cases draft regarding this discussion? What I have below is likely too much detail.

Thanks,
Dave

Guidance:
------------
The term "guidance" was proposed by Steve Hanna and was supported by Gunnar Engelbach on the list. Steve proposed that this concept encompasses "attribute identifiers, lists of attributes that are to be collected and when and perhaps how, and policies about what attribute values are acceptable for different kinds or groups of endpoints. This indicates that these things (and maybe others to be added later) guide the attribute collection and evaluation process." I would also add that this should include instructions supporting data-driven evaluation processes.

This data tends to change in units of months and days. In cases where assessments are made more dynamic, it may be necessary to handle changes in the scope of hours or minutes. This data will typically be provided by large organizations, product vendors, and some 3rd-parties. Thus, it will tend to be shared across large enterprises and customer communities. In some cases access may be controlled to specific authenticated users. In other cases, the data may be provided broadly with little to no access control.

This would include:

- Applicability Statements: Data the identifies the conditional applicability of an assessment (and related assessment "guidance") based on automated collection of endpoint posture attribute values and other endpoint characterizations.
    + Endpoint Identification and Assessment Planning BB: Identify Endpoint Targets - This supports the targeting concept allowing characterizations and collected posture attribute values to be evaluated to determine what set of endpoints should be assessed.

- Collection Instructions: Data that drives the identification of the set of posture attributes needed to perform an assessment and the collection of posture attributes.
    + Endpoint Identification and Assessment Planning BB: Posture Attribute Identification - Used to determine the set of posture attributes for which values must be collected (or retrieved) to support the evaluation.
    + Endpoint Posture Attribute Value Collection BB: Collection Content Acquisition - Data is retrieved based on its relationship to the assessment and other applicability concepts to drive collection activities.
    + Endpoint Posture Attribute Value Collection BB: Posture Attribute Value Collection - Directives on how the posture attributes should be collected. This may include: events to monitor, scheduling info, instructions for using specific models/protocols, etc.

- Evaluation instructions: Data the identifies what endpoint posture attribute values are to be evaluated and how this evaluation should be performed.
    + Endpoint Identification and Assessment Planning BB: Posture Attribute Identification - Use to determine the set of posture attributes for which values are required for evaluation.
    + Posture Evaluation BB: Evaluation Content Acquisition - Data is retrieved based on its relationship to the assessment and other applicability concepts to drive evaluation activities.
    + Posture Evaluation BB: Posture Attribute Evaluation - Instructs the evaluation of collected endpoint posture attribute values. This will likely include comparison operators and logical constructs. Additional instructions might indicate the required result output format.

Our information model will need to relate the concepts of applicability statements, collection instructions, and evaluation instructions in support of assessment activities.

Endpoint/Asset Attribute Data:
--------------------------------------

This category includes contextual metadata pertaining to endpoints and the assets that they are composed of (e.g., hardware, software, accounts), and collected endpoint posture attributes, their associated values, and related context.

This data tends to change in units of days, minutes, a seconds with posture attribute values typically changing more frequently than endpoint characterizations. This data tends to be organizationally and endpoint specific, with specific operational groups of endpoints tending to exhibit similar attribute profiles. This data will generally not be shared outside an organizational boundary and will generally require authentication with specific access controls.

This would include:

- Characterizations of endpoints: Endpoint metadata that describes the endpoint type, organizationally expected function/role, etc.
    + Endpoint Identification and Assessment Planning BB: Endpoint Characterization - Gathered through automated posture attribute collection, manual data entry/query, and interactions with other data stores.

- Collected Posture Attributes: Posture attributes and associated values collected from endpoints representing hardware, software inventory, configurations, accounts/privileges, and other aspects of endpoint state.
    + Endpoint Identification and Assessment Planning BB: Endpoint Component Inventory - This provides context to identify what posture attributes should be collected as part of the assessment based on the current endpoint posture (e.g., what software is installed).
    + Endpoint Posture Attribute Value Collection BB: Posture Attribute Value Collection - For each targeted endpoint, the set of collected posture attributes and associated values. The result of collection.
    + Posture Evaluation BB: Posture Attribute Value Query - The retrieval of a set of previously collected posture attributes and associated values. This may not be required if collected data is fed directly to an evaluator after collection.

An important dimension here may be the separation between actual and expected posture attribute values. It may be possible to express guidance in a way that allows expected values to parameterized. In this case it will be necessary to allow for organizationally expected values to be associated with defined guidance (perhaps provided by a different party).

Our information model will need to relate the concepts of endpoint characterizations and collected posture information to the types of guidance data.

Processing Artifacts:
-------------------------

This type of data is generated by and is specific to an individual assessment process. This data may be used as part of the interactions between SACM architectural components to drive and coordinate collection and evaluation activities. Its lifespan will be bounded by the lifespan of the assessment. It may also be exchanged and stored to provide historic context around an assessment activity.

This would include:

- Endpoint Targets: An identified set of endpoints for which an assessment should be performed.
    + Endpoint Identification and Assessment Planning BB: Identify Endpoint Targets - The endpoints targeted for an assessment.

- Identified Posture Attributes
    + Endpoint Identification and Assessment Planning BB: Posture Attribute Identification - The required posture attributes that need to be collected to perform the evaluation.

- Assessment Results: The resulting output generated by the evaluation process.
    + Posture Evaluation BB: Posture Attribute Evaluation - The results produced through the comparison of actual collected posture attribute
         values against their expected state. The specific output an evaluation criteria may be specified by the required evaluation instructions.

Our information model will need to relate these concepts to those provided in the other two categories. I know this is not saying much, but I am tired and ready to end this email. ;-)

---

From: Gunnar Engelbach <[email-masked]>
Date: Tue, 11 Feb 2014 21:27:03 -0500

> 1) Do you like these groupings? If not, is there a better set of groupings?

Categorizing in this manner makes a lot of sense to me.  I can see an 
argument for lumping Attribute Artifacts and Processing Artifacts 
together, but I would prefer that they not be.

> 2) Do you like the labels for each of the groups? If not, do you have a better label(s)?

"Guidance" is great.

I'd like to simplify "Endpoint/Asset Attribute Data" to just "Attribute 
Data."  "Collected Artifacts" also works for me, except that it's easier 
to confuse the terms if two of them have "Artifacts" in their name.

"Processing Artifacts" isn't really clicking with me, but I can't think 
of anything better.  Before reading your descriptive text I knew from 
the name what you were getting at, so it's got that going for it.

> 3) Are there other types of content that should be included?

Yes.  Guidance should also include the guidance that the automated 
guidance is meant to fulfill.  The prose guidance that the STIGS are 
drawn from, for example.  Vendor best practices documents.  Legal 
guidance and legislation.  Guidance can also be instructional materials 
related to the other content.

> 4) What level of detail should be included in the use cases draft regarding this discussion? What I have below is likely too much detail.

I have no problem with this level of detail, and more, but if it's too 
much then maybe the bulk of it could live in an appendix.

---

From: "Watson, Kimberly" <[email-masked]>
Date: Wed, 12 Feb 2014 17:30:30 +0000

The categorization makes sense. Processing Artifacts and Attribute Data must be associated, but they are very separate things, so I prefer to keep them apart.

Agree with removing endpoint/asset and sticking with Attribute Data to keep things simpler.
I could not come up with a better label for Processing Artifacts - since they seems to represent meta-data about collection and/or evaluation such that individual assessments can be grouped/evaluated/reported in an enterprise context.
Guidance is fine label.

There is definitely a hierarchy of policy, so it makes perfect sense to me that there is a hierarchy of guidance. It might just be best to acknowledge that hierarchies can be represented and make sure we have content that supports that...

There is a need to be able to know what guidance was used for any processing, since it can be dynamic (as stated in the text). That means we now have what was assessed, what it was assessed against, what was used, and what the results were...i think we are missing when the attributes were collected and when the attributes were evaluated. I think this is necessary to support the independent aspects of the two processes to support assessments. I fully expect guidance to specify how old collected data can be to be used for evaluation - which means we need to be able to report a result that assessment didn't occur because collection did not occur and previous collection was too old. Maybe a time artifact  is already included (by default), but I want to make sure it isn't  in there because it doesn't need to be...

I have no background to make any suggestion about this, but I like the level of detail used here to describe things, and I would be willing to accept even more detail where necessary to make the delineating/defining point.

--
Kim Watson
Office of Cybersecurity and Communications
Federal Network Resilience
Blackberry: (202) 494-2534

-----Original Message-----
From: Waltermire, David A. [mailto:[email-masked]] 
Sent: Tuesday, February 11, 2014 7:13 PM
To: Stephen Hanna; Gunnar Engelbach; Panos Kampanakis (pkampana); [email-masked]
Subject: Re: [sacm] Types of content

Thanks,
Dave

Guidance:
------------

This would include:

This would include:

Processing Artifacts:

This would include:

> To: Gunnar Engelbach; Waltermire, David A.; Panos Kampanakis 
> (pkampana); [email-masked]
> Probably eventually. But there also seems to be a need for broad terms 
> since people want to be able to put all the "guidance" into one 
> repository
> Maybe. Listing alternative terms in common usage today can be 
> confusing since different people may use those terms differently.
> 
> Thanks,
> 
> Steve
> 
> >
> >
> > What I frequently see:
> >
> > Input:
> >    Guidance
> >    Content
> >    Checklist
> >    Questionnaire
> >    Data stream
> >    Benchmark
> >    Definitions
> >
> >
> > Product:
> >    Collected Attributes
> >    System Characteristics
> >    Assessment Results
> >    Posture
> >
> >
> >
> >
> > >> I would prefer to reserve the term "content" for the 
> > >> instructional documentation.  That is, the documents that cause 
> > >> an action, be it legislation, prose guidance, or a set of machine 
> > >> instructions that implement that guidance.
> > >>
> > >>> values. IETF examples of this concept might include attributes 
> > >>> in a
> > >> MIB
> > >>> or a NETMOD model.
> > >>>
> > >>> I agree that we need to "persist" posture attribute values. I 
> > >>> have always been on the fence personally about calling 
> > >>> collection and evaluation results "content". I agree 100% that 
> > >>> we need to be
> > crystal
> > >>> I also share the same personal view around interacting with 
> > >>> stores
> > of
> > >>> approaches. I also believe that the temporal nature of this 
> > >>> data,
> > the
> > >>> I'm working through the UC draft addressing discussion items 
> > >>> from yesterday and the previous comments we have received on the 
> > >>> list on
> > -
> > >> 05.
> > >>>
> > >>> need to be more specific. Please find below an attempt to put 
> > >>> some "bounds" around different types of content.
> > >>>
> > >>>      o  Expressions that provide instructions for using a 
> > >>> specific
> > >> API,
> > >>>

---

From: Gunnar Engelbach <[email-masked]>
Date: Wed, 12 Feb 2014 13:27:24 -0500

> There is a need to be able to know what guidance was used for any processing, since it can be dynamic (as stated in the text). That means we now have what was assessed, what it was assessed against…

Thanks for the reminder.

The age of a collected artifact is something that came up in early 
discussions, along with information to help determine the level of trust 
that should be placed on the value such as the source of the artifact, 
what entities it has passed through from collection to usage, a 
signature, etc.

That kind of metadata falls within the Collected Artifacts category but 
is more specific to SACM -- something we need to remember to get into 
the requirements.

---

From: "ietfdbh" <[email-masked]>
Date: Thu, 13 Feb 2014 00:24:19 -0500

Hi,

I am concerned about lumping both characterization/metadata and attribute
values together.
In my view of the world,
 the metadata/characterization is determined during discovery and then
doesn't change much unless the endpoint is physically modified.
The attribute values, however, would include operational state, which can be
short lived and changes dynamically.

David Harrington
[email-masked]
+1-603-828-1401
> I'd like to get the discussion around this topic to a point where I have
clear
> editorial direction on what text we want included in the updated use case
> draft for this Friday's submission cutoff deadline.
> 
> Based on the mailing list discussions to date, I'd like to propose the
following
> categories of security automation data for inclusion in the use cases
draft:
> Guidance, Endpoint/Asset Attribute Data, and Processing Artifacts.
> 
> The following sections roughly define each of the proposed categories. For
> each section some insights are provided into the how the data might be
> sourced and how often it may change. I have also included examples of the
> classes of information for each category with mappings to the use cases
and
> building blocks they support to insure we have the necessary coverage.
> 
> 2) Do you like the labels for each of the groups? If not, do you have a
better
> label(s)?
> 4) What level of detail should be included in the use cases draft
regarding this
> discussion? What I have below is likely too much detail.
> 
> Thanks,
> Dave
> 
> Guidance:
> ------------
> The term "guidance" was proposed by Steve Hanna and was supported by
> Gunnar Engelbach on the list. Steve proposed that this concept encompasses
> "attribute identifiers, lists of attributes that are to be collected and
when and
> perhaps how, and policies about what attribute values are acceptable for
> different kinds or groups of endpoints. This indicates that these things
(and
> maybe others to be added later) guide the attribute collection and
evaluation
> process." I would also add that this should include instructions
supporting
> data-driven evaluation processes.
> 
> This data tends to change in units of months and days. In cases where
> assessments are made more dynamic, it may be necessary to handle changes
> in the scope of hours or minutes. This data will typically be provided by
large
> organizations, product vendors, and some 3rd-parties. Thus, it will tend
to be
> shared across large enterprises and customer communities. In some cases
> access may be controlled to specific authenticated users. In other cases,
the
> data may be provided broadly with little to no access control.
> 
> This would include:
> 
> - Applicability Statements: Data the identifies the conditional
applicability of
> an assessment (and related assessment "guidance") based on automated
> collection of endpoint posture attribute values and other endpoint
> characterizations.
>     + Endpoint Identification and Assessment Planning BB: Identify
Endpoint
> Targets - This supports the targeting concept allowing characterizations
and
> collected posture attribute values to be evaluated to determine what set
of
> endpoints should be assessed.
> 
> - Collection Instructions: Data that drives the identification of the set
of
> posture attributes needed to perform an assessment and the collection of
> posture attributes.
>     + Endpoint Identification and Assessment Planning BB: Posture
Attribute
> Identification - Used to determine the set of posture attributes for which
> values must be collected (or retrieved) to support the evaluation.
>     + Endpoint Posture Attribute Value Collection BB: Collection Content
> Acquisition - Data is retrieved based on its relationship to the
assessment and
> other applicability concepts to drive collection activities.
>     + Endpoint Posture Attribute Value Collection BB: Posture Attribute
Value
> Collection - Directives on how the posture attributes should be collected.
This
> may include: events to monitor, scheduling info, instructions for using
> specific models/protocols, etc.
> 
> - Evaluation instructions: Data the identifies what endpoint posture
attribute
> values are to be evaluated and how this evaluation should be performed.
>     + Endpoint Identification and Assessment Planning BB: Posture
Attribute
> Identification - Use to determine the set of posture attributes for which
> values are required for evaluation.
>     + Posture Evaluation BB: Evaluation Content Acquisition - Data is
retrieved
> based on its relationship to the assessment and other applicability
concepts
> to drive evaluation activities.
>     + Posture Evaluation BB: Posture Attribute Evaluation - Instructs the
> evaluation of collected endpoint posture attribute values. This will
likely
> include comparison operators and logical constructs. Additional
instructions
> might indicate the required result output format.
> 
> Our information model will need to relate the concepts of applicability
> statements, collection instructions, and evaluation instructions in
support of
> assessment activities.
> This category includes contextual metadata pertaining to endpoints and the
> assets that they are composed of (e.g., hardware, software, accounts), and
> collected endpoint posture attributes, their associated values, and
related
> context.
> 
> This data tends to change in units of days, minutes, a seconds with
posture
> attribute values typically changing more frequently than endpoint
> characterizations. This data tends to be organizationally and endpoint
> specific, with specific operational groups of endpoints tending to exhibit
> similar attribute profiles. This data will generally not be shared outside
an
> organizational boundary and will generally require authentication with
> specific access controls.
> 
> This would include:
> 
> - Characterizations of endpoints: Endpoint metadata that describes the
> endpoint type, organizationally expected function/role, etc.
>     + Endpoint Identification and Assessment Planning BB: Endpoint
> Characterization - Gathered through automated posture attribute
collection,
> manual data entry/query, and interactions with other data stores.
> 
> - Collected Posture Attributes: Posture attributes and associated values
> collected from endpoints representing hardware, software inventory,
> configurations, accounts/privileges, and other aspects of endpoint state.
>     + Endpoint Identification and Assessment Planning BB: Endpoint
> Component Inventory - This provides context to identify what posture
> attributes should be collected as part of the assessment based on the
> current endpoint posture (e.g., what software is installed).
>     + Endpoint Posture Attribute Value Collection BB: Posture Attribute
Value
> Collection - For each targeted endpoint, the set of collected posture
> attributes and associated values. The result of collection.
>     + Posture Evaluation BB: Posture Attribute Value Query - The retrieval
of a
> set of previously collected posture attributes and associated values. This
may
> not be required if collected data is fed directly to an evaluator after
> collection.
> 
> An important dimension here may be the separation between actual and
> expected posture attribute values. It may be possible to express guidance
in
> a way that allows expected values to parameterized. In this case it will
be
> necessary to allow for organizationally expected values to be associated
with
> defined guidance (perhaps provided by a different party).
> 
> Our information model will need to relate the concepts of endpoint
> characterizations and collected posture information to the types of
guidance
> data.
> This type of data is generated by and is specific to an individual
assessment
> process. This data may be used as part of the interactions between SACM
> architectural components to drive and coordinate collection and evaluation
> activities. Its lifespan will be bounded by the lifespan of the
assessment. It
> may also be exchanged and stored to provide historic context around an
> assessment activity.
> 
> This would include:
> 
>     + Endpoint Identification and Assessment Planning BB: Identify
Endpoint
> Targets - The endpoints targeted for an assessment.
> 
>     + Endpoint Identification and Assessment Planning BB: Posture
Attribute
> Identification - The required posture attributes that need to be collected
to
> perform the evaluation.
>     + Posture Evaluation BB: Posture Attribute Evaluation - The results
> produced through the comparison of actual collected posture attribute
>          values against their expected state. The specific output an
evaluation
> criteria may be specified by the required evaluation instructions.
> 
> Our information model will need to relate these concepts to those provided
> in the other two categories. I know this is not saying much, but I am
tired and
> ready to end this email. ;-)
> 
> (pkampana);
> since

---

From: Stephen Hanna <[email-masked]>
Date: Thu, 13 Feb 2014 13:57:01 +0000

Sorry for being late in responding. Very busy this week!
Still, I want to answer Dave's questions briefly:

> 1) Do you like these groupings?

I'm pleased with these groupings. Very good!

> 2) Do you like the labels for each of the groups?

The labels are fine. I'm not delighted with all of them
but I don't have any better suggestions. Let's go with
them. They are the right concepts. If someone comes up
with a better label later, we can switch to it.

> 3) Are there other types of content that should be included?

No new groupings. As we've seen in the last few days,
there will probably be more bits of information that
will need to be added to the groupings. More kinds of
info will probably be thought of as time goes on,
extending out for years or even decades. Let's not
make the system so rigid that it can't accommodate
new types of content being added, both proprietary
or (preferably) using a standard format.

> 4) What level of detail should be included in the
>    use cases draft regarding this discussion?
>    What I have below is likely too much detail.

Clearly, opinions differ on this topic. I believe
that for the Use Cases document, this is too much
detail. In that document, you just need to lay out
what the categories are and give some salient
examples of content that will be in each category.
Other documents can list and describe the content
in more detail.

I'd say that a good guideline would be that if you
need to mention a kind of content in the use cases,
it should be mentioned in the list earlier in the
document. If some content isn't mentioned in the
rest of the document, it can be omitted.

Thanks for figuring this out,

Steve

> I'd like to get the discussion around this topic to a point where I
> have clear editorial direction on what text we want included in the
> updated use case draft for this Friday's submission cutoff deadline.
> 
> following categories of security automation data for inclusion in the
> use cases draft: Guidance, Endpoint/Asset Attribute Data, and
> Processing Artifacts.
> 
> The following sections roughly define each of the proposed categories.
> For each section some insights are provided into the how the data might
> be sourced and how often it may change. I have also included examples
> of the classes of information for each category with mappings to the
> use cases and building blocks they support to insure we have the
> necessary coverage.
> 
> better label(s)?
> regarding this discussion? What I have below is likely too much detail.
> 
> Thanks,
> Dave
> 
> Guidance:
> ------------
> Gunnar Engelbach on the list. Steve proposed that this concept
> encompasses "attribute identifiers, lists of attributes that are to be
> collected and when and perhaps how, and policies about what attribute
> values are acceptable for different kinds or groups of endpoints. This
> indicates that these things (and maybe others to be added later) guide
> the attribute collection and evaluation process." I would also add that
> this should include instructions supporting data-driven evaluation
> processes.
> 
> assessments are made more dynamic, it may be necessary to handle
> changes in the scope of hours or minutes. This data will typically be
> provided by large organizations, product vendors, and some 3rd-parties.
> Thus, it will tend to be shared across large enterprises and customer
> communities. In some cases access may be controlled to specific
> authenticated users. In other cases, the data may be provided broadly
> with little to no access control.
> 
> This would include:
> 
> applicability of an assessment (and related assessment "guidance")
> based on automated collection of endpoint posture attribute values and
> other endpoint characterizations.
> Endpoint Targets - This supports the targeting concept allowing
> characterizations and collected posture attribute values to be
> evaluated to determine what set of endpoints should be assessed.
> 
> - Collection Instructions: Data that drives the identification of the
> set of posture attributes needed to perform an assessment and the
> collection of posture attributes.
> Attribute Identification - Used to determine the set of posture
> attributes for which values must be collected (or retrieved) to support
> the evaluation.
>     + Endpoint Posture Attribute Value Collection BB: Collection
> Content Acquisition - Data is retrieved based on its relationship to
> the assessment and other applicability concepts to drive collection
> activities.
> Value Collection - Directives on how the posture attributes should be
> collected. This may include: events to monitor, scheduling info,
> instructions for using specific models/protocols, etc.
> 
> attribute values are to be evaluated and how this evaluation should be
> performed.
> Attribute Identification - Use to determine the set of posture
> attributes for which values are required for evaluation.
> retrieved based on its relationship to the assessment and other
> applicability concepts to drive evaluation activities.
>     + Posture Evaluation BB: Posture Attribute Evaluation - Instructs
> the evaluation of collected endpoint posture attribute values. This
> will likely include comparison operators and logical constructs.
> Additional instructions might indicate the required result output
> format.
> 
> support of assessment activities.
> 
> This category includes contextual metadata pertaining to endpoints and
> the assets that they are composed of (e.g., hardware, software,
> accounts), and collected endpoint posture attributes, their associated
> values, and related context.
> 
> posture attribute values typically changing more frequently than
> endpoint characterizations. This data tends to be organizationally and
> endpoint specific, with specific operational groups of endpoints
> tending to exhibit similar attribute profiles. This data will generally
> not be shared outside an organizational boundary and will generally
> require authentication with specific access controls.
> 
> This would include:
> 
> collection, manual data entry/query, and interactions with other data
> stores.
> 
> - Collected Posture Attributes: Posture attributes and associated
> values collected from endpoints representing hardware, software
> inventory, configurations, accounts/privileges, and other aspects of
> endpoint state.
> Value Collection - For each targeted endpoint, the set of collected
> posture attributes and associated values. The result of collection.
>     + Posture Evaluation BB: Posture Attribute Value Query - The
> retrieval of a set of previously collected posture attributes and
> associated values. This may not be required if collected data is fed
> directly to an evaluator after collection.
> 
> expected posture attribute values. It may be possible to express
> guidance in a way that allows expected values to parameterized. In this
> case it will be necessary to allow for organizationally expected values
> to be associated with defined guidance (perhaps provided by a different
> party).
> 
> guidance data.
> assessment process. This data may be used as part of the interactions
> between SACM architectural components to drive and coordinate
> collection and evaluation activities. Its lifespan will be bounded by
> the lifespan of the assessment. It may also be exchanged and stored to
> provide historic context around an assessment activity.
> 
> This would include:
> 
> Endpoint Targets - The endpoints targeted for an assessment.
> 
> Attribute Identification - The required posture attributes that need to
> be collected to perform the evaluation.
> 
> evaluation criteria may be specified by the required evaluation
> instructions.
> 
> Our information model will need to relate these concepts to those
> provided in the other two categories. I know this is not saying much,
> but I am tired and ready to end this email. ;-)
> 
> confusing
> instructional
> > > >>> and "persist" them, as described in section 2.1.3. So we'll
> need
> > > >>> to
> > > >> have
> > > >>> considered content. They may be a different kind of content
> from
> > > the
> > > >>> yesterday and the previous comments we have received on the
> list
> > > >>> on
> > > -
> > > >> 05.
> > > >>>
> some
> > > >>>      o  Policies that define the selected posture attributes
> for
> > > which
> > > >>>

---

From: "Nancy Cam-Winget (ncamwing)" <[email-masked]>
Date: Sat, 15 Feb 2014 00:30:39 +0000

Hi Dave:

Thanks for putting more clarity, I have some comments and questions below:

On 2/11/14 4:13 PM, "Waltermire, David A." <[email-masked]>
wrote:

>clear editorial direction on what text we want included in the updated
>use case draft for this Friday's submission cutoff deadline.
>
>following categories of security automation data for inclusion in the use
>cases draft: Guidance, Endpoint/Asset Attribute Data, and Processing
>Artifacts.
>
>be sourced and how often it may change. I have also included examples of
>the classes of information for each category with mappings to the use
>cases and building blocks they support to insure we have the necessary
>coverage.
>
[NCW] It is an improvement, but please see my further comments and
questions below.

>2) Do you like the labels for each of the groups? If not, do you have a
[NCW] I'm OK with the current groupings, though I'd like to keep the model
as simple as possible.

>regarding this discussion? What I have below is likely too much detail.
[NCW] I agree with Steve H. that this is probably too much details for the
use cases.  
>large organizations, product vendors, and some 3rd-parties. Thus, it will
>tend to be shared across large enterprises and customer communities. In
>some cases access may be controlled to specific authenticated users. In
>other cases, the data may be provided broadly with little to no access
>control.
>
>This would include:
>
>applicability of an assessment (and related assessment "guidance") based
>on automated collection of endpoint posture attribute values and other
>endpoint characterizations.
>characterizations and collected posture attribute values to be evaluated
>to determine what set of endpoints should be assessed.
>
>of posture attributes needed to perform an assessment and the collection
>of posture attributes.
>assessment and other applicability concepts to drive collection
>activities.
[NCW] I am not seeing strong distinctions of the Applicability Statements
vs. Collection Instructions. As I see this converged through an endpoint
profile that defines the attributes to be assessed for that particular
endpoint.  Given that profile, I would see how both applicability and
collection duties are fulfilled.

>Attribute Identification - Use to determine the set of posture attributes
>for which values are required for evaluation.
>likely include comparison operators and logical constructs. Additional
>instructions might indicate the required result output format.
>
[NCW] Why couldn't the evaluation instruction be extracted from the
endpoint assessment profile?

>posture attribute values typically changing more frequently than endpoint
>specific, with specific operational groups of endpoints tending to
>exhibit similar attribute profiles. This data will generally not be
>shared outside an organizational boundary and will generally require
>authentication with specific access controls.
>
>This would include:
>
>in a way that allows expected values to parameterized. In this case it
>will be necessary to allow for organizationally expected values to be
>associated with defined guidance (perhaps provided by a different party).
>
>between SACM architectural components to drive and coordinate collection
>and evaluation activities. Its lifespan will be bounded by the lifespan
>of the assessment. It may also be exchanged and stored to provide
>historic context around an assessment activity.
>
>This would include:
>
[NCW] Is it correct for me to conclude that "Processing artifacts" are
differentiated from "Endpoint attribute data" in that "Processing
artifacts" are basically the types of attributes that result from an
"evaluation result"?  My definition of "evaluation result" could be the
processing result based on success in (endpoint) attribute collection or
assessment (and the assessment could have been affected from 1 or more
[endpoint] attributes)?

>provided in the other two categories. I know this is not saying much, but
>I am tired and ready to end this email. ;-)
>

---

From: "Waltermire, David A." <[email-masked]>
Date: Wed, 19 Feb 2014 01:33:03 +0000

Nancy,

Thanks for your feedback. I have some comments below:

> From: Nancy Cam-Winget (ncamwing) [mailto:[email-masked]]
> Sent: Friday, February 14, 2014 7:31 PM
> To: Waltermire, David A.; Stephen Hanna; Gunnar Engelbach; Panos
> Kampanakis (pkampana); [email-masked]
> [NCW] I agree with Steve H. that this is probably too much details for the use
> cases.

[DAW: Putting on my editor hat... Nancy et al... Please take a look at the -06 use case draft and let me know if the level of detail is good. I tried to strike a balance. I'm not sure if it is the right balance though.]

> [NCW] I am not seeing strong distinctions of the Applicability Statements vs.
> Collection Instructions. As I see this converged through an endpoint profile
> that defines the attributes to be assessed for that particular endpoint.  Given
> that profile, I would see how both applicability and collection duties are
> fulfilled.

[daw: I don't think it is practical to specifically identify which attributes need to be assessed for every endpoint. This won't scale well to environments that have many thousands of assets. It also won't scale well in environments that are dynamic with new endpoints joining/leaving a network on a regular basis. What is needed to address these challenges is a way to target assessments dynamically based on logic that defines the attributes of applicable endpoint targets in a way that allows endpoints to be targeted as they are identified. We need to be able to describe classes of endpoints for targeting.

Using this kind of approach, an endpoint can be identified dynamically as the target of an assessment if it matches the targeted class. Then based on the assessment policy, instructions can be sent to a collector to collect the appropriate posture attributes, and evaluation instructions can sent to an evaluator to evaluate the collected data. The evaluator can then either directly evaluate collected data and/or query previously collected data as instructed.

> >Additional instructions might indicate the required result output format.
> >

[DAW: I believe there is a desire to decouple guidance for collection and evaluation from policies that direct that this guidance should be applied to a class of endpoints with specifics about how and when collection and evaluation should be performed. This allows collection and evaluation rules to be defined by software publishers and 3rd parties, and allows organizations to define what events or time intervals trigger collection, and how often evaluations should occur. Furthermore, by allowing evaluations to parameterized, organizations can specify specific expected values that are used for evaluation.

For example, a device and OS vendor might define collection guidance that is based on a standardized management protocol to identify what attributes should be collected, they may also define evaluation guidance that is targeted towards a specific use of their device (e.g. function) with recommended expected values. A customer organization may define a corresponding collection policy that requires that the identified posture attributes (by referencing the vendor collection guidance) be completely collected to establish a baseline and then monitored for change events over time triggering incremental collection as changes occur. The same organization may define an evaluation policy that is targeted to only devices on a specific LAN that requires evaluations to be performed daily at midnight UTC based on the current effective collected posture of the device. As part of these evaluations they override the recommended expected values for a subset of specific posture attributes with organizationally defined stronger values. The organization may then use different collection and evaluation timings and expected values for similar devices on a different LAN that has different security, operational,  or performance constraints.

This is also a good example as to why collection and evaluation components need to be separate architectural entities to account for the specialized interactions with a collector and evaluator as described in the example.]

> differentiated from "Endpoint attribute data" in that "Processing artifacts"
> are basically the types of attributes that result from an "evaluation result"?
> My definition of "evaluation result" could be the processing result based on
> success in (endpoint) attribute collection or assessment (and the assessment
> could have been affected from 1 or more [endpoint] attributes)?

[DAW: "Processing artifacts" are distinct from "Endpoint attribute data", also called "Collected Posture Attributes" above. "Evaluation results" named "Assessment Results" above (maybe this should be named "Evaluation Results" for clarity) are one type of "Processing Artifact". Other types of "Processing Artifacts" include: "Endpoint Targets" which are the resolved set of endpoints that are targeted for collection and evaluation operations, and "Identified Posture Attributes" which are the individual posture attributes that need to be collected from the "Endpoint Targets" as part of the collection operation. I'm happy to name these differently if some other name is more clear. IMHO, calling both the result of collection and the result of evaluation "evaluation results" will confuse these concepts and violates the desired allowable separation we are working to achieve between collection and evaluation interactions/processes.]

> >> > And for the final result of applying the collected attributes to
> >> > the guidance, how about "assessment results?"
> >>
> >> Yes, I like that also.
> >>
> >> Probably eventually. But there also seems to be a need for broad
> >>terms since  people want to be able to put all the "guidance" into one
> >>repository
> >> > > collected from endpoints and maybe persisted. It's just not
> >> > > useful to use one term for all those kinds of things.
> >> > >
> >> > > attributes that are to be collected and when and perhaps how, and
> >> > > policies about what attribute values are acceptable for different
> >> > > kinds or groups of endpoints. I suggest that we use the
> >> > > collective term "guidance" for these things. This indicates that
> >> > > these things (and maybe others to be added
> >> > >> an action, be it legislation, prose guidance, or a set of
> >> > >> machine instructions that implement that guidance.
> >> > >>
> >> > >> on this list.  System characteristics are the product of
> >> > >> applying content to an endpoint.
> >> > >>
> >> > >> for all data that could flow through a SACM implementation but
> >> > >> is specific to SACM?
> >> > >>
> >> > >>
> >> > >> Also, while I agree with the need to allow the ability to
> >> > >> persist collected data items (and the results of posture
> >> > >> evaluations), I
> >> > don't
> >> > >>> Are there other opinions on this? Does anyone feel strongly
> >> > >>> that collected posture attribute values and evaluation results
> >> > >>> SHOULD
> >> > NOT
> >> > >> be
> >> > >>> considered content?
> >> > >>>
> >> > >>> Thanks,
> >> > >>>
> >> > >>> Dave
> >> > >>>
> >> > >>> need to
> >> > >>> from yesterday and the previous comments we have received on
> >> > >>> the list on
> >> > -
> >> > >> 05.
> >> > >>>

---

From: "Grance, Tim" <[email-masked]>
Date: Wed, 19 Feb 2014 02:19:45 +0000

________________________________________
From: sacm <[email-masked]> on behalf of Waltermire, David A. <[email-masked]>
Sent: Tuesday, February 18, 2014 8:33:03 PM
To: Nancy Cam-Winget (ncamwing); Stephen Hanna; Gunnar Engelbach; Panos Kampanakis (pkampana); [email-masked]
Subject: Re: [sacm] Types of content

Nancy,

For example, a device and OS vendor might define collection guidance that is based on a standardized management protocol to identify what attributes should be collected, they may also define evaluation guidance that is targeted towards a specific use of their device (e.g. function) with recommended expected values. A customer organization may define a corresponding collection policy that requires that the identified posture attributes (by referencing the vendor collection guidance) be completely collected to establish a baseline and then monitored for change events over time triggering incremental collection as changes occur. The same organization may define an evaluation policy that is targeted to only devices on a specific LAN that requires evaluations to be performed daily at midnight UTC based on the current effective collected posture of the device. As part of these evaluations they override the recommended expected values for a subset of specific posture attributes with organiz
 ationally defined stronger values. The organization may then use different collection and evaluation timings and expected values for similar devices on a different LAN that has different security, operational,  or performance constraints.