Subject: [sacm] Comments to SACM Vulnerability Assessment Scenario

From: "Wolfkiel, Joseph L CIV DISA ID (US)" <[email-masked]>
Date: Tue, 17 Nov 2015 14:26:15 +0000

I sent Danny Haynes comments on the vulnerability assessment document at: https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/  Since they may be relevant to some of our discussions, I'm attaching a document with the comments.  With my apologies, it was easier to insert comments in a word document for review, so I'm attaching that.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52
Fort Meade DISA Acquisiton Bldg Cube A4A58E
Work: (301) 225-8820
Gov Cell: (571) 814-8231
[email-masked]

---

From: Adam Montville <[email-masked]>
Date: Wed, 18 Nov 2015 07:18:27 -0600

Hi Joe, thanks for your review and comments.  It’s definitely easier to embed comments in a Word document, but the issue with that is that (IIRC) document attachments aren’t stored in the list’s archive, so anyone joining the group later or reviewing the history via the archive won’t actually be able to see the comments.

Would you, or someone you task, be able to transcribe your comments into the body of an e-mail?

Kind regards,

Adam

> On Nov 17, 2015, at 8:26 AM, Wolfkiel, Joseph L CIV DISA ID (US) <[email-masked]> wrote:
> 
> <Vulnerability Assessment Doc-jlw markups.docx>_______________________________________________
> sacm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sacm

---

From: Adam Montville <[email-masked]>
Date: Wed, 18 Nov 2015 07:19:26 -0600

Alternatively, if the draft’s authors are willing and able, they could transcribe the comments with their responses.

> On Nov 18, 2015, at 7:18 AM, Adam Montville <[email-masked]> wrote:
>

---

From: "Wolfkiel, Joseph L CIV DISA ID (US)" <[email-masked]>
Date: Thu, 19 Nov 2015 21:42:37 +0000

Here are my comments to the scenario document:

Section 1, 2nd paragraph: " whether or not the flaw is present within an enterprise, when
   compared to information about the state of the enterprise's   endpoints"
Comment: Does it say which vulnerabilities were discovered on what software on which endpoints?  There's not a definition or example of what "vulnerability report" means.

Section 2, bulleted list:
"The document begins with the assumption that the enterprise has
      received a vulnerability report , and that the report data has
      already been processed into a format that the enterprise's
      security software tools can understand and use.  In particular,
      this document:"
Comment: The text alludes to a requirement that the report be machine readable, but never states it.

Section 2, bulleted list:

"Does not discuss how the enterprise identifies a potentially
         relevant  vulnerability report."
Comment: It's not clear what a "potentially relevant" report is.

Section 2, bulleted list:
" Does not discuss how the enterprise collects that report"
Comment: Does this address how the report was acquired, or how the data in the report was acquired?

Section 2, bulleted list:
   o  The document assumes that the enterprise has a means of
      identifying enterprise endpoints.  This could mean identifying
      endpoints as they join the network, actively scanning for
      connected endpoints, passive scanning of network traffic to
      identify connected endpoints, or some other method of accounting
      for the presence of all endpoints in the enterprise.
Comment: Does the statement about endpoint identification imply that the report contains data about endpoints?

Section 2, bulleted list:
*  Does not specify how extracted endpoint information and
         vulnerability alert information are normalized to be
         compatible.
Comment: What are alerts normalized to be compatible with?

Section 2, bulleted list:
o  The document assumes that the enterprise has a policy by which
      assessment of endpoints based on vulnerability reports  are
      prioritized.  The document:
Comment: It appears that an organization will "assess" something on receipt of a vulnerability report.  What does it assess?

Section 3, first paragraph:
"The first step in this scenario involves identifying endpoints and
   collecting basic system information from them.  This occurs prior to
   the receipt of any specific vulnerability report and is part of the
   regular, ongoing monitoring of endpoints within an enterprise.  This
   process is not meant to report on, or gather data for any specific
   vulnerabilities.  The information gathered during this step could be
   applied in many enterprise automation efforts.  Specifically, in
   addition to vulnerability management, it could be used by
   configuration and license management tasks.  All of the information
   collected during this step is stored in a central location such as a
   CMDB ."
Comment: Please spell out "CMDB" on first use and provide a reference that explains what the authors think a CMDB would contain.  A configuration management database (CMDB) isn't necessarily assumed to contain a comprehensive device inventory.  Arguably, it stores baseline configurations and deviations to them.

Section 3.1, first paragraph:
Unique endpoint IDs are used to
   allow for systems to be tracked over time  and also allow for proper
   counts of assets during inventories and other similar collections.
Comment: Is this the scope of how/what endpoint IDs are used for?  Are IDs used to correlate information between sensors?

Section 3.1, second paragraph:
This sub-step aligns with the Endpoint Discovery, Endpoint
   Characterization (automated collection data only) , and Endpoint
   Target Identification building block capabilities.
Comment: Please provide rationale on why manually collected data is prohibited

Section 3.3, bulleted list:
Suggest changing " Endpoint type- e.g., standard computer, printer, router, mobile
      device, tablet, etc." to read " Endpoint function or general role - e.g., standard computer, printer, router, mobile
      device, tablet, etc."
Comment: "type" is weakly defined and ambiguous.

Section 3.3, bulleted list:
o  Hardware version/firmware
Comment: version or firmware of what?  The BIOS, cellular modem, hard drive?

Section 3.3, bulleted list:

"o  Open ports/running services 

   o  Operating system optional component inventory - some OS' have
      optional components that can be installed which may not show up as
      separate pieces of software (e.g., web and ftp servers, demo web
      pages, shared libraries, etc.).  Note that this could also occur
      within third-party applications as well. "
Comment:  Suggest that neither of these bullets describes "basic" data, since they are long, detailed lists that aren't specifically required to identify or characterize a device.

Section 3.3, bulleted list:
"Human-assigned endpoint attributes are yet another type of data item
   that could be collected here.  This would include any attributes that
   cannot be collected programmatically.  The attributes could be
   manually entered into a CMDB by a human after the initial data
   collection occurs.   The attributes could help with system
   categorization and may have influence on later prioritizations.  The
   human-assigned attributes could include:"
Comment: Recommend consideration of placing attributes on the endpoints so they can be discovered by multiple sensors and also viewed and edited locally.  This would enhance accuracy and ensure a common assessment by all sensors collecting data from an endpoint.

Section 3.3, 5th paragraph:
" There is direct alignment with the Endpoint Component
   Inventory, Posture  Attribute Identification, and Posture Attribute
   Value Collection building block capabilities since the purpose of
   this sub-step is to perform an initial inventory of the endpoint and
   collect basic attributes and their values."
Comment: Please define "posture"

Section 4, first paragraph:
"These stored vulnerability reports can be used and compared
   with later vulnerability reports for the purpose of duplicate
   detection and in some cases guidance on how to handle similar issues."
Comment: storing all individual reports in a data store and trying to dynamically combine and collate them at run time shouldn't be a prescribed behavior.

Section 4, second paragraph:
   All vulnerability reports should be assigned an internal tracking ID
   by the enterprise as a first step.  Incoming vulnerability reports
   might not have a global identifier when they are received, and might
   never have one assigned.
Comment: Suggest reports should be assigned a unique ID by the transmitter so the receiver isn't tasked with providing the ID

Section 4, third paragraph, bulleted list:
o  Ingest Date - the date that the report was received by the enterprise .
Comment 1: Add "and time" so if two reports are received on the same day, you can tell which one was first
Comment 2: Is "enterprise" the same as "CMDB", since the report is apparently machine readable and received by an application?

Section 4, third paragraph, bulleted list:
Date and Report Release
Comment: Change to read " Date and time of Report Release"

Section 4, third paragraph, bulleted list:
identify the period when a particular endpoint was first vulnerable
Change to read: " identify the period when a particular endpoint was first assessed as vulnerable" - because you only know the first time a sensor detected a vulnerability, which is dependent on someone discovering the vulnerable state and a sensor looking for it on a given endpoint.  The endpoint may have been vulnerable for years prior to the detection.

Section 4, third paragraph, bulleted list:
   o  Severity Score (if available) - these may be useful for later mitigation prioritization.
Comment: Recommend that severities not be transmitted with the vulnerability since they change over time and should be obtained from a continuously updated lookup source

Section 5, first paragraph:
If this is
   not sufficient to determine endpoint applicability, a secondary data
   collection for additional data and attributes may be performed to
   determine status with regard to the vulnerability report.
Comment: This doesn't consider current remediation methodologies where remediation packages that contain multiple patches are sent out and determine whether/how to "fix" an endpoint at runtime.

Section 5.1, first paragraph:
This data would  already have been collected in the pre-assessment data collection.
Change to read: " This data may have been collected in the pre-assessment data collection."  There is no guarantee this has happened.

Section 5, second paragraph, bulleted list:
o  Date of assessment - The date that the assessment was performed.
Change to read: " o  Date and time  of assessment - The date that the assessment was performed."

Section C.2 "Endpoint" bulleted list:
Endpoint type - the device type of the endpoint (e.g., standard
      computer, printer, router, mobile device, tablet, etc.)
Comment: This is ambiguous.  Typically a "tablet" is a type of "mobile device", that may also function as a WIFI router when used as a hot spot.  I think this list has a bunch of different, incompatible attributes jumbled together.  Suggest it be broken out into multiple lists, each describing the same "type" of characteristic (e.g. functionality, color, portability, form factor, etc)

Section C.2 "Endpoint" bulleted list:
o  Hardware version/firmware - the hardware or firmware version (if applicable)
Comment: This is meaningless without describing which hardware component you're collecting this data for

Section C.2 "Endpoint" bulleted list:
o  Operating system - Operating system name
Comment: Please state which part of the operating system descriptive text you consider to be the "name" (e.g. Windows?  Windows 10?  Windows 10 Home?)

Section C.2 "Endpoint" bulleted list:
o  Installed software name  - List of all installed software packages
      (i.e., software inventory).  May or may not include software
      installed by the operating system.
Comment: Same issue for software names as for OS names

Section C.2 "Endpoint" bulleted list:
 o  Open ports/services - Listening network ports (TCP and UDP ) and
      running services
Comment: Why is this list constrained to TCP and UDP?  I would expect GRE, IPsec, and other protocols to also be of interest

Section C.2 "Endpoint" bulleted list:
   o  Operating system optional component inventory - Operating system
      specific components and software (when NOT already included in the
      general software inventory)
Comment: Give examples or define a list.  The OS is made up of thousands of "components"

Section C.2 "Endpoint" bulleted list:
   o  Location  - The physical location of an enterprise endpoint
      (department, room, etc.)
Comment: Describe how the location would be quantified.  Is it GPS coordinates, is it a region name, is it a name pulled from an enumeration defined at a URL?

Section C.2 "Endpoint" bulleted list:
o  Role - The function  that the endpoint serves within the enterprise
      (e.g., end user system, database server, public web server, etc.)
Comment: Defining a role as a function is circular and simply makes the two words synonyms.  I suggest keeping "function" as a separate item from role.  For example, a role may be "workstation", while the functions would be office automation, video playback, e-mail client, etc.

Section C.2 "Endpoint" bulleted list:
o  File system attributes - Attributes that describe the state of a
      file or directory (e.g., versions, size, write date, modified
      date, checksum, etc.)
Comment: Remove "the state of" since other attributes that aren't state-based are also of interest

Section C.2 "External Vulnerability Report"
Comment: Prior to listing properties of an external vulnerability report, please describe what you mean.  Is a report "external" because it was created by some one outside of your organization?  Is it a report about a newly discovered vulnerability, or a vulnerability discovered on one of your endpoints by an external entity?

Overall Comment: Please provide a description of the different "reports" described in this paper in the introduction.  After reading through twice, I think you are using vulnerability "report" and vulnerability "definition" to mean the same thing.  A vulnerability assessment is apparently the process of detecting vulnerabilities on specific endpoints leading to a "vulnerability assessment report".  This is never explicitly defined or addressed, so it is confusing to try to build a mental model of the ideas in the scenario.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52

-----Original Message-----
From: sacm [mailto:[email-masked]] On Behalf Of Wolfkiel, Joseph L CIV DISA ID (US)
Sent: Tuesday, November 17, 2015 9:26 AM
To: [email-masked]
Cc: Haynes, Dan
Subject: [Non-DoD Source] [sacm] Comments to SACM Vulnerability Assessment Scenario

All active links contained in this email were disabled.  Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.  

----

I sent Danny Haynes comments on the vulnerability assessment document at: Caution-https://datatracker.ietf.org/doc/draft-coffin-sacm-vuln-scenario/  Since they may be relevant to some of our discussions, I'm attaching a document with the comments.  With my apologies, it was easier to insert comments in a word document for review, so I'm attaching that.

Joseph L. Wolfkiel
SCM Engineering Lead
DISA ID52