Subject: Re: [sacm] IM Template Change Proposal

From: "Cokus, Michael S." <[email-masked]>
Date: Fri, 1 Jul 2016 19:12:17 +0000

All,

It has been a while since this was discussed.  So I have put together a summary of recommendations from the thread and proposed a few more specifics.  I have also included some questions I think the group should address.  I'm taking the perspective of starting with IPFIX [1] and discussing what we want to keep, not keep, modify, or add, to arrive at the SACM IM syntax.

1. Information Element Properties

Propose we adopt the use of an information element specification template as described in [2].  We need to decide which of the existing IPFIX properties to use (perhaps with modified definitions), and whether we need additional properties for the SACM IM.  I propose we adopt the following information element properties to get us started, and discuss which other properties to add.

name

description

dataType

references

Questions:

- Are the information element properties above acceptable to the group as an initial set?

- What additional information element properties (adopted from IPFIX, developed by the group, etc.) are needed?   (Note that it may take some time/experience with developing information elements before the group can settle on the final properties.)

2. Naming Conventions

Propose we adopt the naming conventions for IPFIX information elements [3], with the exception of the Middleboxes subsection.  (I think that information described by the Middleboxes Taxonomy [4] should be accounted for in SACM metadata, rather than by modifying the name of the information element.)

3. Data Types

As recommended by Inacio [5], I think we should not adopt the IPFIX list datatypes [6].  I think we should define two new list datatypes: list and orderedList, as Jim proposed in [7].  Concerning the IPFX non-list data types, we need to decide which of these (if any) to adopt and what other data types need to be developed/added.  Note that it may take some time/experience with developing information elements before we settle on the final collection of datatypes.

We will need to determine the utility of the IPFX datatype semantics [8] (including list semantics [9]) for the SACM IM.  Right now my inclination would be to avoid them unless we find, during developing the information elements, that they're absolutely necessary.  This is, of course, something the group will need to discuss/decide.

Questions:

- Which of the existing IPFIX non-list data types should we adopt?

- What other datatypes do we need to develop/add?

- Does SACM need IPFIX datatype semantics?

4. New List Datatypes

Propose that the new list types define named groups of IEs, and indicate whether the order of the IEs must be preserved.

- The "list" datatype indicates that information element order is not significant but MAY be preserved.

- The "orderedList" datatype indicates that information element order is significant and MUST be preserved.

Propose that list-based Information elements be defined using "information element expressions."  Informally, an IE expression is based on regular expressions.  IE expressions use the familiar regex syntax and operators, but the terms are information element names instead of character classes.  Here is a simple example of the concept:

personInfo = list(firstName, middleNames?, lastName)

firstName = string

middleNames = orderedList(middleName+)

middleName = string

lastName = string

For an example consider the name, "John Ronald Reuel Tolkien".  Here are some logical "instances" of this name, using the personInfo definition.  A and B are legal, but C is not because the order in middleNames is not preserved.

A. personInfo = (firstName="John", middleNames(middleName="Ronald", middleName="Reuel"), lastName="Tolkien") [legal]

B. personInfo = (middleNames(middleName="Ronald", middleName=" Reuel"), lastName="Tolkien", firstName="John") [legal]

C. personInfo = (firstName="John", middleNames(middleName=" Reuel", middleName="Ronald"), lastName="Tolkien") [illegal]

Questions:

- Does the group want to explore this concept further?  (If so, I can provide more examples, potential syntax options, and more formal explanation.)

Thanks,

--mike

[1] https://tools.ietf.org/html/rfc7012

[2] https://tools.ietf.org/html/rfc7012#section-2.1

[3] https://tools.ietf.org/html/rfc7012#section-2.3

[4] https://tools.ietf.org/html/rfc3234

[5] https://mailarchive.ietf.org/arch/msg/sacm/1eRDuNM-Z0cNOIfJUUK0BMj9X5c

[6] https://tools.ietf.org/html/rfc7012#section-3.1.21

[7] https://mailarchive.ietf.org/arch/msg/sacm/0UjHYyEfvqKYy4Yb2OtRi6yGQj0

[8] https://tools.ietf.org/html/rfc7012#section-3.2

[9] https://tools.ietf.org/html/rfc6313#section-4.4

Mike Cokus
Systems Engineer
The MITRE Corporation

From: Adam Montville [mailto:[email-masked]]
Sent: Thursday, May 19, 2016 8:08 PM
To: Jim Schaad <[email-masked]>
Cc: Cokus, Michael S. <[email-masked]>; [email-masked]
Subject: Re: [sacm] IM Template Change Proposal

Do any others have opinions on this discussion?

On May 17, 2016, at 3:18 PM, Jim Schaad <[email-masked]<mailto:[email-masked]>> wrote:

-----Original Message-----
From: Cokus, Michael S. [mailto:[email-masked]]
Sent: Tuesday, May 17, 2016 12:56 PM
To: Jim Schaad <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: RE: [sacm] IM Template Change Proposal

Thanks for a good discussion Jim.  I encourage others in the group to
weigh in.

[MSC] This brings up the question of which of the IPFX list semantics
we want to keep.  It looks like oneOf is useful, but we could use
regex "|" to represent this instead.  But borrowing  "|" from regex
also gives additional flexibility.  Are there other IPFIX list semantics
we want to

keep?

[JLS] At this point I don't think so, but until we finish we probably
don't know for sure.

[MSC] As the group develops/proposes IE definitions, it should become more
clear whether we are missing something.  If anyone is/becomes aware of
IPFIX

list semantics [1] we should think about keeping, please let the rest of
the group

know.

[1] https://tools.ietf.org/html/rfc6313#section-4.4

softwareVersion = list( simpleVersion | rpmVersion |
ciscoTrainVersion |
...)
[JLS] Also need to think about the question of extensibility.  Are
lists closed or open for additional elements.  I believe that the ...
above is not needed because we have an implicit statement that all
lists
can have additional elements added.

The question would just be where they can be added for ordered lists.

[MSC] The open/closed list question hadn't occurred to me.  My
implicit assumption was that the lists were closed.  Now that you
bring it up, though, I agree that we need open lists.

[MSC] It strikes that the rest of the group may not be clear about what
open/extensible means in this context.  I won't speak for Jim, since we
might

have actually have different ideas here.  To me, extensible means that you
can

append IEs to a list which aren't specified in the list definition.
Referring to Jim's example above, additional version IEs might be needed
before

they actually get added to the official definition of softwareVersion.  In
this

case, the flexibility to add/use it before going through a complete cycle
of

configuration management would be handy.
We must be very careful about when/where this is allowed, how recipients
are

supposed to interpret additional information which they may not
"understand"

and what follow-up actions in the community are expected.

[JLS]  I want to also point to the fact that this is going to need to be
supported for dealing with data models as well which extend the SACM IM in
some way.  That is they may add additional items to a list which need to  be
dealt with in some fashion.  Thus I think that the extensibility is forced
from a SACM requirement.

Jim

Allowing for sublists is going to make the document shorter
(potentially) as new templates do not need to be defined for every
structured element.

Common structured elements can be extracted as needed.

[MSC] I wonder whether the benefit of sub-lists is worth the
additional complexity.  Not saying that right now I know the full
extent of the benefit/complexity trade off.  But I think it's worth
discussing.

[JLS] I think that I would start with allowing them and it would be
"easy"

to remove them later if it looks to be too much of a mess.

[MSC] In the mean-time, I plan to play around with some examples to figure
out

what kinds of combinations may be problematic.

Thanks,

--mike

Mike Cokus
Systems Engineer
The MITRE Corporation

_______________________________________________
sacm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/sacm

---

From: "Jim Schaad" <[email-masked]>
Date: Fri, 1 Jul 2016 14:21:03 -0700

A couple of notes in line

Jim

From: sacm [mailto:[email-masked]] On Behalf Of Cokus, Michael S.
Sent: Friday, July 01, 2016 12:12 PM
To: [email-masked]
Subject: Re: [sacm] IM Template Change Proposal

All,

It has been a while since this was discussed.  So I have put together a
summary of recommendations from the thread and proposed a few more
specifics.  I have also included some questions I think the group should
address.  I'm taking the perspective of starting with IPFIX [1] and
discussing what we want to keep, not keep, modify, or add, to arrive at the
SACM IM syntax.  

Propose we adopt the use of an information element specification template as
described in [2].  We need to decide which of the existing IPFIX properties
to use (perhaps with modified definitions), and whether we need additional
properties for the SACM IM.  I propose we adopt the following information
element properties to get us started, and discuss which other properties to
add.

name
description
dataType
references

Questions:

- Are the information element properties above acceptable to the group as an
initial set?
- What additional information element properties (adopted from IPFIX,
developed by the group, etc.) are needed?   (Note that it may take some
time/experience with developing information elements before the group can
settle on the final properties.)

[JLS] Note that the integer identifier element is not in this list.  This is
because this is really a data serialization optimization field rather than
an IM field.  If you refer to the element in an SQL query you want to use
the name of the field and not the integer value.  This means that we will
additionally require that the name field be unique.

[JLS] I think we probably do want to include a status field, even though in
the first version of the IM the field is a fixed value.  I would prefer to
define it as a "added in version X" and "deprecated in version Y" type
string.

[JLS] We may want to create some type of field to describe flags or
sub-structure of a field rather than leave it in the description field as it
currently is.  I am not sure if this should be placed in the dataType field
as part of a RE type definition or if it should be a separate field.

[JLS] Is there any reason why we should need to define a pattern matcher for
fields? 

2. Naming Conventions

Propose we adopt the naming conventions for IPFIX information elements [3],
with the exception of the Middleboxes subsection.  (I think that information
described by the Middleboxes Taxonomy [4] should be accounted for in SACM
metadata, rather than by modifying the name of the information element.)

3. Data Types

As recommended by Inacio [5], I think we should not adopt the IPFIX list
datatypes [6].  I think we should define two new list datatypes: list and
orderedList, as Jim proposed in [7].  Concerning the IPFX non-list data
types, we need to decide which of these (if any) to adopt and what other
data types need to be developed/added.  Note that it may take some
time/experience with developing information elements before we settle on the
final collection of datatypes.

We will need to determine the utility of the IPFX datatype semantics [8]
(including list semantics [9]) for the SACM IM.  Right now my inclination
would be to avoid them unless we find, during developing the information
elements, that they're absolutely necessary.  This is, of course, something
the group will need to discuss/decide.

Questions:

[JLS] I seem to remember having somebody form the IPFIX world saying that
datatype semantics is something that we should not adopt, but I cannot find
the message where this was said.  I don't know that it makes any sense to
have this.

4. New List Datatypes

Propose that the new list types define named groups of IEs, and indicate
whether the order of the IEs must be preserved.  

- The "list" datatype indicates that information element order is not
significant but MAY be preserved.  
- The "orderedList" datatype indicates that information element order is
significant and MUST be preserved.

Propose that list-based Information elements be defined using "information
element expressions."  Informally, an IE expression is based on regular
expressions.  IE expressions use the familiar regex syntax and operators,
but the terms are information element names instead of character classes. 
Here is a simple example of the concept:

For an example consider the name, "John Ronald Reuel Tolkien".  Here are
some logical "instances" of this name, using the personInfo definition.  A
and B are legal, but C is not because the order in middleNames is not
preserved.

A. personInfo = (firstName="John", middleNames(middleName="Ronald",
middleName="Reuel"), lastName="Tolkien") [legal] 

B. personInfo = (middleNames(middleName="Ronald", middleName=" Reuel"),
lastName="Tolkien", firstName="John") [legal]

C. personInfo = (firstName="John", middleNames(middleName=" Reuel",
middleName="Ronald"), lastName="Tolkien") [illegal]

Questions:

- Does the group want to explore this concept further?  (If so, I can
provide more examples, potential syntax options, and more formal
explanation.)

Thanks,

--mike

Mike Cokus
Systems Engineer
The MITRE Corporation

From: Adam Montville [mailto:[email-masked]] 
Sent: Thursday, May 19, 2016 8:08 PM
To: Jim Schaad <mailto:[email-masked]>
Cc: Cokus, Michael S. <mailto:[email-masked]>; mailto:[email-masked]
Subject: Re: [sacm] IM Template Change Proposal

On May 17, 2016, at 3:18 PM, Jim Schaad <mailto:[email-masked]>
wrote:

-----Original Message-----
From: Cokus, Michael S. [mailto:[email-masked]]
Sent: Tuesday, May 17, 2016 12:56 PM
To: Jim Schaad <mailto:[email-masked]>; mailto:[email-masked]
Subject: RE: [sacm] IM Template Change Proposal

Jim

Thanks,

--mike

Mike Cokus
Systems Engineer
The MITRE Corporation

mailto:[email-masked]

---

From: Jerome Athias <[email-masked]>
Date: Sat, 2 Jul 2016 09:12:51 +0300

Inline

> - Are the information element properties above acceptable to the group as an

[JA] Seems acceptable for me

> additionally require that the name field be unique.

[JA] I'm ok with unique name fields

> define it as a "added in version X" and "deprecated in version Y" type

[JA] Could be acceptable. It seems a simple way for tracking status/versioning

> as part of a RE type definition or if it should be a separate field.

[JA] Having it in the dataType seems good for me

> [JLS] Is there any reason why we should need to define a pattern matcher for

[JA] no comment

[JA] Regarding "references", if it is for "Identifies additional
specifications that more precisely define this item or provide
additional context for its use.";
I recommend (based on my experience with that as part of my XORCISM
research [1]) that we consider an approach for referencing other
specifications/standards "from" the IE. This would offer benefits of
reuse, interoperability and add semantic advantages.
I found it CORE element in my research.
It has been introduced and used in XORCISM with a VOCABULARY
Information Element (IE).
While built on (reusing) existing specifications and standards (and so
their respective Information Models, Data Models, etc.), I would say
that all IEs in XORCISM have a relationship/reference to a VOCABULARY,
which translates to (in the XORCISM PoC Relational Database Schema) a
relationship/association of 'each' tables with the Vocabulary table.

The reduced list of (as-is) properties of the VOCABULARY IE/Table are:

(VocabularyGUID)
VocabularyName
VocabularyVersion
VocabularyReference
(CreatedDate)
(ValidFromDate)
(ValidUntilDate)

As of know, the Vocabulary Table is used to store (and reference to)
Specifications, Standards, Frameworks, Taxonomies, Controlled
Vocabularies, Guidelines or other 'Documents' (e.g. Architectures or
Protocols description)
Some examples for the SACM context are:
OVAL
CCE
CVE
CPE
NCP
CIS

[1] http://www.frhack.org/research/xorcism.php

> metadata, rather than by modifying the name of the information element.)

[JA] A naming convention is needed.
"Names of Information Elements SHOULD be descriptive."
Yes
"Names of Information Elements MUST be unique"
Yes
"Names of Information Elements MUST start with lowercase letters." (+
"Composed names MUST use capital letters for the first letter of each
component (except for the first one).")
Personally, I don't like this one too much...

The reason is mainly because I like to have 'consistency' and favorite
usability (for implementation*) while taking into consideration this
approach:
Aligning source code variables names with the data model names (e.g.
tables/properties names in a RDBMS) with the IEs names.
(This is just because it is faster and clearer* (often less ambiguity
or human disturbance; improved User Experience), when using, for
example, some kind of Rapid Application Development (RAD) method. And
improves readability in the source code, API, storage, etc.)

> final collection of datatypes.

[JA] Seems fine at this stage

> the group will need to discuss/decide.

[JA] Seems quite fine at this stage

> [email-masked]

---

From: Adam Montville <[email-masked]>
Date: Sat, 2 Jul 2016 09:32:33 -0500

> On Jul 1, 2016, at 4:21 PM, Jim Schaad <[email-masked]> wrote:
> 

Chris Inacio made the following comment at IETF 95 during the IM discussion (pertaining to Slide 13): “Recommend that you not reuse the IPFIX abstract data structures (e.g., the list structures)”.

Is this what you’re thinking of?

IETF 95 Minutes: https://www.ietf.org/proceedings/95/minutes/minutes-95-sacm
IETF 95 IM Presentation: https://www.ietf.org/proceedings/95/slides/slides-95-sacm-4.pdf

---

From: "Jim Schaad" <[email-masked]>
Date: Sat, 2 Jul 2016 10:27:20 -0700

No, it had to do specifically with the datatypesemantics field of the template.

Jim

From: sacm [mailto:[email-masked]] On Behalf Of Adam Montville
Sent: Saturday, July 02, 2016 7:33 AM
To: Jim Schaad <[email-masked]>
Cc: Cokus, Michael S. <[email-masked]>; [email-masked]
Subject: Re: [sacm] IM Template Change Proposal

On Jul 1, 2016, at 4:21 PM, Jim Schaad <[email-masked] <mailto:[email-masked]> > wrote: