Subject: Re: [mile] IODEF predicate logic

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Thu, 1 Aug 2013 18:55:48 +0000

Hi Roman,

Thank you for the comments. I answer them below. ">>>" is in front of your questions.

>>>  In your email below you used the terminology  "multiple indicators on one folder" which I hadn't read before.  Did you mean multiple indicators in a single IODEF document?

Yes

>>> Isn't that  reversed, "When an IODEF **System** consists of two or more **Node** classes with the category attribute set to watchlist-source or watchlist-destination ...", since Node sub-element of System?  However, this isn't consistent with the -00 draft since System can only contain exactly 1 node.

You are right, it wasnot clear. I will rephrase it to something like 
" When a Flow Class that consists of multiple Systems with  
   watchlist-source and watchlist-destination attributes (watchlist of
   Systems) the System information should be ORed for the Flow Class
  described... "
Looks better?

>>><EventData> ...
<Flow>
  <System category=" watchlist-source"> <Node><Address>IP1</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP2</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP3</Address>... </System> </Flow> <Flow>
  <System category=" watchlist-source"> <Node><Address>IP4</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP5</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP6</Address>... </System> </Flow> </EventData>
Is the watchlist defined as "(IP1 OR IP2 or IP3) AND (IP4 OR IP5 OR IP6)"?

Yes. If an company saw IP1 or 2 or 3 only  they should not think they are seeing the indicator.

>>> Is this AND logic applied across different Flow classes or any other indicator in the EventData class parent outside of the Flow parent of the given System class of interest?  

Yes, I will rephrase it to " The content in 
		the EventData Class the Node belongs to should be combined with 
		the watchlist of Systems using AND logic. Different Flow classes 
		that consist of different System classes (with watchlist-source 
		or destination as attributes) follow AND logic in their parent 
		EventData Class. " 
in order to make it clearer. Makes more sense?

Panos

-----Original Message-----
From: Roman D. Danyliw [mailto:[email-masked]] 
Sent: Tuesday, July 30, 2013 5:54 AM
To: Panos Kampanakis (pkampana); [email-masked]
Subject: RE: IODEF predicate logic

Hi Panos!

Could you describe a little more about what you had in mind with the predicate logic?  In your email below you used the terminology  "multiple indicators on one folder" which I hadn't read before.  Did you mean multiple indicators in a single IODEF document?

In seeking clarity in Section 4.4 of draft-ietf-mile-iodef-guidance-01 I had a few questions.  Citing this section ...

> [I-D.ietf-mile-rfc5070-bis] defines two new category attributes in the 
> System Class.  These are watchlist-source and watchlist- destination 
> and they serve for watchlist indicator groupings.  When an IODEF Node 
> consists of two or more System Classes with various watchlist-source 
> and watchlist-destination attributes (watchlist of
> Systems)

Isn't that  reversed, "When an IODEF **System** consists of two or more **Node** classes with the category attribute set to watchlist-source or watchlist-destination ...", since Node sub-element of System?  However, this isn't consistent with the -00 draft since System can only contain exactly 1 node.

[snip]
    <xs:element name="System">
      <xs:complexType>
        <xs:sequence>
          <xs:element ref="iodef:Node" maxOccurs="unbounded"/> ...

> the System information should be ORed with the information in the Flow 
> Class.

Could you clarify what it means to be OR multiple Systems in the flow class?  For example, with a snippet as follows,

<EventData> ...
<Flow>
  <System category=" watchlist-source"> <Node><Address>IP1</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP2</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP3</Address>... </System> </Flow> <Flow>
  <System category=" watchlist-source"> <Node><Address>IP4</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP5</Address>... </System>
  <System category=" watchlist-source"> <Node><Address>IP6</Address>... </System> </Flow> </EventData>

Is the watchlist defined as "(IP1 OR IP2 or IP3) AND (IP4 OR IP5 OR IP6)"?

> In other words, either System description should be considered as a 
> watchlist indicator.  The rest of the content in the EventData Class 
> the Node belongs to should be combined with the watchlist of Systems 
> using AND logic.

Is this AND logic applied across different Flow classes or any other indicator in the EventData class parent outside of the Flow parent of the given System class of interest?  

Roman

From: [email-masked] [mailto:[email-masked]] On Behalf Of Panos Kampanakis (pkampana)
Sent: Monday, July 15, 2013 4:02 PM
To: [email-masked]
Subject: [mile] IODEF predicate logic

Hello everyone,

In the past people have asked how IODEF can combine multiple indicators in one IODEF definition. For example a question can be, if there are more than two malicious addresses, how they can be added in one definition and what is the logic behind how they are combined?

draft-ietf-mile-rfc5070-bis-00 introduces some watchlist fields that allow for multiple indicators on one folder. In draft-ietf-mile-iodef-guidance-01 I have added section http://tools.ietf.org/html/draft-ietf-mile-iodef-guidance-01#section-4.4 that describes how the predicate logic works.

Comments and thoughts are welcome.

Rgs,
Panos

---

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Thu, 1 Aug 2013 21:52:07 +0000

Hi Roman,

Yes

>>> Isn't that  reversed, "When an IODEF **System** consists of two or more **Node** classes with the category attribute set to watchlist-source or watchlist-destination ...", since Node sub-element…

You are right, it wasnot clear. I will rephrase it to something like " When a Flow Class that consists of multiple Systems with  
   watchlist-source and watchlist-destination attributes (watchlist of
   Systems) the System information should be ORed for the Flow Class
  described... "
Looks better?

<Flow>

Panos

-----Original Message-----
From: Roman D. Danyliw [mailto:[email-masked]] 
Sent: Tuesday, July 30, 2013 5:54 AM
To: Panos Kampanakis (pkampana); [email-masked]
Subject: RE: IODEF predicate logic

Hi Panos!

[snip]

<EventData> ...
<Flow>

Roman

Hello everyone,

Rgs,
Panos

---

From: "Roman D. Danyliw" <[email-masked]>
Date: Mon, 5 Aug 2013 12:02:26 +0000

Hi Panos,

Response inline ...

> From: Panos Kampanakis (pkampana) [mailto:[email-masked]]
> Sent: Thursday, August 01, 2013 2:56 PM
> To: Roman D. Danyliw; [email-masked]
> >>>  In your email below you used the terminology  "multiple indicators on
> one folder" which I hadn't read before.  Did you mean multiple indicators in a
> single IODEF document?
> 
> Yes
> 
> >>> Isn't that  reversed, "When an IODEF **System** consists of two or more
> **Node** classes with the category attribute set to watchlist-source or
> watchlist-destination ...", since Node sub-element of System?  However, this
> isn't consistent with the -00 draft since System can only contain exactly 1
> node.
> 
> You are right, it wasnot clear. I will rephrase it to something like " When a
> Flow Class that consists of multiple Systems with

I'm still having a little bit of trouble.  To reason about it, I listed the interpretation of the example below into a series of parsing rules:

(a) Multiple Systems in a given Flow are combined together into single expression where each IP is OR-ed
(b) Multiple Flow classes in a given EventData are combined into a series of expressions each AND-ed together

Is that right? This leads me to ask the following questions:

** What's the semantics of multiple EventData classes into a parent EventData?

** What is the difference between annotating a System to be a watchlist-source or watchlist-destination?  In the example XML snippet below, an expression of "(IP1 OR IP2 or IP3) AND (IP4 OR IP5 OR IP6)" was created.  How would have using "watchlist-destination" have changed the expression?  Is it that things marked as source warns of inbound connections to a network?

** Can one create a Flow with a System@category="source" and System@category="watchlist-source"?  If so, what would that mean?

** Can one create an EventData with two Flows, one Flow has several System@category="watchlist-source" and another exclusively of several System@category="source"?  If so, what would that mean?

** Do these expressions only apply to the IP address, or other things that can be indicators too?  For example, say I had:

<Flow>
<System category=" watchlist-source">
     <Node>
         <Address>IP1</Address>
         <Service ip_protocol="6"><Port>80</Port></Service>
     </Node>
</System>
<System category=" watchlist-source"><Node><Address>IP2</Address>...</System>
</Flow>
 
Would the expression be "(IP1 OR 80 OR IP2)"? "(IP1 AND 80 AND IP2)"? "((IP1 OR 80) OR IP2)"?  "((IP1 AND 80) OR IP2)?  What I'm asking is what is the parsing rule of multiple indicators in a given Node?

> >>> Is this AND logic applied across different Flow classes or any other
> indicator in the EventData class parent outside of the Flow parent of the
> given System class of interest?
> 
> 		the watchlist of Systems using AND logic. 

It's the use of Node here that still confuses me since a Node is inside a System.  Could you express this sentence as a parsing statement like above?

> Different Flow
> classes

The double use of different confuses me also.  Do you mean "Distinct Flow classes with the same parent EventData classes are ANDed together", aka parsing rule (b) from above?

Roman
 
> Could you describe a little more about what you had in mind with the
> predicate logic?  In your email below you used the terminology  "multiple
> indicators on one folder" which I hadn't read before.  Did you mean multiple
> indicators in a single IODEF document?
> 
> Isn't that  reversed, "When an IODEF **System** consists of two or more
> **Node** classes with the category attribute set to watchlist-source or
> watchlist-destination ...", since Node sub-element of System?  However, this
> isn't consistent with the -00 draft since System can only contain exactly 1
> node.
> 
> [snip]
> Is this AND logic applied across different Flow classes or any other indicator in
> the EventData class parent outside of the Flow parent of the given System
> class of interest?
> 
> Roman
> 
> In the past people have asked how IODEF can combine multiple indicators in
> one IODEF definition. For example a question can be, if there are more than
> two malicious addresses, how they can be added in one definition and what
> is the logic behind how they are combined?
> 
> draft-ietf-mile-rfc5070-bis-00 introduces some watchlist fields that allow for
> multiple indicators on one folder. In draft-ietf-mile-iodef-guidance-01 I have
> added section http://tools.ietf.org/html/draft-ietf-mile-iodef-guidance-
> 01#section-4.4 that describes how the predicate logic works.
>

---

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Wed, 7 Aug 2013 18:02:31 +0000

Hi Roman,

>>>Is that right? This leads me to ask the following questions:

Yes.

>>>** What's the semantics of multiple EventData classes into a parent EventData?

IODEF doesn't prevent you of having nested EventData. The predicate logic described doesn't change anything in the logic of nested EventDatas. If an incident contains multiple EventData each one  is supposed to be describing various aspects of the incident (Assuming the AND logic for the EventDatas here).

>>>** What is the difference between annotating a System to be a watchlist-source or watchlist-destination?  In the example XML snippet below, an expression of "(IP1 OR IP2 or IP3) AND (IP4 OR IP5 OR…

Correct, traffic destination indicator or source of traffic.

>>>** Can one create a Flow with a System@category="source" and System@category="watchlist-source"?  If so, what would that mean?

IODEF allows it. In this case the logic is AND exactly as if you had two sources. Now that logically cannot be true, but you can define it in IODEF.

>>>** Can one create an EventData with two Flows, one Flow has several System@category="watchlist-source" and another exclusively of several System@category="source"?  If so, what would that mean?

The Flow with the watchlists means that either source of this traffic is an indicator and then that would be ANDed with the (ANDed sources of the second Flow). I know that is not logical either, but IODEF allows it. A tool would never see such traffic in order to flag it as incident though.

         <Address>IP1</Address>
         <Service ip_protocol="6"><Port>80</Port></Service>
Means traffic from IP1 AND port 80. Now the Flow says "traffic from (IP1 and port 80) OR (traffic from IP2).

In general, in IODEF, previously without the predicate logic, it is my understanding that everything was AND. Like if you see this and that indicators then you have an incident.

I have rephrased the Predicate logic section as follows in an attempt to make it clearer

   Multiple indicators occasionally need to be combined in an IODEF
   document.  For example, a botnet might have multiple command and
   control servers.  A consistent predicate logic should be followed in
   order to present such relationships in IODEF.

   In [RFC5070], predicate logic only consisted of logical AND.  For
   example, if an Flow Class contained two System classes with "source"
   and "destination" as category attributes, it was assumed that both
   Systems should be present in order for the Flow to be true and thus
   marked as an indicator.

   [I-D.ietf-mile-rfc5070-bis] defines two new category attributes in
   the System Class that can enhance the IODEF predicate logic
   functionality.  These are watchlist-source and watchlist-destination
   and they serve for watchlist indicator groupings.  When a Flow Class
   that consists of multiple Systems with watchlist-source and
   watchlist-destination attributes (watchlist of Systems) the System
   information should be ORed for the Flow Class described.  In other
   words, either System description should be considered as a watchlist
   indicator.  The content in the EventData Class the Node belongs to
   should be combined with the watchlist of Systems using AND logic.
   Different Flow classes that consist of different System classes (with
   watchlist-source or destination as attributes) follow AND logic in
   their parent EventData Class.

   IODEF's grouping predicate logic follows the above pattern
   consistently.  [I-D.ietf-mile-rfc5070-bis] defined the
   HashInformation Class that describes a file hash information as also
   described in [RFC5901].  The HashInformation Class is of
   HashSigDetails type which consists of elements that describe the file
   hash details.  Some of the attributes of the HashSigDetails are
   introduced to describe watchlist groupings (i.e.
   PKI_email_ds_watchlist, PGP_email_ds_watchlist, file_hash_watchlist,
   email_hash_watchlist).  If any of these attributes are used in two or
   more HashInformation Classes of a Record then HashInformation content
   is ORed for the Record.  For example, if two HashInformation types
   are set to file_hash_watchlist, the list of hash details provided are
   just alternate representations for the same hash (SHA256.  SHA1 etc).
   Similarly, if multiple HashInformation classes, with "watchlist" in
   their category attribute, are in a Record using Reference elements or
   others, they should all be treated as different representations of
   the same file hash, assuming the FileName element is not used in the
   HashInformation.

   In some cases the predicate logic in IODEF can slightly change.
   [I-D.ietf-mile-rfc5070-bis] introduces the WindowsRegistryKeyModified
   Class which is of type RegistryKeyModified.  RegistryKeyModified has
   an optional type attribute which has watchlist as an option in order
   to include the ability to group WindowsRegistryKeyModified.  In order
   to group multiple WindowsRegistryKeyModified of the same watchlist of
   indicators multiple WindowsRegistryKeysModified should be used in the
   same RecordData or EventData Class.  If the RegistryKeyModified
   Classes are not under the same RecordData or EventData Class they
   should be treated as different indicator Keys modified.

Panos

-----Original Message-----
From: Roman D. Danyliw [mailto:[email-masked]]
Sent: Monday, August 05, 2013 8:02 AM
To: Panos Kampanakis (pkampana); [email-masked]
Subject: RE: IODEF predicate logic

Hi Panos,

Response inline ...

> To: Roman D. Danyliw; [email-masked]<mailto:[email-masked]>
> one folder" which I hadn't read before.  Did you mean multiple
> **Node** classes with the category attribute set to watchlist-source
> or watchlist-destination ...", since Node sub-element of System?
> However, this isn't consistent with the -00 draft since System can
> only contain exactly 1 node.
>
> You are right, it wasnot clear. I will rephrase it to something like "
> When a Flow Class that consists of multiple Systems with

<Flow>

> indicator in the EventData class parent outside of the Flow parent of
> the given System class of interest?
>

Roman

> To: Panos Kampanakis (pkampana); [email-masked]<mailto:[email-masked]>
> predicate logic?  In your email below you used the terminology
> "multiple indicators on one folder" which I hadn't read before.  Did
> you mean multiple indicators in a single IODEF document?
>
> In seeking clarity in Section 4.4 of draft-ietf-mile-iodef-guidance-01
> I had a few questions.  Citing this section ...
>
> > [I-D.ietf-mile-rfc5070-bis] defines two new category attributes in
> > the System Class.  These are watchlist-source and watchlist-
> > destination and they serve for watchlist indicator groupings.  When
> > an IODEF Node consists of two or more System Classes with various
> **Node** classes with the category attribute set to watchlist-source
> or watchlist-destination ...", since Node sub-element of System?
> However, this isn't consistent with the -00 draft since System can
> only contain exactly 1 node.
>
> [snip]
> > the System information should be ORed with the information in the
> > Flow Class.
>
> indicator in the EventData class parent outside of the Flow parent of
> the given System class of interest?
>
> Roman
>
> From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf
> Of Panos Kampanakis (pkampana)
> To: [email-masked]<mailto:[email-masked]>
> In the past people have asked how IODEF can combine multiple
> indicators in one IODEF definition. For example a question can be, if
> there are more than two malicious addresses, how they can be added in
> one definition and what is the logic behind how they are combined?
>
> draft-ietf-mile-rfc5070-bis-00 introduces some watchlist fields that
> allow for multiple indicators on one folder. In
> draft-ietf-mile-iodef-guidance-01 I have added section
> http://tools.ietf.org/html/draft-ietf-mile-iodef-guidance-