Subject: Re: Comment's on Cynthia McLain's proposals

From: Cynthia McLain <[email-masked]>
Date: Fri, 26 Oct 2001 16:16:19 -0400

Hi Dave,

I'm replying to you since that's the only address that
was included in your e-mail. Please pass it along as
you see appropriate. I've asked a colleague to reply,
in a separate e-mail, to your concerns about the
confidence tag.

  David> My comments on Cynthia McLain's recent proposals.
  David> 
  David> =============================================
  David> Comments on "Host-Based IDS" (Message 2 of 4)
  David> =============================================

  Cynthia> A. Alert impact types:
  Cynthia>
--------------------------------------------------------------------
  Cynthia> If it is not feasible to return to the use of enumerated
types, due
  Cynthia> to concerns about extensibility, the potential variability in
this
  Cynthia> attribute could be mitigated by recommending specific strings
in the
  Cynthia> document.

  David> Comment: The reason we changed the "impact" attribute from a
list of values
  David>   to a free-format string was that nobody was happy with the
list of values
  David>   that we had, and nobody had any brilliant ideas for how to
solve this.  I
  David>   don't think anyone is really happy with a free-format string
either, but
  David>   it seemed better to allow the flexibility than to restrict
implementors
  David>   to a lousy set of choices.

Since I've gone with a free-format string in a number of places myself,
for the more or less the same reason, I can certainly agree with your
reasoning. My concern is that without specific recommendations about
what strings to use, a free-for-all will result. Is it possible to
come up with a subset that represents a "good" set of choices?

    Cynthia> B. Additional Node category type:
    Cynthia>
--------------------------------------------------------------------
    Cynthia> Keyword Description
    Cynthia> ------- -----------
    Cynthia> hosts Local hosts file (Unix file system)

  David> Comment: I would remove the "(Unix file system)" restriction
here, and just
  David>   make it a generic local hosts file (Windows does have one,
although it is
  David>   not used very often).

Fine with me, it was intended as more of a note than a restriction.

    Cynthia> C. Additional Analyzer categories
    Cynthia> ---------------------------------
    Cynthia> The two additional attributes are:
    Cynthia> ostype
    Cynthia>    Optional. Operating system name (e.g., Linux).
    Cynthia> osversion
    Cynthia>    Optional. Version of the operating system.

  David> Comment: This seems fine, but I wonder if there's a way we can
be more clear
  David>   about what exactly we want.  For example, is it "Solaris 8"
or "SunOS 5.8"?
  David>   And is it "Linux" or "RedHat Linux" or "RedHat" or ...  Maybe
this is just
  David>   something we have to live with (it's easily enough addressed
with aliases
  David>   or whatever in the manager).

My sense is that this is something that we have to live with.
Having said that, more thought should be given to the example
that is provided. I'm thinking that "RedHat Linux" would provide
more information than "Linux". However, it is somewhat harder
to obtain the RedHat information since uname gives the kernel
release (Linux something) whereas the OS name (and kernel release)
is in /etc/issue on RedHat, which many people may not know about.
I don't know about other versions of Linux but can find out if
you think it makes sense.

    Cynthia> D. Additional sub-element to Target
    Cynthia> -----------------------------------
    Cynthia> In XML the "File" element would be added to "Target" as
follows:
    Cynthia>
    Cynthia> <!ELEMENT Target (
    Cynthia>     Node?, User?, Process?, File*, Service?
    Cynthia> )>

  David> Comment: I believe this should be File? and not File* (i.e.,
zero or one, not
  David>   zero or more).  The thinking for the other subclasses is that
if you have
  David>   more than one, then you have multiple targets, and repeat at
the <Target>
  David>   level, not below.   
  David> 
  David> Question: Is there a reason you chose File* instead of File? ?

I chose "File*" in part because of the "original" and "current"
attributes for the "File" element. It seemed convenient to be
able to embed "File" elements with each of the two attributes
into a single "Target" element. Another reason for the "File*" is
that a number of files could have been modified at roughly the
same time (within an update interval for the host-based IDS),
so it seemed more efficient to embed them all in a single "Target"
element since the other parts of target could be the same for all.
I don't feel strongly about this issue but do want the decision
to use "File?" to be a considered one.

    Cynthia> The "File" element consists of several sub-classes:
    Cynthia>
    Cynthia> path
    Cynthia>   [...]
    Cynthia>   If the path field is eliminated, the name information
should
    Cynthia>   include the path in order to avoid ambiguity.

  David> Comment: I disagree with this.  If you know the path, put it in
<path>.  If
  David>   you don't know it, leave <path> out.  The <name> element
should never have
  David>   the path in it.

I concur on leaving the path out of name. I'd like to retain the
requirement for a <path>, since it provides important information
about which specific file was affected since multiple files can
have the same name as long as they have different paths.

    Cynthia> create_time
    Cynthia>    Zero or one. DATETIME as described in Section 4.4.6 of
    Cynthia>    draft-ietf-idwg-idmef-xml-04.txt. Create time of this
file.

  David> Question: Do we need to include a clarification about Unix
"ctime" in this?
  David>   (The issue being that Unix "ctime" is commonly misinterpreted
as creation
  David>   time, when really it is last inode change time.  Unix does
not keep track
  David>   of file creation times.)  I know that's not an IDMEF issue,
but I keep
  David>   having this vision of lots of misinformed developers filling
that field
  David>   in on their Unix-based IDSes...

This issue is a bit of a can of worms...

    Cynthia> ++++ GroupId element ++++
    Cynthia> name
    Cynthia>    Zero or one. STRING. A user or group name. To represent
the Unix
    Cynthia>    permissions for "other", the name "unix_other" should be
used.

  David> Question: Why do we need to rename "other" as "unix_other"? 
This seems like
  David>   an arbitrary special case that's just going to get us into
trouble later
  David>   (as people start adding other special cases).

The idea was to provide guidance on how to handle the "world"/"other"
level of permissions in Unix. The rationale for providing a special
case was that someone might choose to use "other" as an actual group
name, it seemed unlikely that a group called "unix_other" would be
created.

My office-mate points out that the UNIX "world"/"other" can be thought
of as default permissions, as can "everyone" in Windows. If looked at
that way "FileAccess" could be represented as follows:

<!ENTITY % attvals.accesscat   "
     ( user | group | default )
">

<!ELEMENT FileAccess   (
    Id? permissions
)>
<!ATTLIST FileAccess   (
    category     %attvals.accesscat   #REQUIRED
)>

Using a single "Id" instead of "UserId" and "GroupId" has a certain
appeal especially since special conditions were placed on "UserId"
(i.e., ignore the attributes).

Some problems that I see with doing it this way is that the categories
aren't necessarily mutually exclusive (because of the MS "everyone",
which is a default) and the Id becomes optional, which doesn't make
sense if the category is "group" or "user". It does remove the special
case though...

  David> Overall, I think this proposal is excellent, and once my issues
and any raised
  David> by others on the list are resolved, we should adopt it into the
IDMEF before
  David> we go to last call.

  David> =====================================
  David> Comments on "Action" (Message 3 of 4)
  David> =====================================

    Cynthia> To provide a more structured representation of response
information,
    Cynthia> it is proposed that an "Action" element be defined and
added as a
    Cynthia> sub-element of "Alert" in lieu of the "action" attribute.
Zero or
    Cynthia> more "Action" elements would be allowed. The description of
the
    Cynthia> action could be represented as a string or as XML.

  David> Comment: I'm okay with this, up to the last sentence.  It's not
clear to me if
  David> XML is really needed here.  Yes, I suppose I could contrive a
use for it, in
  David> the future, but I think strings are plenty for the moment.
  David>
  David> Other than that, what's proposed here sounds reasonable. 
Cynthia's points that
  David> it offers more flexibility, and gives the opportunity to add
attributes in the
  David> future as experience is gained, are both well taken.  Again, I
think we should
  David> adopt this into the IDMEF before we go to last call, once any
other issues have
  David> been resolved.

I'd like to make one more argument for the XML. I was thinking
of it as being similar to the AdditionalData but specific to
Action. That is, it would make it more convenient for extensions
to the Action element to be integrated into IDMEF. If you think
it makes more sense for extensions to Action to be tested in
the AdditionalData section, I can certainly live with that.

  David> =========================================
  David> Comments on "Confidence" (Message 4 of 4)
  David> =========================================

    Cynthia> The confidence attribute was removed from the Alert data
model due
    Cynthia> to concerns within the IDWG over the ability to assign a
meaningful
    Cynthia> value to this field.

  David> Comment: I think the problem was a little more complex than
that.  The issue
  David> was no so much whether or not a meaningful value could be
assigned to this
  David> attribute by a single IDS.  Rather, the issue was what happens
when you get
  David>   these values from different IDSes back at the manager -- are
they comparable
  David>   in any meaningful way?

  David>   For example, if IDS-A says 0.7 and IDS-B says 0.8, does that
mean B is more
  David>   certain than A?  Not necessarily, depending on how the number
was calculated.
  David>   This becomes a real problem if both IDSes are reporting on
the same event.
  David>   Say, for example, that a network-based IDS reports an event
with confidence
  David>   0.8, and a host-based IDS reports the same event with a 0.2. 
Does that mean
  David>   the network-based IDS is pretty sure it happened, and the
host-based IDS is
  David>   pretty sure it didn't?  Not necessarily, since the
confidences are not
  David>   directly comparable.  So which IDS does the manager believe?

  David> That's the problem we couldn't solve.  I think.
  David>
  David> I'm not an expert on confidence calculations, and will defer to
those who are
  David> for whether or not this attribute can be kept in a useful
form.  I'm inclined
  David> not to include it in the IDMEF unless and until that can be
shown.

I've asked Oliver Dain to respond to this issue.

  David> --Dave

--Cindy
------------------------------------------------------------
Cynthia D. McLain                 M.I.T. Lincoln Laboratory
Information Systems Technology    244 Wood Street
phone:  781-981-3943              Lexington, MA 02420-9108
e-mail: [email-masked]       Fax: 781-981-0186
------------------------------------------------------------

---

From: "David A. Curry" <[email-masked]>
Date: Sat, 27 Oct 2001 10:43:59 -0400

[I neglected to send my original comments to the list (oops), and so Cindy's
response didn't go there either.  I've re-sent them all to the list, so all
should be caught up now. --Dave]

  [On the subject of the free-format "impact" attribute]

  Cynthia> Since I've gone with a free-format string in a number of places
  Cynthia> myself, for the more or less the same reason, I can certainly
  Cynthia> agree with your reasoning. My concern is that without specific
  Cynthia> recommendations about what strings to use, a free-for-all will
  Cynthia> result. Is it possible to come up with a subset that represents a
  Cynthia> "good" set of choices?

I don't personally have any objection, and it's probably a Good Thing.  But
it would need to be done (by whoever volunteers to do it) before the current
November 4th deadline so I can get it put into the last-call draft, otherwise
it will just have to wait until "IDMEF 2."

  [On the subject of the "ostype" and "osversion" attributes]

  Cynthia> My sense is that this is something that we have to live with.
  Cynthia> Having said that, more thought should be given to the example that
  Cynthia> is provided.

My sense is the same, we just have to live with it.  The number of special
cases out there is so large that writing rules for it is probably just an
exercise in frustration.  It should be relatively easy to deal with in the
manager anyway:

	case "$ostype $osversion" in
	Solaris 2*|SunOS 5*)
		...
	*Linux*)
		...
	esac

and so on...

  [on the subject of File* -vs- File?]

  Cynthia> I chose "File*" in part because of the "original" and "current"
  Cynthia> attributes for the "File" element. It seemed convenient to be able
  Cynthia> to embed "File" elements with each of the two attributes into a
  Cynthia> single "Target" element. Another reason for the "File*" is that a
  Cynthia> number of files could have been modified at roughly the same time
  Cynthia> (within an update interval for the host-based IDS), so it seemed
  Cynthia> more efficient to embed them all in a single "Target" element since
  Cynthia> the other parts of target could be the same for all.  I don't feel
  Cynthia> strongly about this issue but do want the decision to use "File?" to
  Cynthia> be a considered one.

The second situation (multiple files being changed) is essentially the same
as an attack against multiple hosts, and the current model for that is to
use multiple Targets, one for each host (file).

The first situation (original and current) though, I had not thought of, and
I agree with you that it makes sense for those two items to be in the same
place.  Perhaps we need

  <!ELEMENT Target (
      Node?, User?, Process?, (File, File?)?, Service?
  )>

And then some language in the draft that says that if you use both of those
Files, then they should describe the same file, with one being "original"
and the other being "current".

  [On the subject of "other" -vs- "unix_other"]

  Cynthia> The idea was to provide guidance on how to handle the
  Cynthia> "world"/"other" level of permissions in Unix. The rationale for
  Cynthia> providing a special case was that someone might choose to use
  Cynthia> "other" as an actual group name, it seemed unlikely that a group
  Cynthia> called "unix_other" would be created.

  Cynthia> My office-mate points out that the UNIX "world"/"other" can be
  Cynthia> thought of as default permissions, as can "everyone" in Windows.

Thinking of group "other" as default permissions is just plain wrong.  Unix
does not have "default" permissions -- it has owner, group, and "everybody
else" permissions.  But the definition of "everybody else" changes on a
file-by-file basis depending on the user and group of the file.

Placing all users into a single group (usually group id 1) and calling that
group "other" is purely convention.  There is no requirement that you put
all users into a single group at all, and there are some decent reasons not
to.  There is no requirement that, if you put them all in a group, you use
group number 1.  And there's no requirement that you call the group "other".

The point is, "other" is not special to Unix -- the kernel does nothing with
it, and the "operating system files" (/usr/bin, etc.) do nothing with it.
The only place it's used is in user file space, and then, there's no actual
requirement to use it.  It may or may not be used the way you think it is
being used.  Other groups may or may not be used the way you expect "other"
to be used.

And at any rate, the manager has no way of knowing anything about any of the
groups you tell it about from a "use this information to deduce something"
standpoint unless it also knows the contents of both /etc/passwd and /etc/group,
which it typically won't.

  [On the subject of using strings and/or XML in the Action element]

  Cynthia> I'd like to make one more argument for the XML. I was thinking of it
  Cynthia> as being similar to the AdditionalData but specific to Action. That
  Cynthia> is, it would make it more convenient for extensions to the Action
  Cynthia> element to be integrated into IDMEF. If you think it makes more
  Cynthia> sense for extensions to Action to be tested in the AdditionalData
  Cynthia> section, I can certainly live with that.

It doesn't make a lot of difference to me, and I can't argue that you're
wrong.  Aesthetically I think we should keep the "free-form xml" part all
in one place, and wait to see what develops, acknowledging that we may be
extending Action in "IDMEF 2."  But that's the full extent of my "argument,"
so I'll just shut up now and go with what the group agrees on.  :-)

--Dave

---

From: Cynthia McLain <[email-masked]>
Date: Mon, 29 Oct 2001 18:29:33 -0500

David> I don't personally have any objection, and it's probably a Good
Thing.  But
  David> it would need to be done (by whoever volunteers to do it)
before the current
  David> November 4th deadline so I can get it put into the last-call
draft, otherwise
  David> it will just have to wait until "IDMEF 2."

I'll send out a list of keywords for an informal vote...

    Cynthia> I chose "File*" in part because of the "original" and
"current"
    Cynthia> attributes for the "File" element. It seemed convenient to
be able
    Cynthia> to embed "File" elements with each of the two attributes
into a
    Cynthia> single "Target" element. Another reason for the "File*" is
that a
    Cynthia> number of files could have been modified at roughly the
same time
    Cynthia> (within an update interval for the host-based IDS), so it
seemed
    Cynthia> more efficient to embed them all in a single "Target"
element since
    Cynthia> the other parts of target could be the same for all.  I
don't feel
    Cynthia> strongly about this issue but do want the decision to use
"File?" to
    Cynthia> be a considered one.

  David> The second situation (multiple files being changed) is
essentially the same
  David> as an attack against multiple hosts, and the current model for
that is to
  David> use multiple Targets, one for each host (file).

  David> The first situation (original and current) though, I had not
thought of, and
  David> I agree with you that it makes sense for those two items to be
in the same
  David> place.  Perhaps we need

  David>   <!ELEMENT Target (
  David>       Node?, User?, Process?, (File, File?)?, Service?
  David>   )>
  David> 
  David> And then some language in the draft that says that if you use
both of those
  David> Files, then they should describe the same file, with one being
"original"
  David> and the other being "current".

Although there are cases when multiple file changes could be considered
equivalent to multiple hosts, for example when the files are on file
systems mounted from different hosts, I have trouble agreeing with that
equivalence for the general case. When the rest of the Target
information
stays constant, it seems sensible to me to be able to specify multiple
File elements in a single Target. Certainly when the other Target
information differs, the File elements should be in different Targets
though.

Because I think there could be confusion (and possible misuse) with
(File, File?)?, my vote would be for either File* or File?, with a
preference for the former.

  [On the subject of "other" -vs- "unix_other" -vs- "default" ]

The following revised FileAccess element is proposed.
Note that GroupId, which was added for the first proposal,
has been dropped. A UserId with type="group-privs" is used
instead. The keyword "other-privs" is added to the entity
attvals.idtype, which is used for the UserId "type" attribute.

++++ FileAccess element ++++

The "FileAccess" element represents the access policy of the file.
The representation of the access policy is intended to be valid
across operating systems. The "FileAccess" element is represented
in XML as follows:

<!ELEMENT FileAccess (
    UserId+, permission+
 )>

The "FileAccess" element consists of two sub-classes:

UserId
   One or more. User to which the full set of permissions apply.
   The "user-privs", "group-privs", and "other-privs" are the only
   valid types. UserIds with other type settings should be ignored
   in this context.

permission
   One or more. STRING. Level of access allowed. Recommended values
   are: "noAccess", "read", "write", "execute", "delete",
   "changePermissions", "takeOwnership", and "executeAs". The
   "changePermissions" and "takeOwnership" strings represent those
   concepts in Windows. On Unix, the owner of the file always has
   "changePermissions" access, even if no other access is allowed
   for that user. 'Full Control' in Windows is represented by
   enumerating the permissions available under 'Full Control'. The
   "executeAs" string represents the setuid and/or setgid settings
   in Unix. A string is used to allow easy extensibility.

Additional UserId type
----------------------

To support use of UserId in the FileAccess element, a new keyword
is needed for the UserId type attribute.

   Rank      Keyword        Description
   ----      -------        -----------
     6       other-privs    Used to specify the "world" or "other"
                            file permission settings in Unix when
                            used in a file context.

---

From: "David A. Curry" <[email-masked]>
Date: Mon, 29 Oct 2001 21:20:30 -0500

>>>>> On Mon, 29 Oct 2001, Cynthia McLain, identified as "Cynthia" below,
>>>>> wrote:

  Cynthia> Although there are cases when multiple file changes could be
  Cynthia> considered equivalent to multiple hosts, for example when the files
  Cynthia> are on file systems mounted from different hosts, I have trouble
  Cynthia> agreeing with that equivalence for the general case. When the
  Cynthia> rest of the Target information stays constant, it seems sensible
  Cynthia> to me to be able to specify multiple File elements in a single
  Cynthia> Target. Certainly when the other Target information differs, the
  Cynthia> File elements should be in different Targets though.

I suppose that could occur, but would you really generate the alert that way?
I would think you'd generate the alert in the context of "somebody is doing
something on this host that modifies this file" with the fact that the file
happens to be stored somewhere else being somewhat peripheral to the alert.
Maybe you include that information if you know it, but otherwise the alert
is identical to one that would be generated for a locally stored file.

I can't really imagine a situation where you'd identify the file from both the
"server" and "client" perspectives in the same alert -- rather, I would expect
to see two separate alerts, probably from different analyzers (e.g., the host-
based one on the client and the host-based one on the server).

Granted, a correlation system might lump them all together into a single bigger
alert, but IDMEF in its current state sort of tacitly assumes just "simple"
analyzers and managers, not correlation engines.  At least, we've made some
other feature-based decisions using that assumption.

So I guess I still think that, for consistency's sake with the way every other
kind of Target is represented (multiple Target elements for multiple targets),
that File should be handled the same way.

  Cynthia> Because I think there could be confusion (and possible misuse) with
  Cynthia> (File, File?)?, my vote would be for either File* or File?, with a
  Cynthia> preference for the former.

I can see your point about possible confusion, although we do have some similar
(but not identical) constructs elsewhere in the DTD.  But I still dont' think
File* is correct, because continuing the reasoning above, you should identify
each file as a separate target.

Getting rid of the (File, File?)? construct, though, breaks the distinction of
"original/current" you had.  Maybe we could try:

	<!ELEMENT File (
	    OriginalAttrs?, CurrentAttrs?
	 )>
	<!ELEMENT OriginalAttrs (
	    name, path, create_time?, modified_time?, accessed_time?,
            data_size?, disk_size?, FileAccess*, Linkage*, Inode?
	 )>
	<!ELEMENT CurrentAttrs? (
	    name, path, create_time?, modified_time?, accessed_time?,
	    data_size?, disk_size?, FileAccess*, Linkage*, Inode?
	 )>
	<!ATTLIST File
	    ident		CDATA		'0'
	    fstype		CDATA		#REQUIRED
	    %attlist.global;

and dispense with the "category" attribute?  The XML tag overhead is about the
same, but this way lets us code all the data about the file in a single File
element.  And that, in turn, lets us do File? inside of Target and get the
results we want.

A minor nit: why is it "create_time," but "modified_time" and "accessed_time"
(i.e., one verb in present tense, two in past tense)?  I think they should all
be in the same tense (my preference is for present tense, but I don't have any
strong feelings either way).

On a related but different topic, I just noticed that you have AdditionalData
inside the File element.  I understand why you did this, but we have carefully
avoided doing it elsewhere (although it's been proposed).  The reasoning has
been that all AdditionalData should be in one single place, making it easier
to either deal with or ignore, depending on the manager receiving it.  The
"popular" things that show up in AdditionalData would then make their way into
revisions of the IDMEF.

So, in the interests of consistency (my "theme for the evening"), I think we
should do the same thing here, and not have an AdditionalData element under
File.

Perhaps we could convince some other folks on the list to chime in with their
opinions on these topics, so this isn't just the "Cindy and Dave Show"?

--Dave

---

From: Andy Walther <[email-masked]>
Date: Tue, 30 Oct 2001 18:32:59 -0800 (PST)

I'll first say that I'm happy that we're finally getting this host/file
information into the spec, and that I'm satified that my own concerns have
been addressed by this proposal.

But I have some thoughts on this topic.

> that File should be handled the same way.

I'm assuming that I missed the discussion as to why the multiple target
elements for multiple targets came about... I know these comments are
coming in late (I apologize if I'm dredging up issus here, but since the
File discussion is relevant, here they are, slap me down if its
inappropriate), but I have two lines of argument here, one with specific
examples, one a little more abtract and logical.

Examples:

You have a forked process (daemon etc) that has multiple PID's associated
with it. A "single" act/action can do something to all of them at the same
time. That would tend to imply that there was a single "target", although
there were peripheral effects. so, if I have an X session open, and I kill
the under lying process for that session, all my shells etc will die, even
if they are't independently targets of their own. I admit that this does
blur the line between simple elements and correlation, but ...

Another example to think about, let's assume I log into a system and
perform the command

rm -rf /

I think this represents an attack on a single target (the / filesystem)
that happens to effect a whole heck of a lot of files, not to mention
multiple phsical filesystems in all likelihood. The collection of files
and filesystems is the real Target (at the IDS level). Unless you want to
include both "File" and "FileList" elements.

Enough for specific examples, I have a more abstract issue with this as
well that may be stronger than these specific examples. I don't really
understand what the difference between having a Target element containing
4 subelements, one User, one Process, one File, and one Service (which is
permitted yes?), and one containing 4 subelements, 4 File(s). At the
abstract level, allowing you to "package" a User target with a Process
target with a File target is no different than "packaging" 4 File targets
together.

But, if you disagree with this assertion (there remains a single File per
Target), would you then need to include the User/Process/Service
information in each of the Target elements that contained one of the
Files?  And if so, then what if there are multiple User's targeted at the
same time... you have to double or triple your Target elements and start
to have a rapidly increasing Alert size. And if not, why not. How is a
manager supposed to interpret an Alert with a Target consisting of 1 File,
1 User, and 1 Process element, and another Target with only a single File.
If you say "it'll just aggregate them all together", whats the difference
between putting them all in one Target then.

I think the only real choices are to permit only 1 of
File/Process/User/Service/Node for each Target, or permit any mixture of
these elements (File*, Process*, User*, Service*, Node*) for a given
Target in order to "stay consistent" while also remaining realistic.

If its too late for the others, I still recommend that (File*) remains. I
don't see a generic "consistency" argument as valid against the File*
recommendation.

> should do the same thing here, and not have an AdditionalData element under

I agree.

> opinions on these topics, so this isn't just the "Cindy and Dave Show"?

sorry for the delay.

Andy

---

From: "David A. Curry" <[email-masked]>
Date: Wed, 31 Oct 2001 08:47:11 -0500

>>>>> On Tue, 30 Oct 2001, Andy Walther, identified as "Andy" below, wrote:

  Andy> But, if you disagree with this assertion (there remains a single File
  Andy> per Target), would you then need to include the User/Process/Service
  Andy> information in each of the Target elements that contained one of the
  Andy> Files?

The classes in Target are not necessarily expected to be present in every
alert.  That's why they're all "zero or one."  For example, a network-based
analyzer might know that Service "X" on Node "Y" is being targeted, but it
will have no way of knowing what User or Process will be associated with
that service.

Certainly, one can contrive a situation in which the alert would use all of
the classes.  And, one can contrive alerts in which you might use more than
one of the classes.  But I think we need to back off of this a bit, and do
a little more thinking about what kind of alerts we're *really* going to
generate, as opposed to what kind we *could* generate given infinitely
powerful, all-knowing analyzers.

To pick on your examples a bit (sorry):

  Andy> You have a forked process (daemon etc) that has multiple PID's
  Andy> associated with it. A "single" act/action can do something to all of
  Andy> them at the same time. That would tend to imply that there was a single
  Andy> "target", although there were peripheral effects. so, if I have an X
  Andy> session open, and I kill the under lying process for that session, all
  Andy> my shells etc will die, even if they are't independently targets of
  Andy> their own. I admit that this does blur the line between simple elements
  Andy> and correlation, but ...

I have difficulty imagining a system that could actually generate an alert for
this as you describe it.  You would have to track down the parent process of
every single process that terminates on the system, and then decide whether
that parent process was attacked to make it exit.  And that's *easier* than
what you describe -- to generate the alert you describe, I would have to, for
each process that terminates, track down every socket connection it has, and
determine whether the other end's process has terminated, and oh, by the way,
when it did terminate, was that because of an attack, or just normal exit?
The complexity is going way up, real fast.

Realistically, you're going to see an alert that someone killed the X server.
And that's all you're going to see.

  Andy> Another example to think about, let's assume I log into a system and
  Andy> perform the command
  Andy> rm -rf /
  Andy> I think this represents an attack on a single target (the / filesystem)
  Andy> that happens to effect a whole heck of a lot of files, not to mention
  Andy> multiple phsical filesystems in all likelihood. The collection of files
  Andy> and filesystems is the real Target (at the IDS level). Unless you want
  Andy> to include both "File" and "FileList" elements.

You're not seriously going to generate such an alert, even if the data model
lets you do it.  Your analyzer would be so busy printing out each and every
path name on the system, that it would do nothing else for an extended amount
of time.

Realistically, you're going to see an alert that someone just deleted "/", and
that's it.  Maybe some "secondary" alerts as random files being monitored get
deleted, and stuff like that, but you're not going to see an alert that just
lists every file on the system.  Nor are you going to see individual alerts
for each file, since there'd be no way to tell *why* it was deleted (normal
or abnormal), except for certain "special" files.

I keep coming back to a discussion at one of the interim meetings regarding the
overall goal here -- are we trying to come up with a representation that can be
used with today's technology and that of the near future, or are we trying to
come up with the One True Format that can be used from now until the heat-death
of the universe?  The agreement, as I recall it (and the chairs can correct me
if I'm wrong), was that we're trying to do the former -- we want to make
something we can use today, and that doesn't purposely preclude new things in
the future, but we cannot predict the future and make it all things to all
people.

  Andy> Enough for specific examples, I have a more abstract issue with this
  Andy> as well that may be stronger than these specific examples. I don't
  Andy> really understand what the difference between having a Target element
  Andy> containing 4 subelements, one User, one Process, one File, and one
  Andy> Service (which is permitted yes?), and one containing 4 subelements,
  Andy> 4 File(s). At the abstract level, allowing you to "package" a User
  Andy> target with a Process target with a File target is no different than
  Andy> "packaging" 4 File targets together.

The problem with the old (non-repeating) Target was that you could not tell
if all the information contained there was actually related or not, or how
it was related.  For example, consider a port scanner that does operating
system identification, so it runs one set of scans against NT and another
against Unix.  Now it hits two hosts, so you list two Nodes in the Target,
and you list all the Services that it hits.  But which Services were on
Node A, and which were on Node B?  Who knows?  Can't tell from the alert.

The repeating Target allows this problem to be addressed -- this Service
got hit on Node A, this Service got hit on Node B, and so on.  Yes, you can
argue that I should be able to list all the Services in one Target, and you're 
probably right.  But to allow that, we have to allow the other behavior above,
and that's no good.

So, we solved the problem at the expense of additional XML tagging.  Is it
ideal?  No.  Is it better?  Yes.  And if you're running over a compressed
data stream (such as you can with IDXP), the extra tagging compresses down
to "almost" nothing anyway, making it not so bad.

Oh yeah -- in answer to your question way back up at the top of this note,
yes, you do have to repeat all the information in each Target; the ability to
reference things in multiple places without repeating it all was removed at
the last interim meeting because it has some serious issues we did not know
how to address.

--Dave

---

From: Andy Walther <[email-masked]>
Date: Wed, 31 Oct 2001 09:53:58 -0800 (PST)

> powerful, all-knowing analyzers.

yes, but my other point is that we should also think about what Managers
want to see too.

> To pick on your examples a bit (sorry):

that's fine.

> the future, but we cannot predict the future and make it all things to all

True enough. And I guess this last example is overblown, but an example
sensor that people are using (Tripwire) would tend to create a separate
Target for each file deleted, and having to carry along the baggae of a
Node grouping for each of these exacerbates the overly long message that
would result. I also wonder how a Manager would tend to deal with this
situation since it'll have alot of redundant information.

> the last interim meeting because it has some serious issues we did not know

I'm not too attached to this idea. It makes alot of sense to me, but I
know that there were discussions previously, and I don't see it as a huge
sticking issue in terms of getting this thing out the door. I guess that
means we need to really just get down to trying to decide how we might
deal with the "change" concept that is inherent in the File idea (I don't
know if there are any other situations where this might be appropriate to
look at, maybe if something about a User changed)?

Andy

---

From: Cynthia McLain <[email-masked]>
Date: Wed, 31 Oct 2001 14:43:37 -0500

Thought it was about time for me to chime in again...

  [AdditionalData]

Guess I got carried away with the AdditionalData. I agree
that only one element, Alert, should contain AdditionalData.

   [create_time vs created_time]

Dave's proposal to change the name for consistency with the
other elements makes sense to me.

   [File* vs File?]

  Andy> True enough. And I guess this last example is overblown, but an
example
  Andy> sensor that people are using (Tripwire) would tend to create a
separate
  Andy> Target for each file deleted, and having to carry along the
baggae of a
  Andy> Node grouping for each of these exacerbates the overly long
message that
  Andy> would result. I also wonder how a Manager would tend to deal
with this
  Andy> situation since it'll have alot of redundant information.

I think having multiple file changes show up at one time is a
valid scenario for any host-based IDS that incorporates a change
detection system. In that case, the system runs at intervals and
examines files on the local disk for changes. It may be that during
a single run, a number of files are found to have been changed
and that an alert is generated for the collection rather than for
each file. As Andy points out, there is a single target which can
be described by the Node (User, Process, and Service would likely
not be relevant in this case). This is one case when using File*
makes sense. For IDSs that analyze auditing information or network
packets, the file information would probably appear as separate
Alert messages so a single File element would suffice.

   [OriginalAttrs/CurrentAttrs]

> > element.  And that, in turn, lets us do File? inside of Target and get the

Supposing that we don't go with File*, how about the
following representation of the OriginalAttrs/CurrentAttrs
idea? This definition supposes that the file name and
path are the same for both the original and current.
There is some additional tag overhead from FileAttrs being
shared by OriginalAttrs and CurrentAttrs but it ensures
that file attributes will be consistently defined (and
we know Dave likes consistency).

<!ELEMENT File (
   name, path, (OriginalAttrs | CurrentAttrs | (OriginalAttrs,
CurrentAttrs))?
 )>
<!ATTLIST File
     ident               CDATA           '0'
     fstype              CDATA           #REQUIRED
     %attlist.global;

<!ELEMENT FileAttrs (
    created_time?, modified_time?, accessed_time?,
    data_size?, disk_size?, FileAccess*, Linkage*, Inode?
 )>
<!ELEMENT OriginalAttrs (
    FileAttrs
 )>
<!ELEMENT CurrentAttrs (
    FileAttrs
 )>

This would yield the following construct:

<File fstype="ufs">
  <name>foo</name>
  <path>/usr/bin/foo</path>
  <OriginalAttrs><FileAttrs>
    <data_size>134</data_size>
  </FileAttrs></OriginalAttrs>
  <CurrentAttrs><FileAttrs>
    <data_size>156</data_size>
  </FileAttrs></CurrentAttrs>
</File>

--Cindy

---

From: "David A. Curry" <[email-masked]>
Date: Wed, 31 Oct 2001 15:33:11 -0500

>>>>> On Wed, 31 Oct 2001, Cynthia McLain, identified as "Cynthia" below,
>>>>> wrote:

  Cynthia> I think having multiple file changes show up at one time is a valid
  Cynthia> scenario for any host-based IDS that incorporates a change detection
  Cynthia> system. In that case, the system runs at intervals and examines
  Cynthia> files on the local disk for changes. It may be that during a single
  Cynthia> run, a number of files are found to have been changed and that an
  Cynthia> alert is generated for the collection rather than for each file. As
  Cynthia> Andy points out, there is a single target which can be described by
  Cynthia> the Node (User, Process, and Service would likely not be relevant in
  Cynthia> this case). This is one case when using File* makes sense.

Okay, you've convinced me, a single target with multiple files is a reasonable
thing.  I'm still not too keen on File*, and so will try one last proposal and
see if you like it:

	<!ELEMENT FileList (
 	    File+		      <-- note: one or more, not zero or more
	  )>

	<!ELEMENT Target (
	    Node?, User?, Process?, Service?, FileList?	<-- note: zero or one
	  )>

This is what we did with User to avoid having a bunch of UserId elements all
over the place, so it's consistent with that (and where would I be without
banging the consistency drum one more time? :-)

  Cynthia> This definition supposes that the file name and path are the same
  Cynthia> for both the original and current.

  Cynthia> <!ELEMENT File (
  Cynthia>     name, path, (OriginalAttrs | CurrentAttrs |
  Cynthia>         (OriginalAttrs, CurrentAttrs))?
  Cynthia>   )>

Do the name/path really never change from original to current?  What if a file
is renamed (the inode would not change, in Unix, anyway, so you could match
them up).  I almost split them out in my last note too, but then the rename
thought occurred to me, and I didn't.  I don't feel strongly about it one
way or the other; I'd say go ahead and do what you propose if there is no
need to support a "renamed" alert.

BTW, (O | C | (O, C))? is identical to O?, C?, so there's no need to re-do
that part, and you can just have

	<!ELEMENT File (
	    name, path, OriginalAttrs?, CurrentAttrs?
	  )>

  Cynthia> <!ELEMENT FileAttrs (
  Cynthia>     created_time?, modified_time?, accessed_time?,
  Cynthia>     data_size?, disk_size?, FileAccess*, Linkage*, Inode?
  Cynthia>   )>
  Cynthia> <!ELEMENT OriginalAttrs (
  Cynthia>     FileAttrs
  Cynthia>   )>
  Cynthia> <!ELEMENT CurrentAttrs (
  Cynthia>     FileAttrs
  Cynthia>   )>

I see what you're doing here, but you could eliminate the FileAttrs element
and accomplish the same thing (functionally) with:

	<!ENTITY % classlist.attrs	"
	    created_time?, modified_time?, accessed_time?, data_size?,
	    disk_size?, FileAccess*, Linkage*, Inode?
	  ">

	    %classlist.attrs;
	  )>

	<!ELEMENT CurrentAttrs (
	    %classlist.attrs;
	  )>

So then you'd have:

  <FileList>
    <File fstype="ufs">
      <name>foo</name>
      <OriginalAttrs>
      </OriginalAttrs>
      <CurrentAttrs>
      </CurrentAttrs>
    </File>
    <File fstype="...">
      ...
    </File>
    <File fstype="...">
      ...
    </File>
  <FileList>

---

From: Andy Walther <[email-masked]>
Date: Wed, 31 Oct 2001 13:15:34 -0800 (PST)

> banging the consistency drum one more time? :-)

Yeah, I fully concur with the FileList concept as a good way of
accomplishing what we want to do, and is both consistent with what the
IDS's are trying to do as well as consistent with the rest of the spec.

Do we need/want to deal with the OriginalAttrs/CurrentAttrs if we go with
the Filelist idea, or stick with Cindy's originaly proposal of how to deal
with them as two separate File objects?

Andy

---

From: Cynthia McLain <[email-masked]>
Date: Wed, 31 Oct 2001 16:42:22 -0500

> banging the consistency drum one more time? :-)

Seems reasonable to me. I like Andy's suggestion that
with the FileList we use the old style File. It reduces
the number of elements, which seems like "a good thing",
and eliminates the need for a decision about where "name"
and "path" should reside. (I did think the entity definition
was a nice touch though).

--Cindy