Subject: Re: Cryptic Vs Verbose IDMEF Alerts

From: Hervé Debar <[email-masked]>
Date: Mon, 3 Dec 2001 15:08:46 +0100 (CET)

I haven't contributed for a while, but I'll take the opportunity of the
last call to make the following points:

- Concerning Tivoli RiskManager: yes, the original data model draft
(-00) was a direct relationship to that work (I wrote quite a bit of the
RM code). Due to a limitation in the TEC, event language, the draft was
only inheritance. This reflected in some clumsy constructions and a lot
of constraints on the implementors. While I freely admit that clumsiness
was undesirable, I still think that a lot of the constrainst were not
bad, because they made a lot of sense to a corelation engine or an
analyst, and they would have forced the vendors to do better than what
they do now. E.G one of the constrains of the -00 data model was that if
you could not give me a target for the attack, you could not give me a
source. While reading this on a mailing list may cause people to send me
flames, I believe that this kind of constraint would have pushed vendors
into thinking "what do I put in there that makes sense" instead of
"where do I throw this data that's available at no cost". BTW, I am not
associated with Tivoli/IBM anymore. Further questions concerning
RiskManager should be addressed to Andreas Wespi (you'll find his
address on the mailing list) or you can read our RAID'2001 paper.

- Concerning verbosity. My interest in this is really the data model,
not the XML. As such, I have an UML design, which of course not only
describes the data but also the way to access it. UML also allows me to
define constraints in the data that result in a semantically more
constrained model. I like that, but it is definitely not the way the
group wanted to go, and it's not so important. What is important is that
XML is just a vehicle; nothing prevents anybody from going directly (as
I do) to C++, java, SQL, or any thing you want to implement. As long as
there is a way to go back to the UML at the edge of any system, then
we're probably fine.

- Concerning the data model. Some things are a bit clumsy. We have
redone a bit of work to reduce the number of different objects and base
a lot of things on relationships. This is still research work and may
benefit the group at a later point in time. Also, this work encompasses
events (network packets, firewall logs, vulnerability scan reports) and
is much more comprehensive from an operator and analyst's point of view.
This vision was probably missing from the work done at IBM (although to
be fair, I have been told that RiskManager now understands firewall logs
and antivirus alerts).

- Concerning AdditionalData. The -00 draft didn't need that, because
inheritance is a much cleaner way of doing extentions. Again, I got a
thourough beating from the group on inheritance, which is why the
current data model is almost aggregation-only, and has to resort to
AdditionalData-like schemes. I'm not so optimistic as Dave into saying
that AdditionalData should not be used much. My own fear is that vendors
will use it a lot, and that this will render the standard less useful
for users. Time will tell, and I'm willing to wait until I see what
happens. On the other hand, doing what Stuart does and putting "hidden"
tags reminds me of the C2 audit/BSM/Shield history. Anybody who has
coded a Solaris-based HIDS in the past 10 years will know what I mean.

End of random (and probably useless) thoughts :-)

Hervé
-- 
Hervé Debar                <mailto:[email-masked]>
Tel: +33 (0)2 31 75 92 61               GSM: +33 (0)6 74 09 09 66
France Télécom R&D                      Fax: +33 (0)2 31 75 93 13
42 rue des Coutures  (-/-)  BP 6243  (-/-)   F-14066 Caen Cedex 4

---

From: Paul Sangree <[email-masked]>
Date: Mon, 03 Dec 2001 12:35:55 -0600

At 04:10 PM 11/30/2001 -0800, Stuart Staniford wrote:
>>Standard compression is a good solution when you have both processing
>>power and batched operations (compressing and decompressing individual
>>events is not very effective). In the kind of environments that
>>we operate in, we find that a firewall can easy generate a few hundred
>>messages per second; and if we are managing multiple firewalls, the
>>message rates goes up to thousands/sec. Even though both batching of
>>events and increasing the processing power is doable, there is a
>>definite price to be paid!
>
>Yeah - handling firewall logs was definitely not part of our requirements 
>(no firewall people were involved in the WG for one).  At the requirements 
>phase, it was pretty clear that no-one was contemplating using this for 
>thousands of alerts per second, so that didn't drive the thinking.
>
>So is the WAP binary idea dead?  (I haven't followed it).  Some of the 
>Cisco guys who were implementing IDMEF a while back had the same idea.
>(Are you still here Paul?)

Our take on this is that communications bandwidth is probably not going to 
limit our overall throughput (this may not apply to other systems).  The 
real bottleneck for us will more likely be CPU cycles.  Therefore 
compression may hurt more than it will help.  I agree with Azim that 
verbosity is a problem, and I also think nesting elements deeply is not a 
good idea because it makes the parser more complex and 
inefficient.  Regardless of how fast the CPU runs, it will probably never 
be as fast as we would like.  However, these are "shades of gray" issues, 
not fundamental flaws.

Paul

---

From: "David Chizmadia" <[email-masked]>
Date: Tue, 4 Dec 2001 08:23:11 -0500

All,

    Reading Hervé's comments, below, leads me to ask whether the data model
document will be updated once the IDMEF document moves to RFC status.

    For quite some time, I've been considering using the final IDMEF RFC -
or preferably an Informational IDMEF Data Model RFC - as the basis for an
OMG RFP that would complement the IDWG work both by creating an alternative
encoding format based on OMG IDL and GIOP CDR (Common Data Representation)
and by creating an administrative and operational model of the services
(e.g., actors) implied by the IDMEF. If there is a normative (UML) model of
the IDMEF encoding, this would be perfect since the OMG is moving towards
the use of UML as the normative notation for its specifications, with
mappings to CORBA and other middleware technologies to reify the model.

    Obviously, this would also address some of the verbosity concerns
expressed in other messages, since CDR (while not as quite as compact as a
custom binary encoding) doesn't have the direct overhead of XML tagging and
can be automatically generated for most common programming languages using
either commercial or open source tools.

-DMC
David Chizmadia
Senior Software Security Architect
Promia, Inc
[email-masked]

----- Original Message -----
From: "Hervé Debar" <[email-masked]>
To: <[email-masked]>
Sent: Monday, December 03, 2001 9:08 AM
Subject: Re: Cryptic Vs Verbose IDMEF Alerts

Hervé

---

From: Stuart Staniford <[email-masked]>
Date: Tue, 4 Dec 2001 23:05:38 -0800

> document will be updated once the IDMEF document moves to RFC status.

No plans to do that - we deliberately merged the data model and XML docs 
into the IDMEF doc (because when the two were separate it was a constant 
pain to keep them in sync, being as how almost everything in one was 
duplicated in the other).  I say "we" - it was really Herve DeBar and 
Dave Curry that did the work.

Stuart.