Subject: Required changes to draft-ietf-idwg-idmef-xml-14

From: Sam Hartman <[email-masked]>
Date: Thu, 26 Jan 2006 17:51:00 -0500

Hi.  I apologize for the long time it has taken me to review the idmef
draft.

One problem with the IESG job is that if something has been delayed
for a long time, you get into the trap of assuming that additional
delay won't hurt much more and so you focus on things that are not yet
behind.  Clearly this is not the correct strategy but it seems to be
the trap I've fallen into here.

I have also been slow because I needed to seek significant input from
the rest of the IESG.

When e last talked, I was going to go off and review the document.  I
said that if no significant changes were needed I'd take it to the
IESG and not think too hard about whether there is still an active
working group here.  

Unfortunately, I believe there are changes required.  So I have
evaluated the question of whether there has been enough active
participation in the last versions of the document and in the first
part of this year to consider IDWG an active working group.  Based on
my own conclusions and on the strong opinions of other area directors
I've consulted with, IDWG no longer constitutes an active working
group.  There is an insufficient core of active
participants--particularly participants not involved in the
document--to generate an informed consensus on changes.

However it is desirable to publish IDMEF in at least some form.  The
IESG said it would be willing to support publication of IDMEF as an
experimental RFC provided that some consistency problems are fixed.
The existing protocol document would be reclassified as an
experimental RFC.

So, what changes are required?

The only critical change is that while the schema is normative, all
the document is written in terms of the DTD not the schema.  As an
example, IDMEF messages are required to include DTD declarations, and
all of the class semantics are written in terms of the schema.

For this reason, I do not believe the schema can be normative.  Also,
based on discussing this transition with others, I have very low
confidence that the schema and DTD will be consistent.  

So I request that the schema be removed and the DTD become normative
again.

We can discuss alternatives, but you will need to convince me that the
resulting document is self consistent and that you have the necessary
resources to make that evaluation.

If we have not made significant progress by the end of IETF 65 (end of
March), then I will close the WG and mark the documents as dead.

---

From: Herve Debar <[email-masked]>
Date: Sun, 29 Jan 2006 20:20:01 +0100

Sam,

thanks for the feedback. Comments and answers below.

> document--to generate an informed consensus on changes.

Yes, that is certainly true of late. However, this document does have an
active user base of implementors, particularly in the academic and
research community. As such, it has gotten quite a bit of feedback and
fine-tuning.

> all of the class semantics are written in terms of the schema.

I do not believe that this is fully the case. The document has DTD
snippets to illustrate the model, but they can be easily replaced by
schema snippets. In fact, I have a version of the IDMEF draft that does
this immediately available for you.

> resources to make that evaluation.

The reasons in favor of the schema, as I see them, are as follows:

- The move from DTD to schema was validated by the WG (when it was more
active) a long time ago.

- The schema introduces typing, which is a big facilitator in our case.
Documents that are valid within the DTD (considering all data as
strings) may not be valid with the schema.

- DTD and schema have been evolving side by side for a long time. They
validate the same example messages and others generated by several IDMEF
output software. As such I believe that they are equivalent.

This last reason is why -- altough i firmly believe that a schema is
better -- I am ready to comply with your requested change and send you
the final draft in this form. I do ask though that the schema be placed
as an appendix.

BTW, what kind of evaluation would convince you ? could you precise what
you have in mind ?

> March), then I will close the WG and mark the documents as dead.

Well, either way I will send you the updated IDMEF draft soon.

Best regards,

Hervé
-- 
Hervé Debar             <mailto:[email-masked]>
Tel: +33 (0)2 31 75 92 61            GSM: +33 (0)6 74 09 09 66
France Télécom R&D              (new)Fax: +33 (0)2 31 37 83 43
42 rue des Coutures  (--)  BP 6243  (--)  F-14066 Caen Cedex 4

---

From: Sam Hartman <[email-masked]>
Date: Mon, 30 Jan 2006 18:36:11 -0500

>>>>> "Herve" == Herve Debar <[email-masked]> writes:
    Herve> BTW, what kind of evaluation would convince you ? could you
    Herve> precise what you have in mind ?

The consensus of an active working group of experts in the area who
had enough reviewers that I could reasonably believe there was an
informed consensus.

So, basically you'd need an active working group.

I'll ask the IESG about a schema appendix, but I suspect we can work
that out.

---

From: "Dr. Robert G. Rains" <[email-masked]>
Date: Mon, 30 Jan 2006 23:45:45 -0500

Gentlemen,

I have a question regarding a "standard" that apparently originated at 
DARPA in the late 90's and then transitioned from DARPA to the IDWG so 
that the IDWG could refine the specification and further its 
acceptance.  The "standard" is the Defense Advanced Research Program 
Agency (DARPA) "Common Intrusion Detection Framework" (CIDF).  I'm 
working on standards for Anti-Terrorism/Force Protection (AT/FP), and an 
integration contractor has proposed that the DoD adopt the "DARPA 
Intrusion Detection Systems" aka "Common Intrusion Detection Framework" 
(CIDF) as a "standard" to be imposed on AT/FP systems.

MY CURRENT UNDERSTANDING
It appears that the standard titled, "DARPA Intrusion Detection Systems (Network)" is not a standard.  It seems to be an area of research.  Although DARPA has funded some leading edge research into concepts for intrusion detection, DARPA is not currently the lead for development of standards related to intrusion detection.  An April 1999 DARPA report stated, "... intrusion detection systems must be able to interoperate with security screens, such as network firewalls, and with response mechanisms that can be used to contain and/or mitigate the effects of an attack.  To enable such interoperation, DARPA has developed a Common Intrusion Detection Framework that allows network components to exchange information and to work together to respond to network-based attacks. The Internet Engineering Task Force, the organization that maintains the standards that govern the Internet, will refine the specification and further its acceptance."  I understand that the standards portion of t!
 he activity was being pursued under the auspices of the IETF.  A somewhat dated history of the CIDF effort may be found at http://www.isi.edu/gost/cidf.  However, many of the organizational references, e-mail addresses, and URLs on that page are now out of date.   Development of intrusion detection standards was being pursued by the Intrusion Detection Exchange Format Working Group (idwg) of the Internet Engineering Task Force (IETF).  However, in the e-mail chain below, Sam Hartman states, "Based on my own conclusions and on the strong opinions of other area directors I've consulted with, IDWG no longer constitutes an active working group.  There is an insufficient core of active participants--particularly participants not involved in the document--to generate an informed consensus on changes."

QUOTE FROM THE CONTRACTOR SUBMITTAL PROPOSING THE CIDF AS A STANDARD
The Defense Advanced Research Program Agency (DARPA) "Common Intrusion Detection Framework" (CIDF) specifies a set of concepts and guidelines that provide a framework within which Intrusion Detection Systems (IDS) can be understood, compared and designed. An IDS system's architecture, capabilities and features can be defined in relation to relationships with the DARPA CIDF.  The IETF has formed an Intrusion Detection Working Group (IDWG) that is also providing guidance in regard to IDS standards.  The Gartner Groups "Seven Key Selection Criteria for Network Intrusion Prevention Systems (IPS)" is also a valuable framework for IDS and IPS. The OASIS Security Assertion Markup Language (SAML 2.0) also relates to standard security measures.

REFERENCES
1)	http://www.toplayer.com/pdf/GartnerBrief.pdf
2)	http://www.isi.edu/gost/cidf/
3)	http://www.isi.edu/gost/cidf/drafts/communication.txt
4)	http://www.isi.edu/gost/cidf/drafts/language.txt
5)	http://www.isi.edu/gost/cidf/tutorial.html
6)	http://www.isi.edu/gost/cidf/drafts/api.txt
7)	http://www.ietf.org/internet-drafts/draft-ietf-idwg-requirements-10.txt
8)	http://www.ietf.org/internet-drafts/draft-ietf-idwg-idmef-xml-12.txt
9)	http://www.ietf.org/internet-drafts/draft-ietf-idwg-beep-idxp-07.txt
10)	http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=security

MY CURRENT CONCLUSION
In conclusion... I don't see any evidence that the DARPA CIDF ever became a commercially accepted standard.  None of the above references seem to refer to a DARPA web site or to a mature, commercially accepted standard.  I understand that, at this time, the IDWG is for all practical purposes, not an active working group.  It appears that this is an area of research that DARPA was interested in for awhile back in the late 90's, and for which DARPA eventually gave responsibility to the IETF to refine the "standard" and seek its acceptance.  It isn't clear that the CIDF ever became a mature, commercially accepted "standard" which would lead to commercial products that comply with the standard being developed and placed on the market.

I would sincerely welcome any additional information that you might have which would shed light on the status of the CIDF.

ONE LAST QUESTION:
Did draft-ietf-idwg-idmef-xml-14 expire on 31 July 2005, or has it been adopted as a standard?  To the best of your knowledge, are at least two different commercial organizations developing products based on this draft standard?

Thanks in advance,

Robert Rains

*****************************************************************
Robert G. Rains, Ph.D.
Force Protection C2 Systems Engineer
Electronic Systems Center
Force Protection Systems Squadron, FPSS/FPTE
5 Eglin St, Bldg 1624
Hanscom AFB, MA 01731-2100
Phone: 781-377-4744
E-Mail:  [email-masked]
*****************************************************************

=============================================================

---

From: "S. Felix Wu" <[email-masked]>
Date: Mon, 30 Jan 2006 22:44:17 -0800

Robert,

Your conclusion is mostly correct.

CIDF was a DARPA funded research effort leaded by Stuart Staniford and
many other DARPA PI's in late 90. The objective was to standardize it
via the IETF process. However, the first BOF in LA, I still remember,
was not very well received by the audience due to the potential
complexity and expressiveness of s-expressions. But, it was clear from
the BOF that there was a significant interest in having an industry
standard.

Therefore, the IDWG working group was formed, and Stuart and Mike were
the first chairs for the working group. IDWG basically abandoned CIDF
and S-expression (at least initially) and started from "requirement
study". We then ran into the debate of issues such as SNMP versus XML.
S-expression was not discussed very much after that point.

I personally believe that an industry standard is still needed as we
have many more powerful networ/host IDS systems today. And, from the
research community, I feel that we know a lot better about what should
be standardized than say 7~8 years ago. And, I see a critical need for
interoperability and inter-domain security operations now and in the
future. I think we should think about all the issues again.

-Felix

Dr. Robert G. Rains wrote:
> It appears that the standard titled, "DARPA Intrusion Detection Systems 
> (Network)" is not a standard.  It seems to be an area of research.  
> Although DARPA has funded some leading edge research into concepts for 
> intrusion detection, DARPA is not currently the lead for development of 
> standards related to intrusion detection.  An April 1999 DARPA report 
> stated, "... intrusion detection systems must be able to interoperate 
> with security screens, such as network firewalls, and with response 
> mechanisms that can be used to contain and/or mitigate the effects of an 
> attack.  To enable such interoperation, DARPA has developed a Common 
> Intrusion Detection Framework that allows network components to exchange 
> information and to work together to respond to network-based attacks. 
> The Internet Engineering Task Force, the organization that maintains the 
> standards that govern the Internet, will refine the specification and 
> further its acceptance."  I understand that the standards portion of t!
> he activity was being pursued under the auspices of the IETF.  A 
> somewhat dated history of the CIDF effort may be found at 
> http://www.isi.edu/gost/cidf.  However, many of the organizational 
> references, e-mail addresses, and URLs on that page are now out of 
> date.   Development of intrusion detection standards was being pursued 
> by the Intrusion Detection Exchange Format Working Group (idwg) of the 
> Internet Engineering Task Force (IETF).  However, in the e-mail chain 
> below, Sam Hartman states, "Based on my own conclusions and on the 
> strong opinions of other area directors I've consulted with, IDWG no 
> longer constitutes an active working group.  There is an insufficient 
> core of active participants--particularly participants not involved in 
> the document--to generate an informed consensus on changes."
> 
> The Defense Advanced Research Program Agency (DARPA) "Common Intrusion 
> Detection Framework" (CIDF) specifies a set of concepts and guidelines 
> that provide a framework within which Intrusion Detection Systems (IDS) 
> can be understood, compared and designed. An IDS system's architecture, 
> capabilities and features can be defined in relation to relationships 
> with the DARPA CIDF.  The IETF has formed an Intrusion Detection Working 
> Group (IDWG) that is also providing guidance in regard to IDS 
> standards.  The Gartner Groups "Seven Key Selection Criteria for Network 
> Intrusion Prevention Systems (IPS)" is also a valuable framework for IDS 
> and IPS. The OASIS Security Assertion Markup Language (SAML 2.0) also 
> relates to standard security measures.
> 
> REFERENCES
> 7)    
> http://www.ietf.org/internet-drafts/draft-ietf-idwg-requirements-10.txt
> In conclusion... I don't see any evidence that the DARPA CIDF ever 
> became a commercially accepted standard.  None of the above references 
> seem to refer to a DARPA web site or to a mature, commercially accepted 
> standard.  I understand that, at this time, the IDWG is for all 
> practical purposes, not an active working group.  It appears that this 
> is an area of research that DARPA was interested in for awhile back in 
> the late 90's, and for which DARPA eventually gave responsibility to the 
> IETF to refine the "standard" and seek its acceptance.  It isn't clear 
> that the CIDF ever became a mature, commercially accepted "standard" 
> which would lead to commercial products that comply with the standard 
> being developed and placed on the market.
> 
> I would sincerely welcome any additional information that you might have 
> which would shed light on the status of the CIDF.
> 
> ONE LAST QUESTION:
> Did draft-ietf-idwg-idmef-xml-14 expire on 31 July 2005, or has it been 
> adopted as a standard?  To the best of your knowledge, are at least two 
> different commercial organizations developing products based on this 
> draft standard?
> 
> Thanks in advance,
> 
> Robert Rains
>

---

From: Herve Debar <[email-masked]>
Date: Tue, 31 Jan 2006 15:31:15 +0100

Robert,

I agree with Felix's statements.

> S-expression was not discussed very much after that point.

The most difficult point of the IDWG work has been the data model.

> future. I think we should think about all the issues again.

That is true. However, given the IDMEF experience, I would really like
that this becomes experimental RFC, that the WG in its current form
closes, and that we start a new effort.

>> Did draft-ietf-idwg-idmef-xml-14 expire on 31 July 2005, or has it
>> been adopted as a standard?  To the best of your knowledge, are at
>> least two different commercial organizations developing products based
>> on this draft standard?

It hasn't yet, but getting close.

Yes, there are several companies that have developped products based on
the standard:

- France Telecom
- Exaprotect
- Prelude IDS technologies

are the ones that come immediately to my mind (caveat: I work for FT). I
can give you contact names and email should you wish to investigate
further with these companies.

Besides, I have seen numerous references to IDMEF in academic papers,
the open source prelude-ids project is based on IDMEF, and there have
been other open source implementations.

Best regards,

Hervé