Subject: [mile] JSON data format

From: Tomáš Čejka <[email-masked]>
Date: Wed, 22 Jul 2015 15:24:53 +0200

Hi all,

I'm quite new in this group and even though I went through all documents
(https://datatracker.ietf.org/wg/mile/documents/), I still haven't read
the whole mailing list archive. That is why I'm sorry in advance in case
my question was discussed anywhere in the past...

As far as I understand the documents, there is a data model that
describes (IODEF) information about security incidents that would be
useful to share. As a proposed data format, the documents use XML.

I would like to ask: would be here a rough consensus to prepare a draft
that would standardize JSON as a compatible data format to IODEF?

>From the technical point of view, I think it would be sufficient to
define a mapping between IODEF and JSON.

My question is motivated by our running code that uses JSON.
I'm a developer of the Nemea project [1] (current version will be
published on github.com soon) developed by CESNET [2] (Czech NREN operator).
Nemea contains detection modules that send discovered alerts in IDEA
format [3] (according to JSON schema [4]). Alerts can be currently sent
into e.g. Warden system [5].

Thank you for your opinions.
Regards,
Tomas Cejka

[1] https://www.liberouter.org/nemea/
[2] http://www.ces.net
[3] https://csirt.cesnet.cz/en/idea/index
[4] https://csirt.cesnet.cz/en/idea/definition
[5] https://wardenw.cesnet.cz/en/index

---

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Thu, 23 Jul 2015 18:02:24 +0000

Actually that has been a question for many XML based formats. How XML is not very flexible and how other languages like JSON are better. There has been work in the STIX community (that uses XML also) to create JSON conversions. I think mappings between IODEF XML to JSON would also be beneficial.

Panos

-----Original Message-----
From: mile [mailto:[email-masked]] On Behalf Of Tomáš Cejka
Sent: Wednesday, July 22, 2015 9:25 AM
To: [email-masked]
Subject: [mile] JSON data format

Hi all,

I'm quite new in this group and even though I went through all documents (https://datatracker.ietf.org/wg/mile/documents/), I still haven't read the whole mailing list archive. That is why I'm sorry in advance in case my question was discussed anywhere in the past...

As far as I understand the documents, there is a data model that describes (IODEF) information about security incidents that would be useful to share. As a proposed data format, the documents use XML.

I would like to ask: would be here a rough consensus to prepare a draft that would standardize JSON as a compatible data format to IODEF?

My question is motivated by our running code that uses JSON.
I'm a developer of the Nemea project [1] (current version will be published on github.com soon) developed by CESNET [2] (Czech NREN operator).
Nemea contains detection modules that send discovered alerts in IDEA format [3] (according to JSON schema [4]). Alerts can be currently sent into e.g. Warden system [5].

_______________________________________________
mile mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mile

---

From: "Takeshi Takahashi" <[email-masked]>
Date: Thu, 23 Jul 2015 22:34:03 +0200

> I think mappings between IODEF XML to JSON would also be beneficial.

+1

Regards,
Take

> From: mile [mailto:[email-masked]] On Behalf Of Panos Kampanakis
> (pkampana)
> Sent: Thursday, July 23, 2015 8:02 PM
> To: Tomáš Čejka <[email-masked]>; [email-masked]
> Subject: Re: [mile] JSON data format
> 
> Actually that has been a question for many XML based formats. How XML is
not
> very flexible and how other languages like JSON are better. There has been
work
> in the STIX community (that uses XML also) to create JSON conversions. I
think
> mappings between IODEF XML to JSON would also be beneficial.
> 
> Panos
> 
> 
> 
the
> whole mailing list archive. That is why I'm sorry in advance in case my
question
> was discussed anywhere in the past...
> 
> As far as I understand the documents, there is a data model that describes
> (IODEF) information about security incidents that would be useful to
share.
> As a proposed data format, the documents use XML.
> 
that
> would standardize JSON as a compatible data format to IODEF?
> 
published
> on github.com soon) developed by CESNET [2] (Czech NREN operator).
format
> [3] (according to JSON schema [4]). Alerts can be currently sent into e.g.
> Warden system [5].
>

---

From: Patrick Curry <[email-masked]>
Date: Thu, 23 Jul 2015 20:16:01 +0100

Hi Panos,

I’d be interested to understand more.  I have colleagues working on a proprietary big data analytics database designed for very high scalability and we were discussing how machine speed threat and other security event & sensor data might perform using JSON vs XML.   IODEF came to mind as being relevant, particularly if/as it expands for more functional requirements and generates more data. Any advice on large scale performance would be helpful.

Patrick

Patrick Curry
Director

British Business Federation Authority - BBFA Ltd
M: +44 786 024 9074
T:   +44 1980 620606
[email-masked]
www.bbfa.info – a not-for-profit, self-regulating body 

On 23 Jul 2015, at 19:02, Panos Kampanakis (pkampana) <[email-masked]> wrote:

Panos

-----Original Message-----
From: mile [mailto:[email-masked]] On Behalf Of Tomáš Cejka
Sent: Wednesday, July 22, 2015 9:25 AM
To: [email-masked]
Subject: [mile] JSON data format

Hi all,

---

From: "Panos Kampanakis (pkampana)" <[email-masked]>
Date: Thu, 23 Jul 2015 21:28:45 +0000

Hi Patrick,
Intuitively JSON will be better from a performance standpoint,  communication and potentially storage overhead. I am not aware of any performance or tests or PoCs that compare XML based representations vs JSON. Specifically for STIX there is https://github.com/freestix that introduced and Jordan mentioned at some point that there is an important speedup with 3 million Cybox indicators being serialized, transmitted, and deserialized using JSON based APIs on a Macbook in 1 minute 14 seconds. The above intuitively will hold true in JSON based IODEF  and other XML based schemas.
Panos

From: Patrick Curry [mailto:[email-masked]]
Sent: Thursday, July 23, 2015 3:16 PM
To: Panos Kampanakis (pkampana)
Cc: Tomáš Čejka; [email-masked]
Subject: Re: [mile] JSON data format

Hi Panos,

Patrick

Patrick Curry
Director

British Business Federation Authority - BBFA Ltd
M: +44 786 024 9074
T:   +44 1980 620606
[email-masked]<mailto:[email-masked]>
www.bbfa.info<http://www.bbfa.info> – a not-for-profit, self-regulating body

On 23 Jul 2015, at 19:02, Panos Kampanakis (pkampana) <[email-masked]<mailto:[email-masked]>> wrote:

Panos

-----Original Message-----
From: mile [mailto:[email-masked]] On Behalf Of Tomáš Cejka
Sent: Wednesday, July 22, 2015 9:25 AM
To: [email-masked]<mailto:[email-masked]>
Subject: [mile] JSON data format

Hi all,

My question is motivated by our running code that uses JSON.
I'm a developer of the Nemea project [1] (current version will be published on github.com<http://github.com> soon) developed by CESNET [2] (Czech NREN operator).
Nemea contains detection modules that send discovered alerts in IDEA format [3] (according to JSON schema [4]). Alerts can be currently sent into e.g. Warden system [5].

_______________________________________________
mile mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mile

_______________________________________________
mile mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/mile

---

From: "Teague, Nik" <[email-masked]>
Date: Thu, 23 Jul 2015 21:41:08 +0000

On 23/07/2015 20:02, "mile on behalf of Panos Kampanakis (pkampana)"
<[email-masked] on behalf of [email-masked]> wrote:

>not very flexible and how other languages like JSON are better. There has
>been work in the STIX community (that uses XML also) to create JSON
>conversions. I think mappings between IODEF XML to JSON would also be
>beneficial.

I wouldn’t like to say that anything is better or less flexible… I think
its an eye of the beholder sort of thing.

I personally find XML difficult because I find JSON more efficient and
readable but there are many who disagree.  For me to do something in JSON
takes a fraction of the effort that XML does but others have their
opinions and preferences that may disagree and just as we have many
differing views on coding/scripting, we have just as many data formats.
For me, and again its personal preference, I find REST and JSON just
easier to deal with…  My preferences/prejudices are based upon my
experience as a network engineer using Python etc. where JSON is very easy
to consume and parse.

I think Kathleen summed it up in the recent meeting and I’m paraphrasing
where it comes down to ‘what is most likely to get used?’.  So, to that
end, I’d be more inclined to use, and there would be fewer initial
barriers (for me) to using JSON representations over XML.

Thanks,

-Nik

---

From: Kathleen Moriarty <[email-masked]>
Date: Fri, 24 Jul 2015 03:46:19 -0400

Good discussion, thanks for the offer Tomas!

> barriers (for me) to using JSON representations over XML.

With that in mind, would the JSON version of IODEF be useful with
translations available?  I'm thinking the JSON version might wind up as the
primary if done quickly along with the current version of IODEF.
Translations may be more from JSON to legacy XML.  I'm not sure there are
too many IODEFv2 implementations yet, so if it is the best chance for
increased deployment, does a change make sense?

We already have a few places where IODEFv1 and IODEFv2 are not compatible.
Or is this community largely XML users (which may be the case)?  If it is
all automated, incident responders should never see one or the other.

Thanks,
Kathleen

-- 

Best regards,
Kathleen