Subject: [IMRG] Available Netflow logs or other similiar flow based logs?

From: Hao Shang <[email-masked]>
Date: Wed, 1 Oct 2003 12:55:25 -0400 (EDT)

Hi there,

Could anyone point out some places which hold public accessable netflow,
argus or other flow based logs? The IP addresses in the log don't have to
be real for privacy reasons, but need fine granularity to distinguish each
individual user (i.e. IP addresses can be hashed to protect privacy, but
masking part of an IP address with zeros will not be proper).

Thanks for your help!

-------
Hao

_______________________________________________
IMRG mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/imrg

---

From: stanislav shalunov <[email-masked]>
Date: 01 Oct 2003 15:09:14 -0400

Hao Shang <[email-masked]> writes:

> Could anyone point out some places which hold public accessable
> netflow, argus or other flow based logs?  The IP addresses in the
> log don't have to be real for privacy reasons, but need fine
> granularity to distinguish each individual user (i.e. IP addresses
> can be hashed to protect privacy, but masking part of an IP address
> with zeros will not be proper).

Note to anyone who provides traces with hashed addresses or parts of
addresses or contemplates providing them: Unless the traces are for a
private network that does not exchange traffic with the Internet, the
privacy of such traces is not significantly enhanced, under realistic
threat models, over simply providing the addresses in the clear.

If an adversary has access to traces where, say, n last bits of all
IPv4 addresses are subject to a permutation f from S_{2^n}, the
adversary can reconstruct f or many interesting values of f by either
of the following two attacks:

1. Knowing that he might at a later time gain access to a trace thus
   obfuscated, the adversary could inject traffic that would allow to
   reconstruct f(x) for any specific value of x, by sending packets
   where x appears both in fields that would be obfuscated and where x
   would not be obfuscated and that are easy to find in the resultant
   trace.  Many possibilities exist: one could, for example, put x in
   the source or destination field and then split it between port
   numbers while using a specific TOS value to help identify these
   test packets later.  Or, if n<=16, one could store x twice in the
   source or destination fields.  If it is necessary that the activity
   looks innocuous, the adversary can make his behavior appear more
   natural by running, e.g., a web crawler and interspersing test data
   in normal TCP connections (by using port numbers and TOS values).

2. If the adversary failed to do (1) in the past and only obtained a
   single historic obfuscated trace from a large network, he could
   attempt to find another, smaller, network that exchanges traffic
   with the larger network and obtain unobfuscated traces from the
   smaller network.  (The adversary might do so by obtaining traces
   from his own institution or lab, legally compelling an institution
   that is known to keep such traces to hand them over to him, or by
   stealing the data: he would have many targets any of which would be
   sufficient and the job would, therefore, be made easier.)  Then,
   correlating time and unobfuscated fields from the two tests, the
   adversary could extract records about the same packets/flows and
   learn f(x) for some values of x not of his choosing.  If n is
   sufficiently small (say, <=16) the adversary would only need a few
   days' worth of data from a lab somewhere that exchanges traffic
   with the target network to know most of the function f.

The first attack allows an adversary to easily learn the clear-text of
all addresses for small n, or if n is large enough (it can only be
large for IPv6), for a set of specific addresses that are of interest
for the adversary.  For IPv6, an adversary could attempt to enumerate
interesting addresses by dictionary attacks on DNS, web crawling, or
crawling specific peer-to-peer networks.

The second attack is more stealthy, but may require the adversary to
process more data.

Both attacks are practical even for attackers with limited resources;
e.g., curious individuals.

-- 
Stanislav Shalunov		http://www.internet2.edu/~shalunov/

"I didn't attend the funeral, but I sent a nice letter saying that I
approved of it."				-- Mark Twain