Subject: Nevil Brownlee: Re: [IMRG] Flow sampling (or selection)

From: Mark Allman <[email-masked]>
Date: Thu, 03 Jul 2003 15:28:54 -0400

[This got caught in the filters.  I approved it last night, but
 don't see it in the IMRG mail archive and so fear that it did not
 make it to the list.  I'm off to look into it... if this is, in
 fact, a repeat, I appologize.]

------- Forwarded Message

Date: Wed, 2 Jul 2003 15:10:50 -0700 (PDT)
From: Nevil Brownlee <[email-masked]>
To: Maurizio Molina <[email-masked]>
Cc: [email-masked], <[email-masked]>
Subject: Re: [IMRG] Flow sampling (or selection)

Hi Maurizio et al:

I haven't seen much discussion of flow sampling, in the sense of
'deciding which flows should be exported' from a flow measuring
device.

There's been some recent work on determining which flows are
the most significant/interesting, e.g.
Cristian Estan's   www.icir.org/vern/imw2001-papers/71.pdf
and my "Dragonflies and Tortoises," IEEE Communications, October 2002

In my work with NeTraMet I've implemented a coarse filter for flows,
you can tell NeTraMet's meter reader to ignore flows which have seen
less than n packets, so you can ignore things like port scans and
concentrate of high-volume or long-running flows.

Mark's description of the IPFIX and PSAMP WGs (below) is about right;
IPFIX has recently published a set of four new drafts, you can find
them via the IETF WG pages.  But IPFIX hasn't considered the question
of flow sampling, and I guess it's out of scope for PSAMP (which looks
at packet sampling).

Cheers, Nevil

- -------------------------------------------------------------
   Nevil Brownlee                     Internet Researcher
   Phone: (858) 534 8338                 CAIDA, San Diego

On Tue, 1 Jul 2003, Maurizio Molina wrote:

>
>
> Mark Allman wrote:
>
> >>As far as I know IPFIX deals with reporting per flow information,
> >>PSAMP with sampling and reporting per packet information, selected
> >>with some criteria.  Has there in any of these groups (or
> >>elsewhere) been any discussion about FLOW sampling? I've seen
> >>papers about it, e.g.
> >>
> >>http://www.research.att.com/~duffield/pubs/DLT02-optimal.pdf
> >>
> >>and I wondered whether this IMRG is the right place to discuss
> >>about it.
> >>
> >>
> >
> >Indeed this is the right place to discuss such things.
> >
> >allman
> >
> >
> >--
> >Mark Allman -- BBN/NASA GRC -- http://roland.grc.nasa.gov/~mallman/
> >
> >
> Actually, what I would try to discuss is if there's the need, in the
> standards, to agree on common procedures for coupling packet sampling
> and flow selection.
> To better clarify, I copy hereafter a section that has been introduced
> in the last revision of the PSAMP draft
> http://www.ietf.org/internet-drafts/draft-ietf-psamp-sample-tech-02.txt
> that I'm co-authoring.
> I'm trying to get some reaction about this in the IETF but also some
> discussion in this IMRG would help.
> P.S.: is there any planned meeting in Vienna?
> Regards,
> Maurizio Molina
>
> ****************
> 3.1.2.2.3    Non-Uniform flow State dependent sampling
>
> Another type of sampling that can be classified as Non-Uniform (and,
> possibly, probabilistic) is closely related to the flow concept as
> defined in [QuZC02], and it is only used jointly with a flow monitoring
> function (IPFIX monitoring function). Packets are selected,
> probabilistically or deterministically, dependent on a selection state.
> The point, here, is that the selection state is determined also by the
> state of the flow the packet belongs to and/or by the state of the other
> flows currently being monitored by the associated flow monitoring
> function. This type of sampling is also content dependent because the
> identification of the flow the packet belongs to requires analyzing part
> of the packet content. If the packet is selected, then it is passed as
> an input to the IPFIX monitoring function (this is called "Local Export"
> in [DuGG03].
> Selecting the packet depending on the state of a flow cache is useful
> when memory resources of the flow monitoring function are scarce (i.e.
> there's no room to keep all the flows that have been scheduled for
> monitoring). The algorithms leading to the selection decision are not
> subject to standardization, but they may take into account factors like
> flow volume, flow grow rate, flow monitoring priority. An example of
> such an algorithm is described in [EsVa01].
>
>
>

------- End of Forwarded Message

_______________________________________________
IMRG mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/imrg