Subject: [IPFIX] Flow in IPFIX -> not only IP Flows

From: Benoit Claise <[email-masked]>
Date: Fri, 03 Sep 2010 12:20:24 +0200

Dear all,

Here is one problem that was anticipated...

For whatever reasons at that point in time, RFC 5101 limits the Flow 
definition to IP packets

    IP Traffic Flow or Flow

       There are several definitions of the term 'flow' being used by the
       Internet community.  Within the context of IPFIX we use the
       following definition:

       A Flow is defined as a set ofIP packets  passing an Observation
       Point in the network during a certain time interval.  All packets
       belonging to a particular Flow have a set of common properties.
       Each property is defined as the result of applying a function to
       the values of:

          1. one or more packet header fields (e.g., destination IP
             address), transport header fields (e.g., destination port
             number), or application header fields (e.g., RTP header
             fields [RFC3550  <http://tools.ietf.org/html/rfc3550>]).

          2. one or more characteristics of the packet itself (e.g.,
             number of MPLS labels, etc...).

          3. one or more of fields derived from packet treatment (e.g.,
             next hop IP address, the output interface, etc...).

       A packet is defined as belonging to a Flow if it completely
       satisfies all the defined properties of the Flow.

       This definition covers the range from a Flow containing all
       packets observed at a network interface to a Flow consisting of
       just a single packet between two applications.  It includes
       packets selected by a sampling mechanism.

However, we know that IPFIX cover more than IP. 
http://www.iana.org/assignments/ipfix/ipfix.xhtml mentions 
sourceMacAddress, ethernetPayloadLength, etc...
And we know that IPFIX became a kind of generic streaming protocol.

Talking in the context of the NGN networks at the ITU, there are some 
reluctance to adopt this Flow definition limited to IP packets.
Isn't it time to do something about this definition?

Regards, Benoit.

---

From: Haruhiko Nishida <[email-masked]>
Date: Fri, 3 Sep 2010 21:16:15 +0900

Benoit,

I support your point. IPFIX should be a next generation of both 
netflow and sflow.

-- Haru Nishida, NTT

On 2010/09/03, at 19:20, Benoit Claise wrote:

> For whatever reasons at that point in time, RFC 5101 limits the Flow definition to IP packets
> 
>    IP Traffic Flow or Flow
> 
>       A Flow is defined as a set of IP packets passing an Observation
>             fields [RFC3550]).
> However, we know that IPFIX cover more than IP. http://www.iana.org/assignments/ipfix/ipfix.xhtml mentions sourceMacAddress, ethernetPayloadLength, etc...
> Talking in the context of the NGN networks at the ITU, there are some reluctance to adopt this Flow definition limited to IP packets.
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix

---

From: sujay gupta <[email-masked]>
Date: Fri, 3 Sep 2010 21:27:04 +0530

Hi Benoit,

I believe it is a stance which should be accepted by all, as the name
(IPFIX) and most usage
IMO is layer 3.

Could someone point out as to why the protocol was called as 'IP'  FIX??

sFlow & other groups clearly focus on L2+L3.

BR,
-Sujay

On Fri, Sep 3, 2010 at 5:46 PM, Haruhiko Nishida <[email-masked]> wrote:
> http://www.iana.org/assignments/ipfix/ipfix.xhtml mentions sourceMacAddress,
> ethernetPayloadLength, etc...

---

From: Kobayashi Atsushi <[email-masked]>
Date: Sun, 05 Sep 2010 18:36:23 +0900

Hi Benoit and all,
 
Benoit, I also support your idea.  IPFIX can expand the functionality to
application layer. We should not limit IPFIX to L3+L2. 

The following term "Flow" is just my idea.

A Flow is defined as a set of  frames, packets, or sessions residing on
the data link layer, IP layer, or application layer passing an
Observation Point in the network during a certain time interval.

Regards,
Atsushi

On Fri, 3 Sep 2010 21:27:04 +0530

> > Isn't it time to do something about this definition?

--- 
Atsushi KOBAYASHI  <[email-masked]>

---

From: Gerhard Muenz <[email-masked]>
Date: Sun, 05 Sep 2010 21:35:02 +0200

Benoit,

I agree: "IP packet" can be replace by "packet".

However, replacing "packet" by something like "packet, frame, or cell" 
would probably cause inconsistencies in this and other IPFIX/PSAMP 
documents. So, I would not go that far.

Regards,
Gerhard

---

From: Brian Trammell <[email-masked]>
Date: Mon, 6 Sep 2010 11:19:40 +0200

Benoit, Gerhard, all,

I agree with Gerhard here; call it a packet, leave the definition of packet ambiguous, so that it applies to any commonly accepted definition thereof.

However, we have two separate problems here. The _other_ problem is how to handle the expansion of IPFIX as a generic flexible data representation into areas which are not directly related to flow measurement as originally envisioned. The prototype here is SIPCLF.

As I see it, this problem should be solved separately from the problem of redefining flows for non-IP monitoring, and there are three separate general solution areas:

1. Do nothing, keep the (slightly expanded) definition of Flow. Here, we limit the applicibility of IPFIX to network management area, and exploit the fact that basically everything in the network management area is at the base of it going to involve packets going from somewhere to somewhere. In this case, every record represented using IPFIX is going to _somehow_ have a relationship back to _some_ set of packets which can be wedged into the present definition (the "packet treatment" clause in the 5101 definition affords us significant flexibility here). This works for SIPCLF as presently defined. Non-flow information can still be exported using Options, so it works for expanding IPFIX to decidedly non-flow things (e.g., physical monitoring scenarios, think periodic temperature reporting) as well, as long as all of these records can be represented as scoped to something (which I believe they can). In this case, we would probably want to mention this in the soon-forthcoming IE-Doctors draft.

2. Define a new category of data records which IPFIX can be used to export. These non-flow records would be represented as flow records (i.e., not Options), and would be equivalent to Flow records, except that the flow-specific provisions (uniqueness of flow keys, reversibility of non-key fields, etc., etc.) would not apply. The way to do this would be, I think, with another RFC which would present the expanded definition, and devices exporting/handling this non-flow data would therefore state compliance with this _new_ RFC. I'm not a big fan of this, personally, because it seems a little complicated, but it does allow us to segregate IPFIX-for-flow and IPFIX-for-not-really-flow better than (1) above.

3. Drastically expand the definition of Flow so that it covers everything you might want to represent with IPFIX. This strikes me as though it would take a great deal of effort, and break a lot of assumptions, basically as noted by Gerhard below. I mention it here only for completeness.

My preference would be (1) (do nothing, note applicability). Thoughts?

Best regards,

Brian

On Sep 5, 2010, at 9:35 PM, Gerhard Muenz wrote:

> However, replacing "packet" by something like "packet, frame, or cell" would probably cause inconsistencies in this and other IPFIX/PSAMP documents. So, I would not go that far.
> 
> Regards,
> Gerhard
> 
>

---

From: Shingo KASHIMA <[email-masked]>
Date: Wed, 08 Sep 2010 03:45:47 +0900

Hi Benoit,

I also support your idea.

IPFIX should support the other protocols than IP with both packetSection-style
and flow-based style. And IPFIX has a potential of exporting various
information not limited to packet and flow information.

IPFIX may be "Improvement of Packet and Flow Information eXport".

Regards.

-- 
Shingo KASHIMA <[email-masked]>
NTT Information Sharing Platform Lab.

---

From: Benoit Claise <[email-masked]>
Date: Tue, 07 Sep 2010 23:02:04 +0200

> documents. So, I would not go that far.
Agreed.

Regards, Benoit.

---

From: Benoit Claise <[email-masked]>
Date: Tue, 07 Sep 2010 22:56:38 +0200

Kashima-san,
> IPFIX may be "Improvement of Packet and Flow Information eXport".
Interestingly, some years, just before the RFC5101, I proposed to rename 
IPFIX to IP Flexible Information Export but I was a little bit too early 
I guess ;-)

Regards, Benoit.
> sujay gupta<[email-masked]>  wrote:
>
>> Hi Benoit,
>>
>> sFlow&  other groups clearly focus on L2+L3.
>>
>> BR,
>> -Sujay
>>
>>
>> On Fri, Sep 3, 2010 at 5:46 PM, Haruhiko Nishida<[email-masked]>  wrote:
>>> Benoit,

---

From: Benoit Claise <[email-masked]>
Date: Tue, 07 Sep 2010 23:13:35 +0200

Brian,
> My preference would be (1) (do nothing, note applicability). Thoughts?
Right, as briefly discussed in the past, we would need an applicability  
version 2

Regards, Benoit.
>>>              fields [RFC3550<http://tools.ietf.org/html/rfc3550>]).
>>>