Subject: Standard Flows: Next Move

From: Nevil Brownlee <[email-masked]>
Date: Sun, 15 Apr 2001 01:26:29 -0700

Hello All:

Here's my attempt to summarise the Mailing List Discussion so far:

Definition of 'flows'

  Collection of TCP or UDP microflows that 
  - are forwarded along the same common path
  - share the same class of service

or
  Unidirectional series of IP packets of a given protocol,
  with the same IP Address / Port endpoints, i.e. a 5-tuple.

'Common path' is a vague concept (except for a MPLS LSP).
'Class of service' is also vague (but we could use DSCodepoint).

Include interface numbers?  Would help with multicast.

Better not to, should stick with data which can be extracted 
directly from packets.  Interfaces can't (an input interface
may not know what the output interface will be), nor can
AS numbers.

Better to have a minimum 'flow key.'  Could allow for more
data (interface numbers, ASNs, LSPs, etc.) as options.

Could be useful to keep 'flow definition' separate from
'flow identification,' e.g. use 5-tuple as definition,
other attributes (IPv6 flow label, FlowID, ifIndex, ...)

We need to start working on 'Requirements' ...

        -------------------------------------

I think it's time we started on a draft charter, and of course
we need one or two Internet Drafts to support a BOF request.

Here's my attempt at a draft WG charter:

IP Flow Export (IFX)

At present there are three public-domain methods in use for 
collecting flow data:
 - Netflow (on Cisco routers)
 - LFAP (on Riverstone routers)
 - RTFM (using NeTraMet on Unix boxes)
Flow data is most commonly required for usage measurement, i.e. it
is seen as a major component of Accounting, the third A in AAA.

Several router vendors have expressed strong support for a WG
to develop a standard method of exporting flow data from routers.
A preliminary meeting was held at the Minneapolis IETF meeting
in March, and vigorous discussion has begun on the RTFM mailing
list.

We believe that the proposed Working Group would fall most naturally
within the Operations and Management Area.  Its goals and requirements
are straightforward; the Group should be able to complete its work
in about one year.

Goals:

 1. Produce a 'requirements' document.  This will identify the
    target community for SFX, expected uses of flow data,
    and the constraints (reliability, timeliness, level of
    detail, etc.) these uses imply.

 2. Define 'IP accounting flow,' and a data model for it.

 3. Specify mechanism(s) for exporting accounting flows.
    These must allow for push and pull models, and be capable
    of handling high volumes of flow data.

 4. Develop methods of configuring and controlling flow export
    in a network device.  For example,
       a. How much detail is required (level of aggregation)?
       b. Will data be read by the accounting system (pull model)?
       c. Or should data be sent to the accounting system at
          specified intervals (push model)?

Milestones:

 1. IP Flow Export Requirements (Informational RFC)

 2. IP Flow Definition and Export Control Mechanisms (Standards Track RFC)

 3. Flow Export: Transport Mechanisms (Informational RFC)

Is this on the right lines?  What have I left out?  

I think we need an initial version of the Requirements draft, as soon
as we can manage to produce it.  I'll be away from email for much of
the next three weeks, but after that I'll be able to put some effort
into editing such a draft.  So, comments/suggestions please ...

Cheers, Nevil

+---------------------------------------------------------------------+
| Nevil Brownlee                     Director, Technology Development |
| Phone: +64 9 373 7599 x8941        ITSS, The University of Auckland |
|   FAX: +64 9 373 7021      Private Bag 92019, Auckland, New Zealand |
+---------------------------------------------------------------------N

---

From: "Gregory R. Ruth" <[email-masked]>
Date: Mon, 16 Apr 2001 14:14:08 -0400

Nevil,
     Your proposed charter sounds fine to me, and the sooner we get
agreement on that and send it to the IESG, the better.  My only reservation
is that one year is only 3 IETFs (4 if you cheat a little) and that seems
such a short time to get a requirements RFC AND a standard RFC done, as the
later may have to wait for good progress on the former.  Nevertheless, our
goals are modest and narrowly defined, and it seems about the right target.
     Do we need to have a BOF before we can get a WG chartered?  Seems like
it might just slow down the process.

Greg

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On
Behalf Of Nevil Brownlee
Sent: Sunday, April 15, 2001 4:26 AM
To: [email-masked]
Subject: Standard Flows: Next Move

Hello All:

Definition of 'flows'

or

IP Flow Export (IFX)

Goals:

Milestones:

Cheers, Nevil

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 17 Apr 2001 10:41:36 +0200

Hello Neville,

thanks for the draft charter. I like it very much.
Some comments are inline.

    Juergen
-- 
Juergen Quittek     [email-masked]     Tel: +49 6221 90511-15
NEC Europe Ltd., C&C Research Laboratories   Fax: +49 6221 90511-55
Adenauerplatz 6, 69115 Heidelberg, Germany   http://www.ccrle.nec.de

> We need to start working on 'Requirements' ...

SFX? You probably mean IFX.

>  3. Specify mechanism(s) for exporting accounting flows.

We are not going to export the flow, but flow accounting information
Furthermore, some of my colleagues misinterpreted the term "export"
because they associated it with exporting it out of an administrative
domain.

Suggestion:

  3. Specify mechanism(s) for exporting flow accounting information
     out of middleboxes.

>     of handling high volumes of flow data.

Do we really need the pull model? It seems to be a nice option next
to the push model, but I'm not sure. Let's discuss this again when
we  have the requirements completed.

>           specified intervals (push model)?

I see b. and c. to be already answered by the requirements.

>  2. IP Flow Definition and Export Control Mechanisms (Standards Track RFC)

I suggest to split these two documents. An IP flow definition would be a 
more general RFC which might be referenced by many other working groups. 
The export control mechanisms are specific to this future WG. Both documents 
should be able to progress independently.

> into editing such a draft.  So, comments/suggestions please ...

... particularly, we need a discussion on requirements!

---

From: Juergen Quittek <[email-masked]>
Date: Tue, 17 Apr 2001 11:05:05 +0200

Hello Greg,

> later may have to wait for good progress on the former.  Nevertheless, our

I share your concerns. However, the way I see it, the IESG wants to have
in general rather short termed and focussed charters, because the number 
of WGs is already terribly high. Consequently, we should request a WG not 
before we have a clear idea what we want to do, and not before we have some 
(potentially competing) concrete suggestions about the output we are planning 
to produce. This means, we should have a requirements document which is not 
too far from maturity and we should have some initial drafts for the other 
documents.

> it might just slow down the process.

That's probably right, but we should make the decision (whether ot ask for
a WG or a BOF) dependent on our progress during the next months. If we can
go on quickly, I agree with you, that we should directly ask for a new WG
(probably referencing the San Diego BOF). But if we go on slowly, I'd prefer
to get some additional input from another more focussed BOF.

    Juergen

---

From: Dave Plonka <[email-masked]>
Date: Mon, 23 Apr 2001 11:28:22 -0500

>   Collection of TCP or UDP microflows that 

What prior work defines "microflows"?  (For the moment I'm presuming
for TCP that that would be a fragment of a stream.)

> We need to start working on 'Requirements' ...

I would prefer "IPFX" to either "IFX" or "SFX".  We should settle on
this ASAP since it is a prerequisite for the BoF prerequisites ;^).

> to develop a standard method of exporting flow data from routers.

Is it relevent to mention that other vendors such as Juniper and
Extreme have adopted the Cisco-defined NetFlow export PDU?

Presumably this is to acheive compatibility with flow collectors, but
since NetFlow is not an Official Standard Protocol, various
implementations do not necessarily use the same flow definition and
therefore do not ensure interoperability of collected accounting data.

I see this as a prime motivating factor for doing this WG _now_.

> Is this on the right lines?  What have I left out?  

Overall, I like your outline.
 
Dave

-- 
[email-masked]  http://net.doit.wisc.edu/~plonka  ARS:N9HZF  Madison, WI

---

From: "Carter Bullard" <[email-masked]>
Date: Mon, 23 Apr 2001 18:25:22 -0400

Hey Guys,
> for TCP that that would be a fragment of a stream.)

My experience has been that people who talk about microflows
are generally backbone oriented.  For these "big pipe" minded folk,
aggregated flows such as AS-to-AS, or Net-to-Net flows,
are "flows", and the traditional 5-tuple flow is a microflow.

I think the draft is using this perspective.

I'm one to think that if you have microflows, you should have
flows and macroflows.  I put the flow at the Type-P level, which
includes the 5-tuple flow.  In this view, AS-to-AS, or Net-to-Net,
flows would be macroflows, and sub-transactions within a Type-P
flow would be a microflow.  This tracks the concepts in
RMON pretty well, such that RMON link stats are macroflow stats,
RMON2 gives you flow stats, and newer technology, such as APM,
give you microflow measures.

Carter

Carter Bullard
QoSient, LLC
300 E. 56th Street, Suite 18K
New York, New York  10022

[email-masked]
Phone +1 212 588-9133
Fax   +1 212 588-9134
http://qosient.com

---

From: Mike MacFaden <[email-masked]>
Date: Tue, 24 Apr 2001 06:19:44 -0700

>this ASAP since it is a prerequisite for the BoF prerequisites ;^).

Since IETF is about IP, I agree that IP should be the key thing we 
want to express. And from what I can see for existing practice, 
what folks want to export anyway.

>Extreme have adopted the Cisco-defined NetFlow export PDU?

This section will be dropped from the charter per AD request.
Suffice it to say that what we are going to standardize is
based on de facto standards not unlike the efforts of
the Secure Shell working group:
http://www.ietf.org/html.charters/secsh-charter.html

It will be very important not to step on existing proprietary
technology and I urge anyone to speak up and issue any assertions
in a timely fahsion should we cross over into ones Intellectual Property.

>I see this as a prime motivating factor for doing this WG _now_.

Exactly. 

It is my opinion that we are standardizing known technology
and not inventing something new hence any working group created
should be able to complete the tasks set forth in less than a year.

Regards,
Mike MacFaden

---

From: [email-masked]
Date: Tue, 24 Apr 2001 16:09:11 +0200 (CEST)

Hi Mike,

Mike MacFaden <[email-masked]> wrote:
 [...]
> This section will be dropped from the charter per AD request.

Are there more requests for changes from the ADs?
It would be nice to see them on this list.

    Juergen

---

From: Kamlesh Shah <[email-masked]>
Date: Tue, 24 Apr 2001 10:11:09 -0400

Folks,

I am just joining the discussions. So pardon my ignorence..

What is AD ?

Also, can someone email me the location of current proposal of this group ?

PS: I have worked extensively on Cisco's NetFlow.

Regards.

>It would be nice to see them on this list.

((((((((((((())))))))))))
Kamlesh Shah  CCIE # 2803
Technical Marketing Engineer - MPLS, ISBU

---

From: Mike MacFaden <[email-masked]>
Date: Tue, 24 Apr 2001 16:59:02 -0700

>It would be nice to see them on this list.

Indeed. This is my understanding of what the
ADs want to see happen:

1) Must involve experienced users in this effort
2) Must discuss the data format, protocol
   protocol by which devices export flow data,
   and how flow data consumers 
   control what they wish to have measured and to receive
3) Must clearly describe the problem being solved
4) Must create a new single purpose mailing list 
   and start working on the charter

Status:

Dave Plonka is currently setting up a new list 
and Nevil has asked me to publish a revised 
charter to that list for continued
discussion and revision. If folks want to continue
discussing on this list for now, that is fine too.

Regards,
Mike

---

From: Joe Abley <[email-masked]>
Date: Wed, 25 Apr 2001 15:45:21 -0400

> 1) Must involve experienced users in this effort

Does this mean vendors who have experience in implementing support
for flow export, or operators who have experience in using it?

Joe

---

From: Kamlesh Shah <[email-masked]>
Date: Wed, 25 Apr 2001 19:05:38 -0700

Wouldn't it make sense to have some representation from both camps ?

The only catch is to hope that the right folks are talking (or we are talking to the right folks)..

---

From: Mike MacFaden <[email-masked]>
Date: Thu, 26 Apr 2001 06:36:01 -0700

>for flow export, or operators who have experience in using it?

The AD requested involving operators.

If you have not subscribed to ipfx mailing list, please
let's move all discussions there now, the list is archived.

http://net.doit.wisc.edu/ipfx/

Regards,
Mike