Subject: [IPFIX] WG last call on draft-ietf-ipfix-mediators-problem-statement-06

From: "Juergen Quittek" <[email-masked]>
Date: Mon, 19 Oct 2009 07:02:06 +0200

Dear all, 

At our session in Stockholm we agreed to start the 2nd working group
last call on the mediation problem state as soon as a new version
of it would be available. The new version is

http://www.ietf.org/internet-drafts/draft-ietf-ipfix-mediators-problem-state
ment-06.txt

The call starts today and will last until Friday October 30.

Please read the draft!
Please send your comments to this mailing list!
Please also send a message if you are fine with the draft as it is!

Thank you, 

    Juergen

---

From: Gerhard Muenz <[email-masked]>
Date: Fri, 30 Oct 2009 10:54:27 +0100

Hi,

Here is my review of draft-ietf-ipfix-mediators-problem-statement-06.

The main content is ok. Some parts of the text still need revision.

I'm not sure if we need to define Mediator types like IPFIX Proxy, IPFIX 
Concentrator, IPFIX Distributor, IPFIX Masquerading Proxy in the 
terminology section. Why not just talk about IPFIX Mediators?

Regards,
Gerhard

> Abstract
> 
>    Flow-based measurement is a popular method for various network
>    monitoring usages.  The sharing of flow-based information for
>    monitoring applications having different requirements raises some
>    open issues in terms of measurement system scalability, flow-based
>    measurement flexibility, and export reliability that IPFIX Mediation
>    may help resolve.  This document describes the IPFIX Mediation
>    applicability examples, along with some problems that network
>    administrators have been facing.

Shouldn't it be the other way round:
- first problems
- then applicability examples
?

> 1.  Introduction
> 
>    One advantage of Flow-based measurement results from easily offering
>    the traffic monitoring of a huge amount of traffic.  While the usage
>    is applied to any networks and to multiple measurement applications,
>    network administrators need to optimize the resource of metering
>    devices and of multiple measurement applications.  IP traffic growth
>    and a wide variety of measurement application make the optimization
>    further difficult.  To achieve system optimization, an intermediate
>    device can generally be applied to the system platform.

Sorry, I can only guess what this paragraph is supposed to say.
Please rewrite in a better understandable and readable way.

BTW, I would be careful with terms like "optimize" and "optimization".

>    The IPFIX requirements defined in [RFC3917] mention examples of
>    intermediate devices, such as IPFIX Proxies or Concentrators, there

missing conjunction, such as "but", "yet", or something like that

The term "intermediate" or "intermediate device" does not appear in 
RFC3917. So, explain why you call these devices like this.
(For example, because they are located between Exporters and Collectors.)

>    are no documents defining a generalized concept for such intermediate
>    devices.  This document addresses that issue by defining IPFIX
>    Mediation, a generalized intermediate device concept for IPFIX, and
>    examining in detail the motivations behind its application.
> 
>    This document is structured as follows: section 2 describes the
>    terminology used in this document, section 3 gives an IPFIX/PSAMP
>    document overview, section 4 introduces general problems related to
>    flow-based measurement, section 5 describes some applicability
>    examples where IPFIX Mediations would be beneficial, and, finally,
>    section 6 describes some problems an IPFIX Mediation implementation
>    might face.

> 2.  Terminology and Definitions
> 
>    The IPFIX-specific and PSAMP-specific terminology used in this
>    document is defined in [RFC5101] and [RFC5476], respectively.  In
>    this document, as in [RFC5101] and [RFC5476], the first letter of
>    each IPFIX-specific and PSAMP-specific term is capitalized along with
>    the IPFIX Mediation-specific term defined here.
> 
>    In this document, we use the generic term "record stream" to denote a

I would not call "record stream" a "term" unless it appears in the list 
below with capitalized first letter.

>    set of flow- or packet-based data records with their additional

I would not say that the records are flow-based or packet-based. They 
contain flow or packet information.

>    information that flows from data sources, whether encoded in IPFIX
>    protocol as IPFIX Data Records, or non-IPFIX protocols.  In IPFIX
>    protocol, we use the generic term Data Records for IPFIX Flow
>    Records, PSAMP Packet Reports, and Data Records defined by Options
>    Templates, unless an explicit distinction is required.

Do we need the last sentence?

>    Original Exporter
> 
>       An Original Exporter is an IPFIX Device that hosts the Observation
>       Points where the metered IP packets are observed.
> 
>    IPFIX Mediation
> 
>       IPFIX Mediation is the manipulation and conversion of a record
>       stream for subsequent export using the IPFIX protocol.
> 
>    The following terms are used in this document to describe the
>    architectural entities used by IPFIX Mediation.
> 
>    Intermediate Process
> 
>       An Intermediate Process takes a record stream as its input from
>       Collecting Processes, Metering Processes, IPFIX File Readers,
>       other Intermediate Processes, or other record sources; performs
>       some transformations on this stream, based upon the content of
>       each record, states maintained across multiple records, or other
>       data sources; and passes the transformed record stream as its
>       output to Exporting Processes, IPFIX File Writers, or other
>       Intermediate Processes, in order to perform IPFIX Mediation.
>       Typically, an Intermediate Process is hosted by an IPFIX Mediator.
>       Alternatively, an Intermediate Process may be hosted by an
>       Original Exporter.
> 
>    IPFIX Mediator
> 
>       An IPFIX Mediator is an IPFIX Device that provides IPFIX Mediation
>       by receiving a record stream from some data sources, hosting one
>       or more Intermediate Processes to transform that stream, and
>       exporting the transformed record stream into IPFIX Messages via an
>       Exporting Process.  In the common case, an IPFIX Mediator receives
>       a record stream from a Collecting Process, but it could also
>       receive a record stream from data sources not encoded using IPFIX,
>       e.g., in the case of conversion from the NetFlow V9 protocol
>       [RFC3954] to IPFIX protocol.
> 
>    Specific types of IPFIX Mediators are defined below.
> 
>    IPFIX Proxy
> 
>       An IPFIX Proxy is an IPFIX Mediator that converts a record stream
>       for the purpose of protocol conversion.
> 
>    IPFIX Concentrator
> 
>       An IPFIX Concentrator is an IPFIX Mediator that receives a record
>       stream from one or more Exporters and performs correlation,
>       aggregation, and/or modification.
> 
>    IPFIX Distributor
> 
>       An IPFIX Distributor is an IPFIX Mediator that receives a record
>       stream from one or more Exporters and exports each record to one
>       or more Collectors, deciding to which Collector(s) to export each
>       record depending on the decision of an Intermediate Process.
> 
>    IPFIX Masquerading Proxy
> 
>       An IPFIX Masquerading Proxy is an IPFIX Mediator that receives a
>       record stream from one or more Exporters to screen out parts of
>       records according to configured policies in order to protect the
>       privacy of the network's end users or to retain sensitive data of
>       the exporting organization.

Do we really need these four terms?
I think that they can be removed. As Brian said, it is difficult to 
classify IPFIX Mediators according to these types.
All later occurrences of these terms can be replaced by "IPFIX Mediator".

> 4.  Problem Statement
> 
>    Network administrators generally face the problems of measurement
>    system scalability, flow-based measurement flexibility, and export
>    reliability, even if some techniques, such as Sampling, Filtering,
>    Data Records aggregation and export replication, have already been
>    developed.  The problems consist of optimizing the resources of the

How can you "optimize the resources"?

>    measurement system while fulfilling appropriate conditions: data
>    accuracy, flow granularity, and export reliability.  These conditions
>    depend on two factors.
> 
>    o  measurement system capacity:
>       This consists of the bandwidth of the management network, the
>       storage capacity, and the performances of the collecting devices
>       and exporting devices.
> 
>    o  application requirements:
>       Different applications, such as traffic engineering, detecting
>       traffic anomalies, and accounting, etc., impose different Flow

remove "etc."

>       Record granularities, and data accuracies.
> 
>    The sustained growth of IP traffic has been overwhelming the
>    measurement system capacities.  Furthermore, a large variety of
>    applications (e.g., QoS measurement, traffic engineering, security
>    monitoring) and the deployment of measurement system in heterogeneous
>    environments have been increasing the demand and complexity of IP
>    traffic measurements.
> 
> 4.1.  Coping with IP Traffic Growth
> 
>    Enterprise or service provider networks already have multiple 10 Gb/s
>    links, their total traffic exceeding 100 Gb/s.  In the near future,
>    broadband users' traffic will increase by approximately 40% every
>    year according to [TRAFGRW].  When operators monitor traffic of 500
>    Gb/s with a packet sampling rate of 1/1000, the amount of exported
>    Flow Records from Exporters could exceed 50 kFlows/s.  This value is
>    beyond the ability of a single Collector.

This paragraph describes the situation today. Are we sure that these 
numbers are still valid next year? Maybe we are then able to process 
50kFlows/s.
I would remove all these figures.

>    To deal with this problem, current data reduction techniques
>    (Sampling and Filtering in [RFC5475], and aggregation of measurement

"packet Sampling and Filtering"?

>    data) have been generally implemented on Exporters.  Note that
>    Sampling technique leads to potential loss of small Flows.  With both

"packet Sampling leads to..."

>    Sampling and aggregation techniques, administrators might no longer
>    be able to detect and investigate subtle traffic changes and
>    anomalies as this requires detailed Flow information.  With
>    Filtering, only a subset of the Data Records are exported.
> 
>    Considering the potential drawbacks of Sampling, Filtering, and Data
>    Records aggregation, there is a need for a large-scale collecting
>    infrastructure that does not rely on data reduction techniques.

Hm, I do not see this problem if a Collector receives data from a single 
Exporter. You should say that these problems arise if multiple Exporters 
send data to a single Collector.

> 4.2.  Coping with Multipurpose Traffic Measurement
> 
>    Different monitoring applications impose different requirements on
>    the monitoring infrastructure.  Some of them require traffic
>    monitoring at a Flow level while others need information about
>    individual packets or just Flow aggregates.
> 
>    To fulfill these divers requirements, an Exporter would need to
>    perform various complex metering tasks in parallel, which is a
>    problem due to limited resources.  Hence, it can be advantageous to
>    run the Exporter with a much simpler setup and to perform appropriate
>    post-processing of the exported Data Records at a later stage.
> 
> 4.3.  Coping with Heterogeneous Environments
> 
>    Network administrators use IPFIX Devices and PSAMP Devices from
>    various vendors, various software versions, various device types
>    (router, switch, or probe) in a single network domain.  Even legacy
>    flow export protocols are still deployed in current network.  This
>    heterogeneous environment leads to differences in Metering Process
>    capabilities, Exporting Process capacity (export rate, cache memory,
>    etc.), and data format.  For example, probes and switches cannot
>    retrieve some derived packet properties in [RFC5102] from a routing
>    table.

remove "in [RFC5102]"

>    To deal with this problem, the measurement system needs to mediate
>    the differences.  However, equipping all collecting devices with this
>    absorption function is difficult.
> 
> 4.4.  Summary
> 
>    In optimizing the resources of a measurement system, it is important

I still do not understand what "optimize the resources" is supposed to mean.

>    to use traffic data reduction techniques as early as possible, e.g.,
>    at the Exporter.  However, this implementation is made difficult by
>    heterogeneous environment of exporting devices.

Please revise the entire paragraph above. It only talks about data 
reduction. I do not think that this is the core of mediation.

>    This implies that a new Mediation function is required in typical
>    Exporter-Collector architectures.  Based on some applicability
>    examples, the next section shows the limitation of the typical
>    Exporter-Collector architecture model and the IPFIX Mediation
>    benefits.

> 5.  Mediation Applicability Examples
> 
> 5.1.  Adjusting Flow Granularity
> 
>    A set of common properties of simplest Flow type is a fixed 5-tuple
>    of protocol, source and destination IP addresses, and source and

As you talk about "Flow Keys", you should use this term and not invent a 
new one!

>    destination port numbers.  A shorter set of common properties, such
>    as a triple, a double, or a single property, (for example network
>    prefix, peering autonomous system number, or BGP Next-Hop fields),
>    creates more aggregated Flow Records.  This is especially useful for
>    measuring traffic exchange in an entire network domain and for easily

What do you mean by "traffic exchange in an entire network domain"?

>    adjusting the performance of Exporters and Collectors.
> 
>    Implementation analysis:
> 
>       Implementations for this case depend on where Flow granularity is
>       adjusted.  More suitable implementations use configurable Metering
>       Processes in Original Exporters.  The cache in the Metering
>       Process can specify its own set of common properties (Flow Keys)
>       and extra fields.  The Original Exporter thus creates directly
>       aggregated Flow Records.

IMO, "aggregated Flow Record" is non-sense. If you look at the 
definition of "Flow", you see that this is a normal Flow Record.
I would say that the Original Exporter generates Flow Records of the 
desired Flow granularity.

>       In the case where the Original Exporter contains a Metering
>       Process that creates fixed tuple Flow Records (no ability to

Replace "fixed tuple Flow Records" by correct IPFIX language.

>       change the Flow Keys), or PSAMP Packet Reports, an IPFIX
>       Concentrator can aggregate Data Records based on a new set of Flow
>       Keys.  Even in the case where the Original Exporter contains a
>       Metering Process for which the Flow Keys can be configured, an
>       IPFIX Concentrator can further aggregate the Flow Records.
> 
> 5.2.  Hierarchical Collecting Infrastructure
> 
>    The increase of IPFIX Exporters, the increase of the traffic, and the
>    variety of treatments expected to be performed over the Data Records

over => on

>    is more and more difficult to handle within a single Collector.
>    Hence to increase the collecting (e.g., the bandwidth capacity) and
>    processing capacity, distributed Collectors must be deployed.  As a
>    possible approach, a hierarchical structure is useful for increasing

I don't understand how the collecting and processing capacity increases 
thanks to a hierarchical structure.

The capacity increases because I implement more resources in the 
network, e.g. more Collectors.

>    the measurement systems capacity, both in export bandwidth capacity
>    and in collecting capacity.
> 
>    Implementation analysis:
> 
>       To cope with the increase of IPFIX Exporters and traffic, one

"number of IPFIX Exporters" (only IPFIX?)

>       possible implementation uses IPFIX Concentrators to build a
>       hierarchical collection system.  To cope with the variety of
>       treatments, one possible implementation uses IPFIX Distributors to
>       build a distributed collection system.  More specific cases are
>       described in section 5.9.

> 5.3.  Correlation for Data Records
> 
>    The correlation amongst Data Records or between Data Record and meta
>    data provides new metrics or information, including the following.
> 
>    o  One-to-one correlation between Data Records
> 
>       *  One way delay from the correlation of PSAMP Packet Reports from
>          different Exporters along a specific path, packet inter-arrival
>          time, etc.

"packet inter-arrival times from the correlation of PSAMP Packet Reports 
generated by a single Exporter, etc."

>       *  Treatment from the correlation of Data Records with the common

remove "the" in front of "common"

>          properties, observed at incoming/outgoing interfaces.  Examples
>          are the rate-limiting ratio, the compression ratio, the
>          optimization ratio, etc.
> 
>    o  Correlation amongst Data Records
> 
>       Average/maximum/minimum values from correlating multiple Data
>       Records.  Examples are the average/maximum/minimum number of
>       packets of the measured Flows, the average/maximum/minimum one way
>       delay, the average/maximum/minimum number of lost packets, etc.
> 
>    o  Correlation between Data Record and other meta data
> 
>       Examples are some BGP attributes associated with Data Record by
>       looking up the routing table.
> 
>    Implementation analysis:
> 
>       One possible implementation for this case uses an IPFIX
>       Concentrator located between the Metering Processes and Exporting

IPFIX Concentrator => Intermediate Process

>       Processes on the Original Exporter, or alternatively a separate
>       IPFIX Concentrator located between the Original Exporters and
>       IPFIX Collectors.
> 
> 5.4.  Time Composition
> 
>    Time composition is defined as the aggregation of consecutive Data
>    Records with common properties.  It leads to the same output as

applies to Flow Records only?
common properties => Flow Keys?

>    setting a longer active interval timer on Original Exporters with one

active timeouts?

>    advantage: the creation of new metrics such as average, maximum and
>    minimum values from Flow Records with a shorter time interval enables
>    administrators to keep track of changes that might have happened
>    during the time interval.

Hm, changes can be much better detected by looking at the short-lived 
values directly instead of looking at the long-term average, maximum, or 
minimum.

> 
>    Implementation analysis:
> 
>       One possible implementation for this case uses an IPFIX
>       Concentrator located between the Metering Processes and Exporting

Intermediate Process

>       Processes on the Original Exporter, or alternatively a separate
>       IPFIX Concentrator located between the Original Exporters and
>       IPFIX Collectors.
> 
> 5.5.  Spatial Composition
> 
>    Spatial composition is defined as the aggregation of Data Records in
>    a set of Observation Points within an Observation Domain, across
>    multiple Observation Domains from a single Exporter, or even across
>    multiple Exporters.  The spatial composition is divided into four
>    types.
> 
>    o  Case 1: Spatial Composition within one Observation Domain
> 
>       For example, in the case where a link aggregation exists, Data

remove "a"

>       Records metered at physical interfaces belonging to the same trunk
>       can be merged.
> 
>    o  Case 2: Spatial Composition across Observation Domains, but within
>       a single Exporter

Original Exporter?

>       For example, in the case where a link aggregation exists, Data

remove "a"

>       Records metered at physical interfaces belonging to a same trunk
>       grouping beyond the line interface module can be merged.

"line card"?

> 
>    o  Case 3: Spatial Composition across Exporters
> 
>       Data Records metered within an administrative domain, such as the
>       west area and east area of an ISP network, can be merged.
> 
>    o  Case 4: Spatial Composition across administrative domains
> 
>       Data Records metered across administrative domains, such as across
>       different customer networks or different ISP networks, can be
>       merged.

Are more cases thinkable?
If yes, I would call the above "cases" "examples".

>    Implementation analysis:
> 
>       One possible implementation for the cases 1 and 2 uses an IPFIX
>       Concentrator located between the Metering Processes and Exporting

Intermediate Process

>       Processes on the Original Exporter.  A separate IPFIX Concentrator
>       located between the Original Exporters and IPFIX Collector is a
>       valid solution for the cases 1, 2, 3, and 4.
> 
> 
> 5.6.  Data Record Anonymization
> 
>    IPFIX exports across administrative domains can be used to measure
>    traffic for wide-area traffic engineering or to analyze Internet
>    traffic trends, as described in the spatial composition across
>    administrative domains in the previous subsection.
>    In such a case, administrators need to adhere to privacy protection
>    policies and prevent access to confidential traffic measurements by
>    other people.  Typically, anonymization techniques enables the

enable

>    provision of traffic data to other people without violating these
>    policies.
> 
>    Generally, anonymization modifies a data set to protect the identity
>    of the people or entities described by the data set from being
>    disclosed.  It also attempts to preserve sets of network traffic
>    properties useful for a given analysis while ensuring the data cannot
>    be traced back to the specific networks, hosts, or users generating
>    the traffic.  For example, IP address anonymization is particularly
>    important for avoiding the identification of the users, hosts, and

remove second "the"

>    routers.  As another example, when ISP provides a traffic monitoring

an ISP

>    service to end customers by their own Exporters, even in case of
>    exporting interface index fields, network administrators take care of
>    anonymizing its fields to avoid disclosing the vulnerability.

Why does an interface represent a vulnerability?

>    Implementation analysis:
> 
>       One possible implementation for this case uses an anonymization
>       function at the Original Exporter.  However, this increases the
>       load on the Original Exporter.  A more flexible implementation
>       uses a separate IPFIX Masquerading Proxy between the Original
>       Exporter and Collector.

> 5.10.  Flow-based Sampling and Selection
> 
>    Generally, the distribution of the number of packets per Flow seems
>    to be heavy-tailed.  Most types of Flow Records are likely to be
>    small Flows consisting of a small number of packets.  The measurement
>    system is overwhelmed with a huge amount of these small Flows.  If
>    statistics information of small Flows is exported as merged data by
>    applying a policy or threshold, the load on the Exporter is reduced.
>    Furthermore, if the flow distribution is known, exporting only a
>    subset of the Data Records might be sufficient.
> 
>    Implementation analysis:
> 
>       One possible implementation for this case uses an IPFIX
>       Concentrator located between the Metering Processes and Exporting

Intermediate Process

>       Processes on the Original Exporter, or alternatively a separate
>       IPFIX Concentrator located between the Original Exporters and
>       IPFIX Collectors.  A set of IPFIX Mediation functions, such as
>       filtering, selecting and aggregation is used in the IPFIX
>       Concentrator.

> 6.  IPFIX Mediators Implementation Specific Problems
> 
> 6.1.  Loss of Original Exporter Information
> 
>    Both the Exporter IP address indicated by the source IP address of
>    the IPFIX Transport Session and the Observation Domain ID included in
>    the IPFIX Message header are likely to be lost during IPFIX
>    Mediation.  In some cases, a IPFIX Masquerading Proxy might drop the

a => an

>    information deliberately.  In general, however, the Collector must
>    recognize the origin of the measurement information, such as the IP
>    address of the Original Exporter, the Observation Domain ID, or even
>    the Observation Point ID.  Note that, if an IPFIX Mediator can not

cannot

>    communicate the Original Exporter IP address, then the IPFIX
>    Collector will wrongly deduce that the IP address of the IPFIX
>    Mediator is that of the Original Exporter.
> 
>    In the following figure, a Collector can identify two IP addresses:
>    10.1.1.3 (IPFIX Mediator) and 10.1.1.2 (Exporter#2), respectively.
>    The Collector, however, needs to somehow recognize both Exporter#1
>    and Exporter#2, which are the Original Exporters.  The IPFIX Mediator
>    must be able to notify the Collector about the IP address of the
>    Original Exporter.
> 
>    .----------.          .--------.
>    |IPFIX     |          |IPFIX   |
>    |Exporter#1|--------->|Mediator|---+
>    |          |          |        |   |
>    '----------'          '--------'   |      .---------.
>    IP:10.1.1.1         IP:10.1.1.3    '----->|IPFIX    |
>    ODID:10             ODID:0                |Collector|
>                                       +----->|         |
>    .----------.                       |      '---------'
>    |IPFIX     |                       |
>    |Exporter#2|-----------------------'
>    |          |
>    '----------'
>    IP:10.1.1.2
>    ODID:20
> 
>    Figure B: Loss of Original Exporter Information.
> 
> 6.2.  Loss of Base Time Information
> 
>    The Export Time field included in the IPFIX Message header represents
>    a reference timestamp for Data Records.  Some IPFIX Information
>    Elements, described in [RFC5102], carry delta timestamps that
>    indicate the time difference from the value of the Export Time field.
>    If the Data Records include any delta time fields and the IPFIX
>    Mediator overwrites the Export Time field when sending IPFIX
>    Messages, the delta time fields become meaningless and, because
>    Collectors cannot recognize this situation, wrong time values are
>    propagated.
> 
> 6.3.  Transport Sessions Management
> 
>    Maintaining relationships between the incoming Transport Sessions and
>    the outgoing ones depends on the Mediator's implementation.  If an
>    IPFIX Mediator relays multiple incoming Transport Sessions to a
>    single outgoing Transport Session, and if the IPFIX Mediators shuts
>    down its outgoing Transport Session, Data Records of the incoming
>    Transport Sessions would not be relayed any more.  In the case of
>    resetting an incoming session, the behavior of the IPFIX Mediator

Transport Session

>    needs to be specified.

> 6.7.  Exporting the Function Item
> 
>    In some case, the IPFIX Collector needs to recognize which specific
>    function(s) the IPFIX Mediation has executed on the Data Records.

remove first "the"

>    The IPFIX Collector cannot distinguish between time composition,
>    spatial composition, and Flow Key aggregation, if the IPFIX Mediator

What is "Flow Key aggregation"? Is this a good expression?
Usually, some Flow Key fields are just dropped or replaced by non-key 
fields.

>    does not export the applied function.  Some parameters related to the
>    function also would need to be exported.  For example, in case of
>    time composition, the active time of original Flow Records is

"active timeout"?

>    required to interpret the minimum/maximum counter correctly.  In case
>    of spatial composition, spatial area information on which Data
>    Records is aggregated is required.
> 
> 6.8.  Consideration for Aggregation
> 
>    Whether the aggregation is based on time or spatial composition,
>    caution should be taken on how to aggregate non-key fields in IPFIX
>    Mediation.  The IPFIX information model [RFC5102] specifies that the
>    value of non-key fields, which are derived from fields of packets or
>    from packet treatment and for which the value may change from packet
>    to packet within a single Flow, is determined by the first packet
>    observed for the corresponding Flow, unless the description of the
>    Information Element explicitly specifies a different semantics.
> 
>    However, this simple rule might not be appropriate when aggregating
>    Flow Records which have different values in a non-key field.  For
>    example, if two Flows with identical Flow Key values are measured at
>    different Observation Points, they may contain identical packets
>    observed at different locations in the network and at different
>    points in time.  On their way from the first to the second
>    Observation Point, some of the packet fields, such as the DSCP, may
>    have changed.  Hence, if the Information Element ipDiffServCodePoint
>    is included as a non-key field, it can be useful to include the DSCP
>    value observed at either the first or the second Observation Point in
>    the resulting Flow Record, depending on the application.
> 
>    Other potential solutions include: removing the Information Element
>    ipDiffServCodePoint from the Data Record when re-exporting the
>    aggregate Flow Record, changing the Information Element
>    ipDiffServCodePoint from a non key-field to a Flow Key when re-
>    exporting the aggregated Flow Record, or assigning a non valid value
>    for the Information Element to express to the Collector that this
>    Information Element is meaningless.
> 
>    Furthermore, rules must be specify on how to aggregate the new
>    Configured Selection Fraction an IPFIX Mediator should report when

What about:
"If packet Sampling or Filtering is applied, the IPFIX Mediator must 
report an adjusted PSAMP Configured Selection Fraction when aggregating..."

>    aggregating IPFIX Flow Records with different sampling rates.
>    Finally, special care must be taken when aggregating Flow Records
>    resulting from different Sampling techniques such as Systematic
>    Count-Based Sampling and Random n-out-of-N Sampling for example.
> 
> 
> 7.  Summary and Conclusion
> 
>    This document described the problems that network administrators have
>    been facing, the applicability of IPFIX Mediation to these problems,
>    and the problems related to the implementation of IPFIX Mediators.
>    To assist the operations of the Exporters and Collectors, there are
>    various IPFIX Mediations from which the administrators may select.
>    Examples of the applicability of IPFIX Mediation are as follows.
> 
>    o  Regarding large-scale measurement system, IPFIX Concentrators or
>       IPFIX Distributors help to achieve traffic analysis with high data
>       accuracy and fine flow granularity even as IP traffic grows.  As
>       IPFIX Mediation capabilities, Flow sampling, aggregation, and
>       composition are effective.

Sampling and aggregation reduce the accuracy or granularity.
Correlation seems to be appropriate.

>    o  Regarding data retention, IPFIX Mediators enhance the export
>       reliability, and the storage of the measurement system.
> 
>    o  Regarding the distribution of Data Records, IPFIX Distributors
>       help to achieve multipurpose traffic analysis for different
>       organizations, or help to achieve respective traffic analysis

remove "respective"?

>       based on Data Record types(IPv4, IPv6, MPLS, and VPN).
> 
>    o  Regarding the IPFIX export across domains, IPFIX Masquerading
>       Proxies help administrators to anonymize or filter Data Records,
>       preventing privacy violations.
> 
>    o  Regarding interoperability, IPFIX Proxies provide interoperability
>       between legacy protocols and IPFIX, even during the migration

even => for example

>       period to IPFIX.
> 
>    As a result, the IPFIX Mediation benefits become apparent.  However,
>    there are still some open issues with the use of IPFIX Mediators.
> 
>    o  Both Observation Point and IPFIX Message header information, such
>       as the Exporter IP address, Observation Domain ID, and Export Time
>       field, might be lost.  This data should therefore be communicated
>       between the Original Exporter and Collector via the IPFIX
>       Mediator.
> 
>    o  IPFIX Mediators are required to manage Transport Sessions,
>       Template IDs, and Observation Domain IDs.  Otherwise, anomalous
>       IPFIX Messages could be created.
> 
>    o  Data Records defined by Options Templates, such as those reporting
>       the Sampling rate and Sampling algorithm used, might be lost
>       during IPFIX Mediation.  If a Collector is not informed of current
>       Sampling rates, traffic information might become worthless.
> 
>    These problems stem from the fact that no standards regarding IPFIX
>    Mediation have been set.  In particular, the minimum set of
>    information that should be communicated between Original Exporters
>    and Collectors, the management between different IPFIX Transport
>    Sessions, and the internal components of IPFIX Mediators should be
>    standardized.

There is a lot of repetition in this section.

> 8.  Security Considerations
> 
>    A flow-based measurement system must prevent potential security
>    threats: the disclosure of confidential traffic data, injection of
>    incorrect data, and unauthorized access to traffic data.  These
>    security threats of the IPFIX protocol are covered by the security
>    considerations section in [RFC5101] and are still valid for IPFIX
>    Mediators.
> 
>    And a measurement system must also prevent the following security

remove "And"

>    threats related to IPFIX Mediation:
> 
>    o  Attacks against IPFIX Mediator
> 
>       IPFIX Mediators can be considered as a prime target for attacks,
>       as an alternative to IPFIX Exporters and Collectors.  IPFIX
>       Proxies or Masquerading Proxies need to prevent unauthorized
>       access or denial-of-service (DoS) attacks from untrusted public
>       networks.
> 
>    o  Man-in-the-middle attack by untrusted IPFIX Mediator
> 
>       The Exporter-Mediator-Collector structure model would increase the
>       risk of the man-in-the-middle attack.

"would increase the risk of..." => "could be misused for 
man-in-the-middle attacks"

>    o  Configuration on IPFIX Mediation
> 
>       In the case of IPFIX Distributors and IPFIX Masquerading Proxies,
>       an accidental misconfiguration and unauthorized access to
>       configuration data could lead to the crucial problem of disclosure
>       of confidential traffic data.

-- 
Dipl.-Ing. Gerhard Münz
Chair for Network Architectures and Services (I8)
Technische Universität München - Department of Informatics
Boltzmannstr. 3, 85748 Garching bei München, Germany
Phone:  +49 89 289-18008       Fax: +49 89 289-18033
E-mail: [email-masked]    WWW: http://www.net.in.tum.de/~muenz

---

From: sujay gupta <[email-masked]>
Date: Sat, 31 Oct 2009 12:57:41 +0530

Hi,
A few comments.
1) I have a slight problem overall with the language structuring
especially from section 1 to 4. It could be made simple.
2) IPFIX mediation as a service or device? is intermixed, can we
differentiate that? also can we call the different type of mediators
as services?
3) Is there any line of thought of having the mediation device
incorporate record compression to reduce b/w?(
draft-muenz-ipfix-compression) as another service, or deal with
compressed data from the original exporter?
BR,
-Sujay

On Mon, Oct 19, 2009 at 10:32 AM, Juergen Quittek <[email-masked]> wrote:
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix
>
>