Subject: Re: [IPFIX] draft-ietf-ipfix-protocol-26.txt

From: "Boschi, Elisa" <[email-masked]>
Date: Thu, 27 Sep 2007 08:56:38 +0100

Scott,

we received similar comments during the transport directorate review of the IPFIX implementation guidelines. The new revision of the document, now available at:

http://www.ietf.org/internet-drafts/draft-ietf-ipfix-implementation-guidelines-07.txt

might address your concerns.

I'm copying the UDP section here below for your convenience.

Elisa

---

6.2.  UDP

   UDP is useful in simple systems where an SCTP stack is not available,
   and where there is insufficient memory for TCP buffering.

   However, UDP is not a reliable transport protocol, and IPFIX messages
   sent over UDP might be lost as with partially-reliable SCTP streams.
   UDP is not the recommended protocol for IPFIX and is intended for use
   in cases in which IPFIX is replacing an existing NetFlow
   infrastructure, with the following properties:

   o  A dedicated network,

   o  within a single administrative domain,

   o  where SCTP is not available due to implementation constraints,

   o  and the collector is as topographically close as possible to the
      exporter.

   Note that because UDP itself provides no congestion control
   mechanisms, it is recommended to use UDP transport only on managed
   networks, where the network path has been explicitly provisioned for
   IPFIX traffic through traffic engineering mechanisms, such as rate
   limiting or capacity reservations.

   An important example of an explicitly provisioned managed network for
   IPFIX is use of IPFIX to replace a functioning NetFlow implementation
   on a dedicated network.  In this situation, the dedicated network
   should be provisioned in accordance with the NetFlow deployment
   experience that flow export traffic generated by monitoring an
   interface will amount to 2-5% of the monitored interface's bandwidth.

   As recommended in [I-D.ietf-tsvwg-udp-guidelines] an application
   SHOULD NOT send UDP messages that result in IP packets that exceed
   the MTU of the path to the destination and SHOULD enable UDP
   checksums (see sections 3.2 and 3.4 of
   [I-D.ietf-tsvwg-udp-guidelines] respectively).

   Since IPFIX assumes reliable transport of templates over SCTP, this
   necessitates some changes for IPFIX template management over UDP.
   Templates sent from the Exporting Process to the Collecting Process
   over UDP MUST be resent at regular time intervals; these intervals
   MUST be configurable.

   We recommend a default Template resend time of 10 minutes,
   configurable between 1 minute and 1 day.

   Note that this could become an interoperability problem, e.g. if an
   Exporter re-sends Templates once per day, while a Collector expires
   Templates hourly, then they may both be IPFIX-compatible, but not be
   interoperable.

   Retransmission time intervals that are too short waste bandwidth on
   unnecessary template retransmissions.  On the other hand, time
   intervals that are too long introduce additional costs or risk of
   data loss by potentially requiring the Collector to cache more data
   without having the Templates available to decode it.

   To increase reliability and limit the amount of potentially lost data
   the Exporting Process MAY resend additional templates using a packet-
   based schedule.  In this case Templates are resent depending on the
   number of data packets sent.  Similarly to the time interval,
   resending a Template every few packets introduces additional overhead
   while resending after a large amount of packets have already been
   sent means high costs due to the data caching and potential data
   loss.

   We recommend a default Template resend interval of 20 packets,
   configurable between 1 and 1000 packets.

   Note that a sufficiently small resend time or packet interval may
   cause a system to become stuck, continually re-sending templates.
   e.g., if the resend packet interval is 2 (i.e., templates are to be
   sent in every other packet) but more than two packets are required to
   send all the templates, then the resend interval will have expired by
   the time the templates have been sent, and templates will be sent
   continuously - possibly preventing any data from being sent at all.
   Therefore the Template resend intervals should be considered from the
   last data packet, and should not be tied to specific sequence
   numbers.

   The Collecting Process SHOULD use the Sequence Number in the IPFIX
   Message header to determine whether any messages are lost.

   The following may be done to mitigate message loss:

   o  Move the Collector topologically closer to the Exporter.

   o  Increase the bandwidth of the links through which the Data Records
      are exported.

   o  Use sampling, filtering, or aggregation in the Metering Process to
      reduce the amount of exported data.

   o  Increase the buffer size at the Collector and/or the Exporter.

   Before using a Template for the first time, the Exporter may send it
   in several different IPFIX Messages spaced out over a period of
   packets in order to increase the likelihood that the Collector has
   received the Template.

   Template Withdraw messages MUST NOT be sent over UDP; the Exporter
   must rely on expiration at the Collector to expire old Templates or
   to reuse Template Ids.

   We recommend that the collector implements a template expiry of three
   times the Exporter refresh rate.

   However, since the IPFIX protocol doesn't provide any mechanism for
   the Exporter to convey any information about the Template expiry time
   to the Collector, configuration must be done out of band.

   If no out of band configuration is made, we recommend to initially
   set a template expiry time at the Collector of 60 minutes.  The
   Collecting Process MAY estimate each Exporting Process's resend time
   and adapt the expiry time for the corresponding Templates
   accordingly. 

Scott O. Bradner wrote:
> I reviewed draft-ietf-ipfix-protocol-26.txt as part of the Transport
> Area review effort.
> 
> I did not find any particular issues in the described technology - a few
> nits:
> 
> section 3.1 "Export Time" someday the IETF needs to stop using 32-bit
> "seconds since 1 jan 1970" for timing - at least within in the next 31
> years
> 
> section 6.1.2 - it might be reasonable to add the IEEE 8-byte MAC
> address as an address type - this is used in ZigBee and may be in wider
> use in the future
> 
> section 10.3.3 - a max packet size of 1280 could be used if the
> connection is known to be running in an IPv6-only environment
> 
> I'm not sure why the packet size discussion is only listed for UDP - it
> seems like the same restriction should apply to all protocols -
> fragmentation is not your friend
> 
> Historically the biggest issue with IPFIX has been that most
> implementers want to run it over UDP with consequences be dammed.  -
> this was weaseled in the IPFIX Requirements document (RFC 3917) by
> requiring (in section 6.3.1) that "For the data transfer, a congestion
> aware protocol must be supported."  This draft meets that requirement by
> making the implementation of SCTP a MUST.  That will not stop many
> implementers from ignoring the requirement for implementation or users
> to enable UDP and thus creating a potentially very high bandwidth
> non-congestion avoiding fire hose that can quite easily wipe out a net
> by misconfiguration or become a DoS engine by purposeful configuration.
> 
> I'm not sure if anything can be actually be done about this risk - It
> might help some to say that UDP is a "MUST NOT" but I doubt it - in any
> case it would help somewhat, imho, to expand section 10.3 to be clearer
> about the threats posed by any use of a non-congestion avoiding
> transport protocol or to do that in the Security Considerations section
> 
> Scott
> 
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ipfix
> 

************************************************************************************************** 
E-mail Confidentiality Notice and Disclaimer. 

This e-mail and any files transmitted with it are confidential and are intended solely for the use 
of the individual or entity to which they are addressed. Access to this e-mail by anyone else is 
unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any
action taken or omitted to be taken in reliance on it, is prohibited. E-mail messages are not 
necessarily secure. Hitachi does not accept responsibility for any changes made to this message 
after it was sent. 
Hitachi checks outgoing e-mail messages for the presence of computer viruses. 
**************************************************************************************************
_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix