Subject: [IPFIX] Fwd: New Version Notification for draft-trammell-ipfix-set9hack-00

From: Brian Trammell <[email-masked]>
Date: Mon, 26 Jul 2010 16:32:49 +0200

Greetings, all,

See the abstract for details; there's a little (but not much) more in the three-page draft itself.

Assume you have a file containing NetFlow V9 PDUs. Assume you'd like to split it into packets by seeking. Well, you can't do this, really, without completely parsing and counting records, because while Sets have lengths, the count header in the Packet header counts records. This implies a rather slow implementation for any V9 file reader, by forcing any process reading V9 files to do all parsing in a single pass.

There's a hack around this: since Set ID 9 is not valid in V9, you can just assume Set ID 9 to be the Version of the next V9 Packet header. When you see Set ID 9, that's a new Packet. Otherwise, it's a set header, and the next set runs for the number of bytes in the set header length.

All is well, but Set ID 9 might be allocated in the future, which would cause this hack to break in a version-mixed file. (If we don't want to support version mixing in files, why bother with a version header?). So if we allocate Set ID 9 to reserved, we ensure this hack can keep working. (While we're at it, let's allocate Set ID 10, so we can recognize IPFIX as well.)

Okay. Yes. This is a hack. Why write it up in a draft? The Set ID registry is modifiable only by Standards Action, so this would appear to be the only way to make the change.

Thoughts?

Cheers,

Brian

Begin forwarded message:

> From: IETF I-D Submission Tool <[email-masked]>
> Date: July 26, 2010 4:14:25 PM GMT+02:00
> To: [email-masked]
> Subject: New Version Notification for draft-trammell-ipfix-set9hack-00 
> 
> 
> A new version of I-D, draft-trammell-ipfix-set9hack-00.txt has been successfully submitted by Brian Trammell and posted to the IETF repository.
> 
> Filename:	 draft-trammell-ipfix-set9hack
> Revision:	 00
> Title:		 Reserved Set IDs to Enable Boundary Finding in Mixed IPFIX Files
> Creation_date:	 2010-07-26
> WG ID:		 Independent Submission
> Number_of_pages: 3
> 
> Abstract:
> This document reserves the IPFIX Set IDs 9 and 10 to prevent them
> from being allocated in the future.  The reason behind this is to
> ensure that heuristic methods for determining PDU boundaries in files
> containing NetFlow V9 PDUs and IPFIX Messages work in a standards-
> compliant way.
> 
> 
> 
> The IETF Secretariat.
>

---

From: Gerhard Muenz <[email-masked]>
Date: Tue, 27 Jul 2010 11:00:29 +0200

Brian,

Assume that the parser starts reading at a message header.
Then, the first field is the version number A. The parser then may
proceed as follows:

if A == 10 then
  parse IPFIX Message, skip Sets as necessary,
  expect next message header at ipfix_hdr->length offset
else if A == 9 then
  parse Netflow.v9 Message, skip Sets as necessary (using length),
  stop parsing if next "Set ID" is in the range 2..255 (because then, it
  is not a new Set but a new message header)
end

So, there does not seem to be a problem as long as no new Set IDs in
Netflow.v9 are introduced (or at least no new Set IDs > 8).

So, if you want to specify a restriction, why not specify it for
Netflow.v9 instead of IPFIX?

Regards,
Gerhard

Brian Trammell wrote:
> Assume you have a file containing NetFlow V9 PDUs. Assume you'd like
> to split it into packets by seeking. Well, you can't do this, really,
> without completely parsing and counting records, because while Sets
> have lengths, the count header in the Packet header counts records.
> This implies a rather slow implementation for any V9 file reader, by
> forcing any process reading V9 files to do all parsing in a single
> pass.
> 
> There's a hack around this: since Set ID 9 is not valid in V9, you
> can just assume Set ID 9 to be the Version of the next V9 Packet
> header. When you see Set ID 9, that's a new Packet. Otherwise, it's a
> set header, and the next set runs for the number of bytes in the set
> header length.
> 
> All is well, but Set ID 9 might be allocated in the future, which
> would cause this hack to break in a version-mixed file. (If we don't
> want to support version mixing in files, why bother with a version
> header?). So if we allocate Set ID 9 to reserved, we ensure this hack
> can keep working. (While we're at it, let's allocate Set ID 10, so we
> can recognize IPFIX as well.)
> 
> Okay. Yes. This is a hack. Why write it up in a draft? The Set ID
> registry is modifiable only by Standards Action, so this would appear
> to be the only way to make the change.
> 
> Thoughts?
> 
> Cheers,
> 
> Brian
> 
> Begin forwarded message:
> 
>> From: IETF I-D Submission Tool <[email-masked]> Date: July
>> 26, 2010 4:14:25 PM GMT+02:00 To: [email-masked] Subject:
>> New Version Notification for draft-trammell-ipfix-set9hack-00
>> 
>> 
>> A new version of I-D, draft-trammell-ipfix-set9hack-00.txt has been
>> successfully submitted by Brian Trammell and posted to the IETF
>> repository.
>> 
>> Filename:	 draft-trammell-ipfix-set9hack Revision:	 00 Title:
>> Reserved Set IDs to Enable Boundary Finding in Mixed IPFIX Files 
>> Creation_date:	 2010-07-26 WG ID:		 Independent Submission 
>> Number_of_pages: 3
>> 
>> Abstract: This document reserves the IPFIX Set IDs 9 and 10 to
>> prevent them from being allocated in the future.  The reason behind
>> this is to ensure that heuristic methods for determining PDU
>> boundaries in files containing NetFlow V9 PDUs and IPFIX Messages
>> work in a standards- compliant way.
>> 
>> 
>> 
>> The IETF Secretariat.
>> 
> 
> _______________________________________________ IPFIX mailing list 
> [email-masked] https://www.ietf.org/mailman/listinfo/ipfix

-- 
Dr.-Ing. Gerhard Münz
Chair for Network Architectures and Services (I8)
Technische Universität München - Department of Informatics
Boltzmannstr. 3, 85748 Garching bei München, Germany
Phone:  +49 89 289-18008       Fax: +49 89 289-18033
E-mail: [email-masked]    WWW: http://www.net.in.tum.de/~muenz

---

From: Brian Trammell <[email-masked]>
Date: Tue, 27 Jul 2010 11:32:45 +0200

Hi, Gerhard,

Because I have no control over Netflow V9. :) However, if we are to assume that the definition/primary implementation of NetFlow V9 will not define any Set IDs lower than 256 other than 0 or 1, then you're right. In the absence of other information, I think we can assume this. :)

(My issue here is that I have been forced to implement this particular hack, and since there's no other good way to deal with V9 files using other than a one-pass, unified parser design, it would be nice to have some confidence that the hack will continue working in the future)

Per our conversation yesterday, I also agree that mixed files themselves, containing V5/9/IPFIX records, would violate at least a SHOULD in RFC5655 - V5 and V9 PDUs can by definition not share a Transport Session with IPFIX Messages (the template scope etc. are exclusive), and therefore should not share a (5655) File.

Cheers,

B