Subject: [IPFIX] Structured Data WGLC comments

From: Brian Trammell <[email-masked]>
Date: Mon, 8 Nov 2010 17:53:26 +0800

Greetings, all,

I have the following two comments on draft-ietf-ipfix-structured-data-03.

First, I don't necessarily find the repeated examples showing how variable length Information Elements work to be particularly useful, but I accept that this might be because I already know how they work.

What I _am_ missing, however, are unified diagrams showing how the structured data Information Elements and variable-length encoding work together:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |      255      |      Length (0 to 65535)      |   Semantic    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |0|   Information Element ID    |        Element Length         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                      basicList content . . .                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Second, Section 5.4 is missing a discussion of when you shouldn't use Structured Data _at all_: i.e., when there is low or no semantic variation among records represented by a template. Consequently, I'm not sure about section 9.3. It seems like although we _could_ use structured data for biflows, the suggested approach misses the 1. semantic clarity and 2. export efficiency that were aims of 5103.

Beyond these points, I've reviewed the draft, and it looks good.

Best regards,

Brian

---

From: Chris Inacio <[email-masked]>
Date: Tue, 9 Nov 2010 10:04:19 -0500

First off, I agree with Brian's comments.

In section 1.1, do you want to keep saying "IP Flow information"?  Especially considering the chances of using IPFIX in other IETF activities.

Section 2: "Indeed, polling short-live Flows" to "Indeed, polling short-lived Flows".

The dangling "Figure F" label on page 22-23 is really annoying.

I would like to see a better discussion about how structured data elements change the TLV tuple contract that is the core of the IPFIX model today.  I personally think that is the largest change with how we will handle these new data types. 

Even more importantly for our application, I would really like to see a way of passing metadata about a structured template as well.  Can we expand structured data to include some options elements so that I can do something like say "template 255 is email"  "template 256 is IMAP" / "template 257 is POP3".  Something like that.  We're really worried about the template issue withdraw problem that seems to be creeping up on us when we use structured data.

As a point of reference, we have structured data fully implemented in our fixbuf library.  We're currently converting the DPI capabilities that are in YAF to use structured data for export.  We're still working through the issues of how to manage templates using structured data.  We have the issue of that, for example, if you want capture details on HTTP, then sometimes the "referer" field will be present, and sometimes it won't.  If we want to encode that efficiently, we need to either change the templates, or have multiples available.  If we have multiples, we always want the collector to understand that these N templates are HTTP, otherwise we have to do a lot of withdraws of templates.

I really like structured data, especially if I find more than one HTTP header in a network stream (proxying anyone?) because it captures the relationship.  But right now, I'm not sure how to manage it.

I'll try to be on the webex if I can.

regards,
Chris Inacio
[email-masked]

On Nov 8, 2010, at 4:53 AM, Brian Trammell wrote:

> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix

---

From: Benoit Claise <[email-masked]>
Date: Wed, 24 Nov 2010 12:36:25 +0100

Dear all,

Let me address a single point from Chris' email in this thread
> In section 1.1, do you want to keep saying "IP Flow information"?  Especially considering the chances of using IPFIX in other IETF activities.
You have a good point.
As we concluded already on the mailing list, IPFIX scope is not IP anymore
As we discussed at the last IPFIX meeting, the meeting minutes expresses:

    Juergen led a discussion on "should we move the IPFIX base standards
    (RFCs 5101, 5102 and 5103) to Draft Standard.  There was consensus
    that this would be worthwhile if it corrected all the bugs discovered
    to date in the documents, and added explanatory text to make them
    easier to understand. _We also need to remove 'IP' from the IPFIX Flow
    definition, since that is proving to be a blocking factor or the ITU-T
    NGN to reference IPFIX._ 

How to address this in the new drafts? Most new IPFIX drafts have 
section like these

            1  <http://tools.ietf.org/html/draft-ietf-ipfix-structured-data-03#section-1>. Overview.................................................7  <http://tools.ietf.org/html/draft-ietf-ipfix-structured-data-03#page-7>
            1.1  <http://tools.ietf.org/html/draft-ietf-ipfix-structured-data-03#section-1.1>. IPFIX Documents Overview............................7  <http://tools.ietf.org/html/draft-ietf-ipfix-structured-data-03#page-7>
            1.2  <http://tools.ietf.org/html/draft-ietf-ipfix-structured-data-03#section-1.2>. Relationship between IPFIX and PSAMP................8  <http://tools.ietf.org/html/draft-ietf-ipfix-structured-data-03#page-8>

I see two solutions:
1. maybe it's time to start changing from IP Flow to Flow in those 
sections for the new drafts.
2. alternatively, we add a generic note (somewhere in the same sections) 
that IPFIX scope has been extended to non IP, even if RFC5101 and 
RFC5102 focuses on IP.

What do you think?

Regards, Benoit.