Subject: Re: [IPFIX] draft-ietf-ipfix-file-00

From: Hitoshi Irino <[email-masked]>
Date: Fri, 08 Feb 2008 17:03:00 +0900

Hello Brian,

I read this draft.
These are my comments.

1:
I think it is better to change the name "IPFIX File Writer" to "IPFIX File 
Writing Process" and "IPFIX File Reader" to "IPFIX File Reading Process".
Because, Terminology of IPFIX File Writer and Reader defines these are processes.
    IPFIX File Reader:   An IPFIX File Reader is a Process which reads
       IPFIX Files from a filesystem,
    IPFIX File Writer:   An IPFIX File Writer is a process which writes
       IPFIX Files to a filesystem,

I am reminded of IPFIX Device from the word "Writer" and "Reader".

I use the word "IPFIX File Writing Process" instead of "IPFIX File Writer", 
"IPFIX File Reading Process" instead of "IPFIX File Reader" in following 
sentence of this mail.

2:
I think it is useful to define "Writing Process" and "Reading Process" which 
are broader concept of "IPFIX File Writing Process" and "IPFIX File Reading 
Process".
"Writing Process" writes the IPFIX Messages to any storage such as file 
system, database with any format.
"Reading Process" reads the IPFIX Messages from any storage such as file 
system, database with any format.

For example, when someone want to implement format converter which converts 
from any proprietary format (e.g. NetFlow v5) to IPFIX File Format. The 
converter contains "Reading Process" and "IPFIX File Writer".

Definition of "Writing Process" and "Storing Process" in 
draft-kobayashi-ipfix-mediator-model-01 are almost same. It is better that 
they should be merged, I think.

(2008/01/11 10:37), Kobayashi Atsushi wrote:
> Dear Brian,
> 
> I checked your draft in less time. I confirmed that ipfix file is useful
> for compatibility and testing. This format makes easily me try to
> implement file writer and reader. To clarify your draft and improve my
> understanding, I write down several comments, as follows.
> 
> Q1 
> I think that main part of this document is section 3,7. 
> This draft has many tips. But, from viewpoint of easy understanding of
> the ipfix file format, I prefer that the draft is divided by main part
> and other parts.
> 
> Q2
> Should a ipfix file include whole message on one session, even if one
> session has different Observation Domain ID messages?
> From viewpoint of easy decoding and reducing of file size, 
> I want to make ipfix file based on same Observation Domain ID.
> It depends on case or policy. 
> 
> Q3
> I could not understand following description in section 7. 
> Please give more specific example.
> 
>    o  File Writers SHOULD emit Data Records described by Options
>       Templates to appear in the file before any Data Records which
>       depend on the scopes defined by those options.
> 
> Generally, Options that present sampling, sampling algorithm and
> filtering method in psamp. These templates and associated data records
> should be included in a ipfix file.
> Does above description mean these points?
> 
> Q4
> Some of your proposed option templates are useful for several
> cases other than file format. For example, there are Anonymization Options
> Template and Message Details Options Template.
> Mediator draft describes similar template in section 5 .
> I think that this part could become independent document after we
> explore sophisticated way that can be used for both functions.
> 
> Thanks,
> Atsushi KOBAYASHI
> 
> 
> On Thu, 3 Jan 2008 19:07:19 -0500
> Brian Trammell <[email-masked]> wrote:
> 
>> All,
>>
>> draft-ietf-ipfix-file-00 is now available. Many thanks to those of  
>> you who have reviewed and commented on the draft. Changes from  
>> individual revision -05 are as follows:
>>
>> 1. Reorganized the structure of the document somewhat for readability.
>>
>> 2. Minor changes to template management language for clarity.
>>
>> 3. Minor changes to options template definitions to clarify which IEs  
>> are scope IEs.
>>
>> 4. Terminology capitalization review.
>>
>> We plan on at least one more revision of this document before WG last  
>> call to address remaining open issues raised by reviews to date:
>>
>> 1. Define a mechanism for containing IPFIX data and non-IPFIX data  
>> (e.g. proprietary metadata, headers and footers) within the same file  
>> in an interoperable way, or declare such a mechanism explicitly out  
>> of scope. It seems there is definite WG interest for such a  
>> mechanism; we'll propose a mechanism for WG discussion this month.
>>
>> 2. Collection details options templates and information element  
>> definitions assume a CP is involved in the writing of a file;  
>> however, routers and other MPs can write IPFIX Files, too, so we will  
>> remove this assumption where it appears.
>>
>> 3. The draft should provide guidelines for a CP collecting from  
>> multiple SCTP streams and writing those records into a file.
>>
>> 4. The security considerations section should treat the security of  
>> IPFIX Files written by CPs using TLS for transport security separately.
>>
>> 5. The document should contain an IPFIX Documents Overview section to  
>> place the file draft in context.
>>
>> 6. The document should contain an XML schema definition appendix for  
>> the IEs it defines.
>>
>> 7. It has been suggested that the file draft's handling of  
>> anonymization use a bitmap IE to be parallel with the protocol's  
>> flowKeyIndicator; however, it is unclear that a bitmap will be  
>> sufficient for anonymization notation, so more discussion is  
>> warranted here.
>>
>> 8. It has also been suggested that the appendices within the file  
>> draft on comparing and translating NetFlow V9 data into IPFIX, used  
>> as an informative background for the section within the document on  
>> applying IPFIX Files to V9 collection infrastructures, might make a  
>> good draft in its own right. We'd like to assess WG interest in such  
>> a draft before splitting the file draft again.
>>
>> Regards,
>>
>> Brian
>>
>> Begin forwarded message:
>>
>>> From: [email-masked]
>>> Date: January 2, 2008 12:20:02 PM EST
>>> To: [email-masked]
>>> Cc: [email-masked]
>>> Subject: [IPFIX] I-D Action:draft-ietf-ipfix-file-00.txt
>>>
>>> A New Internet-Draft is available from the on-line Internet-Drafts  
>>> directories.
>>> This draft is a work item of the IP Flow Information Export Working  
>>> Group of the IETF.
>>>
>>>
>>> 	Title           : An IPFIX-Based File Format
>>> 	Author(s)       : B. Trammell, et al.
>>> 	Filename        : draft-ietf-ipfix-file-00.txt
>>> 	Pages           : 50
>>> 	Date            : 2008-01-02
>>>
>>> This document describes a file format for the storage of flow data
>>> based upon the IPFIX Message format.  It proposes a set of
>>> requirements for flat-file, binary flow data file formats, then
>>> applies the IPFIX message format to these requirements to build a new
>>> file format.  This IPFIX-based file format is designed to facilitate
>>> interoperability and reusability among a wide variety of flow
>>> storage, processing, and analysis tools.
>>>
>>> A URL for this Internet-Draft is:
>>> http://www.ietf.org/internet-drafts/draft-ietf-ipfix-file-00.txt
>>>
>>> To remove yourself from the I-D Announcement list, send a message to
>>> [email-masked] with the word unsubscribe in the body of
>>> the message.
>>> You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce
>>> to change your subscription settings.
>>>
>>> Internet-Drafts are also available by anonymous FTP. Login with the
>>> username "anonymous" and a password of your e-mail address. After
>>> logging in, type "cd internet-drafts" and then
>>> 	"get draft-ietf-ipfix-file-00.txt".
>>>
>>> A list of Internet-Drafts directories can be found in
>>> http://www.ietf.org/shadow.html
>>> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>>>
>>> Internet-Drafts can also be obtained by e-mail.
>>>
>>> Send a message to:
>>> 	[email-masked].
>>> In the body type:
>>> 	"FILE /internet-drafts/draft-ietf-ipfix-file-00.txt".
>>>
>>> NOTE:   The mail server at ietf.org can return the document in
>>> 	MIME-encoded form by using the "mpack" utility.  To use this
>>> 	feature, insert the command "ENCODING mime" before the "FILE"
>>> 	command.  To decode the response(s), you will need "munpack" or
>>> 	a MIME-compliant mail reader.  Different MIME-compliant mail readers
>>> 	exhibit different behavior, especially when dealing with
>>> 	"multipart" MIME messages (i.e. documents which have been split
>>> 	up into multiple messages), so check your local documentation on
>>> 	how to manipulate these messages.
>>>
>>> Below is the data which will enable a MIME compliant mail reader
>>> implementation to automatically retrieve the ASCII version of the
>>> Internet-Draft.
>>> Content-Type: text/plain
>>> Content-ID: <2008-01-02121517.I-D\@ietf.org>
>>>
>>> _______________________________________________
>>> IPFIX mailing list
>>> [email-masked]
>>> https://www1.ietf.org/mailman/listinfo/ipfix
>>
>> _______________________________________________
>> IPFIX mailing list
>> [email-masked]
>> https://www1.ietf.org/mailman/listinfo/ipfix
> 
> --- 
> Atsushi KOBAYASHI  <[email-masked]>
> NTT Information Sharing Platform Lab.
> tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637
> 
> 
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ipfix
> 

-- 
Hitoshi Irino
NTT Network Service Systems Laboratories
9-11 Midori-cho 3-Chome, Musashino-shi, Tokyo 180-8585 Japan
Tel: +81-422-59-4403  Fax: +81-422-59-4549
_______________________________________________
IPFIX mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/ipfix

---

From: Hitoshi Irino <[email-masked]>
Date: Fri, 08 Feb 2008 17:03:00 +0900

Hello Brian,

I read this draft.
These are my comments.

1:

2: