Subject: Re: [IPFIX] WG last call on draft-ietf-ipfix-export-per-sctp-stream-00

From: Benoit Claise <[email-masked]>
Date: Thu, 23 Oct 2008 13:24:03 +0200

Kobayashi-san,

Thank you very much for your review.
Note that the draft excerpts are in this email contained already the 
improvements proposed by Brian.
> Dear all,
>
> I did not have enough time for reviewing for the draft in Aug.
> Although the time limit exceeds, I have reviewed the draft.
>
> Please see my questions and comments and so on in lines.
>
> Q1)
> I think that the following condition is configurable choice.
> - One stream treats one Template Record and associated Data Records.
> - One stream treats some Template Records and associated Data Records
> related to same Observation Domain, or same Observation Points.
> It can deduce Data Record Loss per observation domain or observation points.
> Does this draft cover the latter pattern ?
>   
Yes, exactly. This is a configurable choice.

"The Exporter SHOULD send a single Template and corresponding Data Sets 
within a single stream in order to enable calculation of the potential 
Data Record loss for this Template ID.  However, the Exporter MAY group 
related Templates and their corresponFrom [email-masked]  Thu Oct 23 04:23:23 2008
Date: Thu, 23 Oct 2008 13:24:03 +0200
From: Benoit Claise <[email-masked]>
To: Atsushi Kobayashi <[email-masked]>
Cc: IETF IPFIX Working Group <[email-masked]>
Subject: Re: [IPFIX] WG last call on
	draft-ietf-ipfix-export-per-sctp-stream-00
Sender: [email-masked]

This is a multi-part message in MIME format.

Kobayashi-san,

Thank you very much for your review.
Note that the draft excerpts are in this email contained already the 
improvements proposed by Brian.
> Dear all,
>
> I did not have enough time for reviewing for the draft in Aug.
> Although the time limit exceeds, I have reviewed the draft.
>
> Please see my questions and comments and so on in lines.
>
> Q1)
> I think that the following condition is configurable choice.
> - One stream treats one Template Record and associated Data Records.
> - One stream treats some Template Records and associated Data Records
> related to same Observation Domain, or same Observation Points.
> It can deduce Data Record Loss per observation domain or observation points.
> Does this draft cover the latter pattern ?
>   
Yes, exactly. This is a configurable choice.

"The Exporter SHOULD send a single Template and corresponding Data Sets 
within a single stream in order to enable calculation of the potential 
Data Record loss for this Template ID.  However, the Exporter MAY group 
related Templates and their correspding Data Sets within a single 
stream so that loss statistics are calculated for the group.  This is 
suitable in cases where there are only slight variations among the 
Templates in a group, e.g. the omission of unavailable fields for export 
efficiency, and may be necessary if the SCTP association does not 
support enough streams to export each Template in its own stream."

> Q2)
> This draft does not describe how to export some statistics option
> template and data. Should they be exported on same stream exporting
> associated Template and Data Records ?
>   
Basically, you can choose. To make sure we stress that, I would like to 
improve the previous sentence

    "The Exporter SHOULD send a single Template and corresponding Data
    Sets within a single stream in order to enable calculation of the
    potential Data Record loss for this Template ID.  However, the
    Exporter MAY group related _(Options) _Templates and their
    corresponding Data Sets within a single stream so that loss
    statistics are calculated for the group.  This is suitable in cases
    where there are only slight variations among the Templates in a
    group, e.g. the omission of unavailable fields for export
    efficiency, and may be necessary if the SCTP association does not
    support enough streams to export each Template in its own stream."

However, if you chose to send them within a single stream, then the 
following text applies:

    "If a SCTP stream contains a mixture of Data Records defined by a
    Template Record and Options Template Record(s), the Data Records
    defined by the Options Template Record(s) SHOULD be sent reliably
    within the same stream so that the Collector does not consider any
    loss to be associated with the options Data Record."

>>       
>>      1. Terminology 
>>
>>         IPFIX-specific terminology used in this document is defined 
>>         in section 2 of [RFC5101].  As in [RFC5101], these IPFIX-
>>         specific terms have the first letter of a word capitalized 
>>         when used in this document. 
>>          
>>         Template Reuse Delay  
>>          
>>            The configurable timeout to allow the Collecting Process 
>>            to receive and process the last Data Record using this 
>>            Template information before which the Template Withdrawal 
>>            Message MUST NOT be sent.  [RFC5101] specifies a default 
>>            value of 5 seconds. 
>>
>>     
>
> Q3)
> In first reading, I could not understand which process uses this timer,
> Collecting process or Exporting Process.
> I think that the description of RFC5101 is more easily understandable.
> RFC5101 say as follows.
>
>    The Template Withdrawal Message MUST NOT be sent until sufficient
>    time has elapsed to allow the Collecting Process to receive and
>    process the last Data Record using this Template information.
>   
Are you fine with Brian's proposal?

    Template Reuse Delay

    The time after sending the last Data Set described by a given
    Template the Exporting Process MUST wait before sending a Template
    Withdrawal Message for the Template.  [RFC5101] specifies a default
    value of 5 seconds.

>
>>      2. Introduction 
>>
>>         The IPFIX working group has specified a protocol to export IP 
>>         Flow information [RFC5101].  This protocol is designed to 
>>         export information about IP traffic Flows and related 
>>         measurement data, where a Flow is defined by a set of key 
>>         attributes (e.g. source and destination IP address, source 
>>         and destination port, etc.).  However, thanks to its template 
>>         mechanism, the IPFIX protocol can export any type of 
>>         information, as long as the relevant Information Element is 
>>         specified in the IPFIX Information Model [RFC5102], 
>>         registered with IANA, or specified as an enterprise-specific 
>>         Information Element. 
>>          
>>         The IPFIX protocol [RFC5101] specifies that IP traffic 
>>         measurements for Flows are exported usionding Data Sets within a single 
stream so that loss statistics are calculated for the group.  This is 
suitable in cases where there are only slight variations among the 
Templates in a group, e.g. the omission of unavailable fields for export 
efficiency, and may be necessary if the SCTP association does not 
support enough streams to export each Template in its own stream."

> Q2)
> This draft does not describe how to export some statistics option
> template and data. Should they be exported on same stream exporting
> associated Template and Data Records ?
>   
Basically, you can choose. To make sure we stress that, I would like to 
improve the previous sentence

    "The Exporter SHOULD send a single Template and corresponding Data
    Sets within a single stream in order to enable calculation of the
    potential Data Record loss for this Template ID.  However, the
    Exporter MAY group related _(Options) _Templates and their
    corresponding Data Sets within a single stream so that loss
    statistics are calculated for the group.  This is suitable in cases
    where there are only slight variations among the Templates in a
    group, e.g. the omission of unavailable fields for export
    efficiency, and may be necessary if the SCTP association does not
    support enough streams to export each Template in its own stream."

However, if you chose to send them within a single stream, then the 
following text applies:

    "If a SCTP stream contains a mixture of Data Records defined by a
    Template Record and Options Template Record(s), the Data Records
    defined by the Options Template Record(s) SHOULD be sent reliably
    within the same stream so that the Collector does not consider any
    loss to be associated with the options Data Record."

>>       
>>      1. Terminology 
>>
>>         IPFIX-specific terminology used in this document is defined 
>>         in section 2 of [RFC5101].  As in [RFC5101], these IPFIX-
>>         specific terms have the first letter of a word capitalized 
>>         when used in this document. 
>>          
>>         Template Reuse Delay  
>>          
>>            The configurable timeout to allow the Collecting Process 
>>            to receive and process the last Data Record using this 
>>            Template information before which the Template Withdrawal 
>>            Message MUST NOT be sent.  [RFC5101] specifies a default 
>>            value of 5 seconds. 
>>
>>     
>
> Q3)
> In first reading, I could not understand which process uses this timer,
> Collecting process or Exporting Process.
> I think that the description of RFC5101 is more easily understandable.
> RFC5101 say as follows.
>
>    The Template Withdrawal Message MUST NOT be sent until sufficient
>    time has elapsed to allow the Collecting Process to receive and
>    process the last Data Record using this Template information.
>   
Are you fine with Brian's proposal?

    Template Reuse Delay

    The time after sending the last Data Set described by a given
    Template the Exporting Process MUST wait before sending a Template
    Withdrawal Message for the Template.  [RFC5101] specifies a default
    value of 5 seconds.

>
>>      2. Introduction 
>>
>>         The IPFIX working group has specified a protocol to export IP 
>>         Flow information [RFC5101].  This protocol is designed to 
>>         export information about IP traffic Flows and related 
>>         measurement data, where a Flow is defined by a set of key 
>>         attributes (e.g. source and destination IP address, source 
>>         and destination port, etc.).  However, thanks to its template 
>>         mechanism, the IPFIX protocol can export any type of 
>>         information, as long as the relevant Information Element is 
>>         specified in the IPFIX Information Model [RFC5102], 
>>         registered with IANA, or specified as an enterprise-specific 
>>         Information Element. 
>>          
>>         The IPFIX protocol [RFC5101] specifies that IP traffic 
>>         measurements for Flows are exported ung a TLV (type, 
>>         length, value) format.  The information is exported using a 
>>         Template Record that is sent once to export the {type, 
>>         length} pairs that define the data format for the Information 
>>         Elements in a Flow.  The Data Records specify values for each 
>>         Flow. 
>>          
>>         The IPFIX protocol [RFC5101] is flexible: it foresees the usage 
>>         of the multiple SCTP streams per association; it allows the 
>>         transmission of Data Sets, Template Sets, and/or Options 
>>         Template Sets on any stream; it offers the full or partial 
>>         reliability export of Data Sets; it proposes the ordered or out-
>>         of-order delivery of Data Sets.  However, due to bandwidth 
>>         restrictions and packet losses in the network as well as 
>>         resource constraints on the Exporter and Collector (e.g., 
>>         limited buffer sizes), it is not always possible to export all 
>>         Data Sets in a reliable way. 
>>          
>>          
>>         Without delving into the details of the specifications described 
>>         later on in this document, the basic idea is to export the 
>>         Template Record and its associated Data Sets into a single 
>>         unique SCTP stream, ideally to limit the Template ID to a single 
>>         stream, while imposing in-order transmission. 
>>          
>>         The specification in this document offers several advantages 
>>         such as: calculation of Data Record losses in case of partially-
>>         reliable SCTP export, immediate export of the Template 
>>         Withdrawal Message, immediate reuse of template ID within a 
>>         stream, reduced likelihood of losing Data Record, and the 
>>         Collecting Process's job is easier. 
>>     
>
> Q4) 
> This feature seems to be able to be applied to SCTP and PR-SCTP. But,
> Introduction emphasizes the usage of PR-SCTP. What is advantages of 
> PR-SCTP situation over normal SCTP ? 
> Is it to deduce the loss count related to Template ID ?
>   
This draft only applies to PR-SCTP, because SCTP (without PR), will try 
to retransmit until it can't anymore and then the SCTP association will 
be shutdown. I tried to confirm that from RFC4960 but I couldn't find 
the appropriate text.
Furthermore, both SCTP and PR-SCTP are MUST in RFC5101

I propose to improve the abstract, to avoid any confusion
OLD:

This document specifies an improvement to the export specified in the 
IPFIX specifications in RFC5101.
This method offers several advantages such as the ability to calculate 
Data Record losses for PR-SCTP, immediate export of Template Withdrawal 
Messages, immediate reuse of Template IDs within an SCTP stream, and 
reduced demands on the Collecting Process.

NEW:

This document specifies an improvement to the _PR-SCTP_ export specified 
in the IPFIX specifications in RFC5101.
This method offers several advantages such as the ability to calculate 
Data Record losses for PR-SCTP, immediate export of Template Withdrawal 
Messages, immediate reuse of Template IDs within an SCTP stream, and 
reduced demands on the Collecting Process.

>>      4. Specifications 
>>
>>      4.1. Template Management 
>>
>>         This section introduces modifications compared to the Template 
>>         Management section 8 in [RFC5101]. 
>>         As specified in [RFC5101], Template Sets and Options Template 
>>         Sets MUST be sent reliably.  In other words, any IPFIX Message 
>>         containing an (Options) Template Set MUST be sent reliably.   
>>          
>>         Any Data Sets associated with a Template Record MUST be sent on 
>>         the same stream on which the Template Record was sent. 
>>          
>>         The Exporter SHOULD send a single Template and associated Data 
>>         Sets within a single stream in order to calculate the potential 
>>         Data Record loss for this Template ID.  However, the Exporter 
>>         MAY group related Templates and their associated Data Sets 
>>         within a single strsing a TLV (type, 
>>         length, value) format.  The information is exported using a 
>>         Template Record that is sent once to export the {type, 
>>         length} pairs that define the data format for the Information 
>>         Elements in a Flow.  The Data Records specify values for each 
>>         Flow. 
>>          
>>         The IPFIX protocol [RFC5101] is flexible: it foresees the usage 
>>         of the multiple SCTP streams per association; it allows the 
>>         transmission of Data Sets, Template Sets, and/or Options 
>>         Template Sets on any stream; it offers the full or partial 
>>         reliability export of Data Sets; it proposes the ordered or out-
>>         of-order delivery of Data Sets.  However, due to bandwidth 
>>         restrictions and packet losses in the network as well as 
>>         resource constraints on the Exporter and Collector (e.g., 
>>         limited buffer sizes), it is not always possible to export all 
>>         Data Sets in a reliable way. 
>>          
>>          
>>         Without delving into the details of the specifications described 
>>         later on in this document, the basic idea is to export the 
>>         Template Record and its associated Data Sets into a single 
>>         unique SCTP stream, ideally to limit the Template ID to a single 
>>         stream, while imposing in-order transmission. 
>>          
>>         The specification in this document offers several advantages 
>>         such as: calculation of Data Record losses in case of partially-
>>         reliable SCTP export, immediate export of the Template 
>>         Withdrawal Message, immediate reuse of template ID within a 
>>         stream, reduced likelihood of losing Data Record, and the 
>>         Collecting Process's job is easier. 
>>     
>
> Q4) 
> This feature seems to be able to be applied to SCTP and PR-SCTP. But,
> Introduction emphasizes the usage of PR-SCTP. What is advantages of 
> PR-SCTP situation over normal SCTP ? 
> Is it to deduce the loss count related to Template ID ?
>   
This draft only applies to PR-SCTP, because SCTP (without PR), will try 
to retransmit until it can't anymore and then the SCTP association will 
be shutdown. I tried to confirm that from RFC4960 but I couldn't find 
the appropriate text.
Furthermore, both SCTP and PR-SCTP are MUST in RFC5101

I propose to improve the abstract, to avoid any confusion
OLD:

This document specifies an improvement to the export specified in the 
IPFIX specifications in RFC5101.
This method offers several advantages such as the ability to calculate 
Data Record losses for PR-SCTP, immediate export of Template Withdrawal 
Messages, immediate reuse of Template IDs within an SCTP stream, and 
reduced demands on the Collecting Process.

NEW:

This document specifies an improvement to the _PR-SCTP_ export specified 
in the IPFIX specifications in RFC5101.
This method offers several advantages such as the ability to calculate 
Data Record losses for PR-SCTP, immediate export of Template Withdrawal 
Messages, immediate reuse of Template IDs within an SCTP stream, and 
reduced demands on the Collecting Process.

>>      4. Specifications 
>>
>>      4.1. Template Management 
>>
>>         This section introduces modifications compared to the Template 
>>         Management section 8 in [RFC5101]. 
>>         As specified in [RFC5101], Template Sets and Options Template 
>>         Sets MUST be sent reliably.  In other words, any IPFIX Message 
>>         containing an (Options) Template Set MUST be sent reliably.   
>>          
>>         Any Data Sets associated with a Template Record MUST be sent on 
>>         the same stream on which the Template Record was sent. 
>>          
>>         The Exporter SHOULD send a single Template and associated Data 
>>         Sets within a single stream in order to calculate the potential 
>>         Data Record loss for this Template ID.  However, the Exporter 
>>         MAY group related Templates and their associated Data Sets 
>>         within a single seam so loss statistics are calculated for the 
>>         group.  This may be suitable in cases where there is 
>>         insufficient SCTP streams to send each Template on its own 
>>         stream and/or the case where there are slight variations on a 
>>         single Template to show that some fields were unavailable at the 
>>         time of monitoring. 
>>       
>>         If a SCTP stream contains a mixture of Data Records defined by a 
>>         Template Record and Options Template Record(s), the Data Records 
>>         defined by the Options Template Record(s) SHOULD be sent 
>>         reliably within the same stream so that the Collector does not 
>>         consider any loss to be associated with the Options Data.  
>>         Indeed, if the Collector does not have the guarantee that the 
>>     
>
> Q5) 
> It seems Exporter, not Collector. Is it correct?
>
>   
The "Collector" was correct, as the loss is calculated from the Collector
>>         Data Records defined by the Options Template Record are sent 
>>         reliably, the Collector can not determine whether the loss in 
>>         that stream belongs to the Data Records defined by the Template 
>>         Record, defined by the Option Template Record, or by both of 
>>         them.  By sending the Options Data reliably (which is usually 
>>         required to interpret the Data Records correctly), any loss will 
>>         be limited to the non-option Data Record and loss can still be 
>>         calculated on a per Template basis. 
>>          
>>         For each (Options) Template Record, the Exporting Process MUST 
>>         send the Data Record Reliability Option Template using an Option 
>>         Template with the following Information Elements:  
>>           
>>          SCOPE:     Template ID  
>>          NOT-SCOPE: dataRecordsReliability 
>>          
>>         The Data Record Reliability Option Template MUST be sent 
>>         reliably. 
>>          
>>     
>
> Q6)
> I cannot clearly understand  how to use this field
> "dataRecordsReliability".
> "Data Record Reliability Option Template" might be applied to Option
> Template only? 
No, to both Template and Options Template.
> How is it handled by Collector for consecutive Data
> Record after receiving the option template?
> If Collector does not receive this template, what's going to happen?
>   
If the Collector doesn't receive it, it means that the Exporters don't 
support the specifications in this document and then there is no 
problem. ;-)
If the Exporters do support these specifications, then Data Record 
Reliability Option Template MUST be exported reliably, so the Collector 
will receive it.

We have improved the text and completed the examples.

>   
>>         The Option Data Record SHOULD be sent before the Data Record 
>>         that needs it so that it arrives first and is available for the 
>>         Collector to use. 
>>       
>>          
>>    
>>    
>>          
>>      4.5. The Collecting Process's Side 
>>
>>         This section introduces modifications to the Collection Process 
>>         as compared to section 9 in [RFC5101]. 
>>          
>>
>>         The Collecting Process SHOULD listen for a new association 
>>         request from the Exporting Process.  The Exporting Process will 
>>         request a number of streams to use for export: the number of 
>>         streams SHOULD be equivalent to the number of simultaneous 
>>         Template Records used in the association. Note that the Template 
>>         Records don't include the Options Template Records. 
>>          
>>         A Collecting Process SHOULD support the procedure for the 
>>         addition of an SCTP stream [SCTP-RESET]. 
>>                        
>>         The IPFIX protocol has a Sequence Number field in the IPFIX 
>>         Message header that increases with the number of IPFIX Data 
>>         Records in the IPFIX Message.  A Collector may detect out-of-
>>         sequence, dropped, or duplicate IPFIX Messages by tracking the 
>>         Sequence Number.  As thtream so loss statistics are calculated for the 
>>         group.  This may be suitable in cases where there is 
>>         insufficient SCTP streams to send each Template on its own 
>>         stream and/or the case where there are slight variations on a 
>>         single Template to show that some fields were unavailable at the 
>>         time of monitoring. 
>>       
>>         If a SCTP stream contains a mixture of Data Records defined by a 
>>         Template Record and Options Template Record(s), the Data Records 
>>         defined by the Options Template Record(s) SHOULD be sent 
>>         reliably within the same stream so that the Collector does not 
>>         consider any loss to be associated with the Options Data.  
>>         Indeed, if the Collector does not have the guarantee that the 
>>     
>
> Q5) 
> It seems Exporter, not Collector. Is it correct?
>
>   
The "Collector" was correct, as the loss is calculated from the Collector
>>         Data Records defined by the Options Template Record are sent 
>>         reliably, the Collector can not determine whether the loss in 
>>         that stream belongs to the Data Records defined by the Template 
>>         Record, defined by the Option Template Record, or by both of 
>>         them.  By sending the Options Data reliably (which is usually 
>>         required to interpret the Data Records correctly), any loss will 
>>         be limited to the non-option Data Record and loss can still be 
>>         calculated on a per Template basis. 
>>          
>>         For each (Options) Template Record, the Exporting Process MUST 
>>         send the Data Record Reliability Option Template using an Option 
>>         Template with the following Information Elements:  
>>           
>>          SCOPE:     Template ID  
>>          NOT-SCOPE: dataRecordsReliability 
>>          
>>         The Data Record Reliability Option Template MUST be sent 
>>         reliably. 
>>          
>>     
>
> Q6)
> I cannot clearly understand  how to use this field
> "dataRecordsReliability".
> "Data Record Reliability Option Template" might be applied to Option
> Template only? 
No, to both Template and Options Template.
> How is it handled by Collector for consecutive Data
> Record after receiving the option template?
> If Collector does not receive this template, what's going to happen?
>   
If the Collector doesn't receive it, it means that the Exporters don't 
support the specifications in this document and then there is no 
problem. ;-)
If the Exporters do support these specifications, then Data Record 
Reliability Option Template MUST be exported reliably, so the Collector 
will receive it.

We have improved the text and completed the examples.

>   
>>         The Option Data Record SHOULD be sent before the Data Record 
>>         that needs it so that it arrives first and is available for the 
>>         Collector to use. 
>>       
>>          
>>    
>>    
>>          
>>      4.5. The Collecting Process's Side 
>>
>>         This section introduces modifications to the Collection Process 
>>         as compared to section 9 in [RFC5101]. 
>>          
>>
>>         The Collecting Process SHOULD listen for a new association 
>>         request from the Exporting Process.  The Exporting Process will 
>>         request a number of streams to use for export: the number of 
>>         streams SHOULD be equivalent to the number of simultaneous 
>>         Template Records used in the association. Note that the Template 
>>         Records don't include the Options Template Records. 
>>          
>>         A Collecting Process SHOULD support the procedure for the 
>>         addition of an SCTP stream [SCTP-RESET]. 
>>                        
>>         The IPFIX protocol has a Sequence Number field in the IPFIX 
>>         Message header that increases with the number of IPFIX Data 
>>         Records in the IPFIX Message.  A Collector may detect out-of-
>>         sequence, dropped, or duplicate IPFIX Messages by tracking the 
>>         Sequence Number.  As is Sequence Number is per SCTP stream, 
>>         the loss for the Data Records sent in that stream can be 
>>         calculated in case of partially-reliable export. 
>>          
>>         If the Collecting Process receives a Template Withdrawal Message 
>>         on a different stream than the one on which the Template ID is 
>>         used, then the Collecting Process MUST reset the association and 
>>         SHOULD log an error message. 
>>     
>
> Q7)
> I think it is difficult to distinguish the session to use the export 
> per stream function. Because  RFC5101 said the Template Withdrawal
> Message may be sent on any SCTP stream.
>   
You're fully right.
We covered that one based on Brian's feedback.
New text:

    If the Collecting Process receives a Template Withdrawal Message on
    a different stream than the one on which the Template ID is used,
    then the Collecting Process SHOULD log an error message.

>   
>>          
>>         The following sentences from [RFC5101] are not applicable in 
>>         this specification: 
>>          
>>            "The Collecting Process normally receives Template Records 
>>            from the Exporting Process before receiving Data Records.  
>>            The Data Records are then decoded and stored by the 
>>            Collector.  If the Template Records have not been received at 
>>            the time Data Records are received, the Collecting Process 
>>            MAY store the Data Records for a short period of time and 
>>            decode them after the Template Records are received."  
>>       
>>          
>>      5. Examples 
>>     
>
>
> Q8)
> I hope that the examples include "Data Record Reliability Option
> Template".
>   
Done.

>   
>>         Figure 1 shows an example where the stream 10 carries a Template 
>>         with the Template ID 256 transmitted with full reliability (FR), 
>>         together with associated Data Records transmitted with partial 
>>         reliability (PR).  Note that, because all IPFIX Messages are 
>>         sent in order within a stream, the Template 256 will always be 
>>         processed before the Data Records by the Collecting Process. 
>>         Therefore, the Collecting Process job is simplified. 
>>         Furthermore, the Data Record loss for the Template 256 can 
>>         easily be calculated on the Collecting Process. 
>>
>>          
>>                       +--------+       +---------+   +----------+   
>>                       |        |       |         |   |          |      
>>         stream 10 ----| Data   | . . . |  Data   |---| Template |---> 
>>                       |   256  |       |    256  |   |     256  | 
>>                       |      PR|       |       PR|   |        FR|       
>>                       +--------+       +---------+   +----------+   
>>                                      Figure 1     
>>          
>>         If an Option Template is necessary to understand the content of 
>>         a Data Record (i.e. the scope in the Options Template Record is 
>>         an Information Element contained in the Data Record), the 
>>         Options Template Record may be sent in the same stream, as 
>>         displayed in figure 2. 
>>          
>>                          +--------+   +--------+     +----------+     
>>                          |        |   |        |     |          |     
>>         stream 20 ... ---| Data   |...| Data   |-----| Template |--- 
>>                          |   258  |   |   258  |     |     258  |    
>>                          |      PR|   |      PR|     |        FR|    
>>                          +--------+   +--------+     +----------+   
>>          
>>                                 +--------+       +----------+       
>>                                 |        |       | Options  |       
>>                           ...---| Data   |-------| Template |------> 
>>                                 |   257  |       |     257  |       
>>                                 |      FR|       |        FR|       
>>                            this Sequence Number is per SCTP stream, 
>>         the loss for the Data Records sent in that stream can be 
>>         calculated in case of partially-reliable export. 
>>          
>>         If the Collecting Process receives a Template Withdrawal Message 
>>         on a different stream than the one on which the Template ID is 
>>         used, then the Collecting Process MUST reset the association and 
>>         SHOULD log an error message. 
>>     
>
> Q7)
> I think it is difficult to distinguish the session to use the export 
> per stream function. Because  RFC5101 said the Template Withdrawal
> Message may be sent on any SCTP stream.
>   
You're fully right.
We covered that one based on Brian's feedback.
New text:

    If the Collecting Process receives a Template Withdrawal Message on
    a different stream than the one on which the Template ID is used,
    then the Collecting Process SHOULD log an error message.

>   
>>          
>>         The following sentences from [RFC5101] are not applicable in 
>>         this specification: 
>>          
>>            "The Collecting Process normally receives Template Records 
>>            from the Exporting Process before receiving Data Records.  
>>            The Data Records are then decoded and stored by the 
>>            Collector.  If the Template Records have not been received at 
>>            the time Data Records are received, the Collecting Process 
>>            MAY store the Data Records for a short period of time and 
>>            decode them after the Template Records are received."  
>>       
>>          
>>      5. Examples 
>>     
>
>
> Q8)
> I hope that the examples include "Data Record Reliability Option
> Template".
>   
Done.

>   
>>         Figure 1 shows an example where the stream 10 carries a Template 
>>         with the Template ID 256 transmitted with full reliability (FR), 
>>         together with associated Data Records transmitted with partial 
>>         reliability (PR).  Note that, because all IPFIX Messages are 
>>         sent in order within a stream, the Template 256 will always be 
>>         processed before the Data Records by the Collecting Process. 
>>         Therefore, the Collecting Process job is simplified. 
>>         Furthermore, the Data Record loss for the Template 256 can 
>>         easily be calculated on the Collecting Process. 
>>
>>          
>>                       +--------+       +---------+   +----------+   
>>                       |        |       |         |   |          |      
>>         stream 10 ----| Data   | . . . |  Data   |---| Template |---> 
>>                       |   256  |       |    256  |   |     256  | 
>>                       |      PR|       |       PR|   |        FR|       
>>                       +--------+       +---------+   +----------+   
>>                                      Figure 1     
>>          
>>         If an Option Template is necessary to understand the content of 
>>         a Data Record (i.e. the scope in the Options Template Record is 
>>         an Information Element contained in the Data Record), the 
>>         Options Template Record may be sent in the same stream, as 
>>         displayed in figure 2. 
>>          
>>                          +--------+   +--------+     +----------+     
>>                          |        |   |        |     |          |     
>>         stream 20 ... ---| Data   |...| Data   |-----| Template |--- 
>>                          |   258  |   |   258  |     |     258  |    
>>                          |      PR|   |      PR|     |        FR|    
>>                          +--------+   +--------+     +----------+   
>>          
>>                                 +--------+       +----------+       
>>                                 |        |       | Options  |       
>>                           ...---| Data   |-------| Template |------> 
>>                                 |   257  |       |     257  |       
>>                                 |      FR|       |        FR|       
>>                               +--------+       +----------+  
>>                                      Figure 2               
>>          
>>         Figure 2 shows an example where stream 20 carries an Options 
>>         Template with Template ID 257 transmitted with full reliability 
>>         (FR), an associated Data Record transmitted with full 
>>         reliability (FR), a Template with Template ID 258 transmitted 
>>         with full reliability (FR), and associated Data Records 
>>         transmitted with partial reliability (PR).  In this example the 
>>         Option Template Record contains information required to decode 
>>         the latter Data Records, such as Common Properties information 
>>         [IPFIX-RED-RED].  So it makes sense to export the Data Sets 257 
>>         reliably.  If some Data Record loss is observed from the 
>>         Sequence Number , the loss can only stem from the Data Sets with 
>>         the Template ID 258, as these are the only Sets not exported 
>>         reliably.  Therefore, the calculation of loss per Template ID 
>>         258 is possible.  
>>
>>          
>>         Note that, because all IPFIX Messages must be sent in order 
>>         within a stream, the Options Template 257 will always arrive 
>>         before its associated Data Records, and that the Template 259 
>>     
>
> Q9)
> Is it correct, 259 or 258? 
>   
258. Good catch.

A new draft version will be posted very soon.
Thanks for your feedback.

Regards, Benoit.
_______________________________________________
IPFIX mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ipfix

---

From: Atsushi Kobayashi <[email-masked]>
Date: Mon, 27 Oct 2008 01:12:23 +0900

Hi Benoit,

Please see in-line.

On Thu, 23 Oct 2008 13:24:03 +0200
Benoit Claise <[email-masked]> wrote:

> related Templates and their corresponding Data Sets within a single 
> Templates in a group, e.g. the omiFrom [email-masked]  Sun Oct 26 09:17:28 2008
Date: Mon, 27 Oct 2008 01:12:23 +0900
From: Atsushi Kobayashi <[email-masked]>
To: Benoit Claise <[email-masked]>
Cc: IETF IPFIX Working Group <[email-masked]>
Subject: Re: [IPFIX] WG last call on
	draft-ietf-ipfix-export-per-sctp-stream-00
Sender: [email-masked]

Hi Benoit,

Please see in-line.

On Thu, 23 Oct 2008 13:24:03 +0200
Benoit Claise <[email-masked]> wrote:

> related Templates and their corresponding Data Sets within a single 
> Templates in a group, e.g. the ossion of unavailable fields for export 

I can image following case.
In general purposes, IPv4/IPv6 templates and data captured by same
metering process and same sampling rate are exported on same sctp stream.
On the other hand, specified application data and template which contain
additional fields, for different purpose e.g. QoS measurement, are
exported on different sctp stream. 

Templates and data will be grouped according to the purposes or scope of
measurement which depend on loss sensitivity. 

>     support enough streams to export each Template in its own stream."

Actually, I meant that statistics reports mentioned on section 4 in
RFC5101 are not based on per template.
In order to count total loss from observation point to collecting process, 
some statistics templates in RFC5101 seem to be improved.

But, it seems be out of scope of this document..

>     Withdrawal Message for the Template.  [RFC5101] specifies a default

Fine.

> >mission of unavailable fields for export 

I can image following case.
In general purposes, IPv4/IPv6 templates and data captured by same
metering process and same sampling rate are exported on same sctp stream.
On the other hand, specified application data and template which contain
additional fields, for different purpose e.g. QoS measurement, are
exported on different sctp stream. 

Templates and data will be grouped according to the purposes or scope of
measurement which depend on loss sensitivity. 

>     support enough streams to export each Template in its own stream."

Actually, I meant that statistics reports mentioned on section 4 in
RFC5101 are not based on per template.
In order to count total loss from observation point to collecting process, 
some statistics templates in RFC5101 seem to be improved.

But, it seems be out of scope of this document..

>     Withdrawal Message for the Template.  [RFC5101] specifies a default

Fine.

> >>         measurements for Flows are exported using a TLV (type, 

Fine.

> >>         measurements for Flows are exported using a TLV (type, 

Fine.

> >>         within a single stream so loss statistics are calculated for the 

Reading more, I understood.

> No, to both Template and Options Template.

OK. I understood that "Data Record Reliability Option Template" can be
applied to both templates, and "dataRecordsReliability" are limited to
use only in "Data Record Reliability Option Template". 

> We have improved the text and completed the examples.

I understood you mentioned. 
This indicates the notification method of Exporters capabil >>
> >>         within a single stream so loss statistics are calculated for the 

Reading more, I understood.

> No, to both Template and Options Template.

OK. I understood that "Data Record Reliability Option Template" can be
applied to both templates, and "dataRecordsReliability" are limited to
use only in "Data Record Reliability Option Template". 

> We have improved the text and completed the examples.

I understood you mentioned. 
This indicates the notification method of Exporters capabities.
This method can be more generalized for upcoming new features.

> >>         Sequence Number.  As this Sequence Number is per SCTP stream, 

Fine.

> >>          ilities.
This method can be more generalized for upcoming new features.

> >>         Sequence Number.  As this Sequence Number is per SCTP stream, 

Fine.

> Thanks for your feedback.

Thanks,
Atsushi

--- 
Atsushi KOBAYASHI  <[email-masked]>
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637

             |        |       |         |   |          |      
> Thanks for your feedback.

Thanks,
Atsushi

--- 
Atsushi KOBAYASHI  <[email-masked]>
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637

---

From: Benoit Claise <[email-masked]>
Date: Fri, 31 Oct 2008 18:41:37 +0100

Hi Kobayashi-san
>> Yes, exactly. This is a configurable choice.From [email-masked]  Fri Oct 31 10:41:44 2008
Date: Fri, 31 Oct 2008 18:41:37 +0100
From: Benoit Claise <[email-masked]>
To: Atsushi Kobayashi <[email-masked]>
Cc: IETF IPFIX Working Group <[email-masked]>
Subject: Re: [IPFIX] WG last call
	on	draft-ietf-ipfix-export-per-sctp-stream-00
Sender: [email-masked]

Hi Kobayashi-san
> measurement which depend on loss sensitivity. 
Exactly.
> But, it seems be out of scope of this document..
Actually, the loss must be deduced at the Collector.
I guess that you have in mind the Mediator that could relay the 
information to another Collector.

Regards, Benoit.

---

From: Paul Aitken <[email-masked]>
Date: Fri, 31 Oct 2008 17:55:27 +0000

Benoit,

>     Withdrawal Message for the Template.  [RFC5101] specifies a default

Please rephrase that in better English, eg:

       The time the Exporting Process MUST wait after sending the last
       Data Set describFrom [email-masked]  Fri Oct 31 10:55:54 2008
Date: Fri, 31 Oct 2008 17:55:27 +0000
From: Paul Aitken <[email-masked]>
To: Benoit Claise <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [IPFIX] WG last call on
	draft-ietf-ipfix-export-per-sctp-stream-00
Sender: [email-masked]

Benoit,

>     Withdrawal Message for the Template.  [RFC5101] specifies a default

Please rephrase that in better English, eg:

       The time the Exporting Process MUST wait after sending the last
       Data Set described by a given Template before sending a Template
       Withdrawal Message for the Template.  [RFC5101] specifies a default
       value of 5 seconds.

-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

ed by a given Template before sending a Template
       Withdrawal Message for the Template.  [RFC5101] specifies a default
       value of 5 seconds.

-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.