Subject: [IPFIX] Call for consensus: changing Intended Status of draft-ietf-ipfix-export-per-sctp-stream from Informational to Standards Track

From: "Juergen Quittek" <[email-masked]>
Date: Thu, 13 Aug 2009 09:40:38 +0200

Dear all,

draft-ietf-ipfix-export-per-sctp-stream is currently under IESG review.
Its Intended Status is 'Informational'.
 
Since it uses normative language in some paragraphs, IESG members
suggested changing the Intended Status to 'Standards Track'.

In the IPFIX WG had consensus about 'Informational' for this draft,
because it had a normative reference to a draft in the transport area
for which it was unclear if it ever would become a normative RFC.

This situation has changed. The draft has been accepted as WG document,
see http://www.ietf.org/id/draft-ietf-tsvwg-sctp-strrst-00.txt of the
TSVWG and is progressing. We can now use it as normative reference in a
'Standards Track' document. We still would have to wait until probably
early 2010 until it gets published, but this seems to be a manageable
period of time. 

Would anyone in the IPFIX WG see a problem with changing the Intended
Status of draft-ietf-ipfix-export-per-sctp-stream from 'Informational'
to 'Standards Track'?

Thanks,

    Juergen

---

From: Nevil Brownlee <[email-masked]>
Date: Wed, 19 Aug 2009 10:49:34 +1200

Hi all:

<wg co-chair hat off>

I think that Gerhard summed up the discussion on this rather nicely.

Further, it seems to me that the per-stream draft spells out a
particular way of using the transport layer (SCTP in this case).
If we want that to depend on having the SCTP Stream Reset extension
(and I believe that we do), that's a good reason for making it a
standards-track document.

In short, I support this change of status for the draft.

</wg co-chair hat off>

Cheers, Nevil

-- 
---------------------------------------------------------------------
  Nevil Brownlee                    Computer Science Department | ITS
  Phone: +64 9 373 7599 x88941             The University of Auckland
  FAX: +64 9 373 7453   Private Bag 92019, Auckland 1142, New Zealand

---

From: Gerhard Muenz <[email-masked]>
Date: Wed, 19 Aug 2009 10:40:10 +0200

Dear all,

For completeness, here is the excerpt of the e-mail Nevil seems to be
referring to. I had sent this e-mail to our WG chairs and Ron Romascanu.

<snip>

"Informational" was decided at IETF 69:
http://tools.ietf.org/wg/ipfix/minutes?item=minutes69.html
(I remember this meeting very well because there were hot discussions
about the SCTP stream issues.)

The majority of the WG believed that per-sctp-stream should be
Informational or Experimental:
"There was support for this as a WG item, with the suggestion that
it should become either an Informational of an Experimental RFC.  If
it becomes widely used, it could later become an option in a revised
Protocol RFC."
(I assume "of" => "or")

It is important to understand the context. The preliminary decision to
adopt this draft as a WG item fell together with a very late but major
change in the IPFIX protocol draft, which concerned the removal of any
SCTP stream usage restrictions. Before, the IPFIX protocol said to send
Templates on stream 0 only, which was non-sense.

This late change of the IPFIX protocol specification seemed to cover the
"normative" content of per-sctp-stream. So, per-sctp-stream appeared to
be just a specific utilization of the IPFIX protocol, which would at no
point touch the later RFC5101.

Note that at that point, dataRecordsReliability did not exist. The
authors thought that no additional Information Elements and Reports
where necessary to inform the Collector. Later, we recognized that we
could not do without a kind of signalization between Exporter and
Collector. The proposed solution is the Data Record Reliability Option
Template using the dataRecordsReliability Information Element.

Looking at the document now, I also think that Standard Track is more
appropriate because it is not just adding new Information Elements like
RFC5473. It also specifies specific behavior regarding the SCTP stream
usage.

</snip>

Regards,
Gerhard

-- 
Dipl.-Ing. Gerhard Münz
Chair for Network Architectures and Services (I8)
Technische Universität München - Department of Informatics
Boltzmannstr. 3, 85748 Garching bei München, Germany
Phone:  +49 89 289-18008       Fax: +49 89 289-18033
E-mail: [email-masked]    WWW: http://www.net.in.tum.de/~muenz

---

From: Brian Trammell <[email-masked]>
Date: Wed, 19 Aug 2009 13:55:11 +0200

Greetings, Nevil, all,

As the draft stands right now, I'm concerned that it may not be  
appropriate for the standards track.

The biggest problem is one of interoperability.

As strictly specified, SCTP per-stream (SPS)-compliant CPs will not  
properly interoperate with 5101-compliant EPs: an EP that sends  
template withdrawals, or data sets on unexpected streams will cause  
errors at the CP side. Or rather, _may_ cause errors at the CP side:  
4.5 states what to do for TW messages, but it is not clear what the CP  
should do in case of an unanticipated data set.

More troublingly, SPS-compliant EPs that use the Template Withdrawal  
mechanism specified in Section 4.4 will _probably_ interoperate with  
5101-compliant CPs in most cases, but may fail in a very difficult to  
detect way. Specifically, since 5101 section 8 paragraph 10 specifies  
that "[t]he Template ID from a withdrawn Template MUST NOT be reused  
until sufficient time has elapsed to allow for the Collecting Process  
to receive and process the Template Withdrawal Message.", 5101- 
compliant CPs are _not_ obliged to immediately process template  
withdrawals in order on the same stream. Indeed, section 4.4 paragraph  
5 _directly_ contradicts this MUST NOT, as "[t]he Template ID from a  
withdrawn Template MAY be reused on the same stream immediately after  
the Template Withdrawal Message is sent".

Problems here: 1. The EP cannot know at runtime whether the CP is SPS- 
compliant or not, and so cannot know whether or not immediate reuse is  
safe; 2. assuming immediate reuse is safe when it isn't may cause data  
to be unframed improperly, in a way not detectable until later  
analysis; and 3. it is unclear whether we can say this _at all_ in a  
Standards Track draft without updating 5101 to strike the MUST NOT.  
This is a terrible idea, as that MUST NOT is there for a reason:  
namely, to make Template Withdrawals work at all in the general case.

It is unclear how to address the fast Template reuse issue, but it  
must be addressed, even without an change of status.

In any case, since SPS and 5101 are not completely and in all  
situations interoperable, the draft needs a very clear specification  
of the handshake between the EP and CP, by which they agree they are  
operating by the SPS rules. The minimum requirements of this handshake  
would seem to be:

1. A Collecting Process maintains a SPS mode state per Transport  
Session.

2. All Transport Sessions begin with SPS mode off.

3. When a Collecting Process determines that a Transport Session is  
operating according to SPS, it sets the SPS mode to on for that  
session, and begins operating according to SPS rules (e.g., smaller  
buffers, immediate template withdrawal).

4. Any receipt of a SPS-illegal template withdrawal or data set (i.e.,  
on the wrong stream) causes the Collecting Process to log an error and  
set the SPS mode to cancelled. Cancelled SPS mode cannot be reset  
without a Transport Session reset.

5. An SPS-compatible Exporting Process MUST provide a way to disable  
SPS mode on a per-Collecting Process basis. Otherwise, an  
administrator has no way to make that Exporting Process work with 5101  
Collecting Processes that cannot handle fast Template reuse.

Complicated? Yes, very. But I can't see another way to guarantee SPS  
interoperability with 5101.

A second problem is one of applicability. Presently, the draft is not  
particularly clear that it is recommended for use only when ALL of the  
following conditions hold:

1. There is a definite mapping between applications and templates  
(multi-template export for data structure variability and export  
efficiency within the same logical application, with sharing of common  
templates across logical applications, for example, breaks the  
assumptions in perstream).

2. There are few enough applications to fit in the number of streams  
supported at each endpoint.

3. Messages are exported using SCTP partial reliability.

4. The underlying PR-SCTP implementation supports SCTP-RESET.

5. There is a requirement for per-application record loss accounting.

Therefore, as a new implementor coming upon IPFIX for the first time,  
seeing this as a Standards Track draft, I might think that I _must_  
implement it in order to be compliant, even if it has no benefits for  
my application (or actively complicates it, as with the example 1.  
above). The draft needs a very clear, _exclusive_ statement of the  
situations in which it applies, and guidance that it should not be  
applied where not applicable.

(I'm also not entirely convinced that dataRecordsReliability should be  
used as an indication that SPS is in effect, as it appears to have  
some applicability outside SPS. It's also unclear what an EP should do  
if it wants to reserve the right to send a Template unreliably in the  
future, or what a CP should to if it detects inconsistency in the use  
of the IE. But these are side issues.)

Once these issues are addressed, then perhaps it will be appropriate  
to discuss an intended status change.

Best regards,

Brian

On Aug 19, 2009, at 12:49 AM, Nevil Brownlee wrote:

>> period of time. Would anyone in the IPFIX WG see a problem with  
>> changing the Intended
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipfix

---

From: Gerhard Muenz <[email-masked]>
Date: Wed, 19 Aug 2009 14:41:14 +0200

Brian,

Thanks for your feedback.

Brian Trammell wrote:
> properly interoperate with 5101-compliant EPs: an EP that sends template
> withdrawals, or data sets on unexpected streams will cause errors at the
> CP side. Or rather, _may_ cause errors at the CP side: 4.5 states what
> to do for TW messages, but it is not clear what the CP should do in case
> of an unanticipated data set.

You are right. It should be clarified that the CP must not assume
per-stream if the EP behaves in a non-compliant way.

> until sufficient time has elapsed to allow for the Collecting Process to
> receive and process the Template Withdrawal Message.", 5101-compliant
> CPs are _not_ obliged to immediately process template withdrawals in
> order on the same stream. Indeed, section 4.4 paragraph 5 _directly_
> contradicts this MUST NOT, as "[t]he Template ID from a withdrawn
> Template MAY be reused on the same stream immediately after the Template
> Withdrawal Message is sent".

I assume that IPFIX processes messages in the order provided by the
transport layer, just like any other protocol. It is strange that you
assume that a CP would process IPFIX Messages in a different order.

RFC5101 Section 9 says that the CP must delete the Template if it
receives the withdrawal messages. It does not give room for an
unspecified additional "processing time":

   If a Collecting Process receives a Template Withdrawal Message, the
   Collecting Process MUST delete the corresponding Template Records
   associated with the specific SCTP association and specific
   Observation Domain, and stop decoding IPFIX Messages that use the
   withdrawn Templates.

> Problems here: 1. The EP cannot know at runtime whether the CP is
> SPS-compliant or not, and so cannot know whether or not immediate reuse
> is safe; 2. assuming immediate reuse is safe when it isn't may cause
> data to be unframed improperly, in a way not detectable until later
> Standards Track draft without updating 5101 to strike the MUST NOT. This
> is a terrible idea, as that MUST NOT is there for a reason: namely, to
> make Template Withdrawals work at all in the general case.
> 
> It is unclear how to address the fast Template reuse issue, but it must
> be addressed, even without an change of status.
> 
> In any case, since SPS and 5101 are not completely and in all situations
> interoperable, the draft needs a very clear specification of the
> handshake between the EP and CP, by which they agree they are operating
> by the SPS rules. The minimum requirements of this handshake would seem
> to be:
> 
> 1. A Collecting Process maintains a SPS mode state per Transport Session.
> 
> operating according to SPS, it sets the SPS mode to on for that session,
> and begins operating according to SPS rules (e.g., smaller buffers,
> immediate template withdrawal).
> 

Agreed. We should clarify this behavior in 4.5.

> 5. An SPS-compatible Exporting Process MUST provide a way to disable SPS
> mode on a per-Collecting Process basis. Otherwise, an administrator has
> no way to make that Exporting Process work with 5101 Collecting
> Processes that cannot handle fast Template reuse.

Not sure because I do not understand the interoperability problem
between per-stream-EP and 5101-CP.

> assumptions in perstream).

I do not understand this point. per-stream works regardless of any
application that uses IPFIX.
If the application does not need per-stream, it does not have to use it.

> supported at each endpoint.

That is not a problem because multiple Templates can be grouped on one
stream as specified in the draft.

> 4. The underlying PR-SCTP implementation supports SCTP-RESET.

I think, this is a MAY, not a MUST.

> implement it in order to be compliant, even if it has no benefits for my
> application (or actively complicates it, as with the example 1. above).
> The draft needs a very clear, _exclusive_ statement of the situations in
> which it applies, and guidance that it should not be applied where not
> applicable.

per-stream will be standardized as an extension of RFC5101. It does not
have to be implemented if the benefits of the extension are not needed.

> used as an indication that SPS is in effect, as it appears to have some
> applicability outside SPS. 

The definition of dataRecordsReliability does not mention any per-stream
context. However, it has a specific meaning if SPS is used. This meaning
should probably be mentioned in the definition.

> It's also unclear what an EP should do if it
> wants to reserve the right to send a Template unreliably in the future,
> or what a CP should to if it detects inconsistency in the use of the IE.

Sending Templates unreliably is against RFC5101 anyway.

Gerhard

> But these are side issues.)
> 
> Once these issues are addressed, then perhaps it will be appropriate to
> discuss an intended status change.
> 
> Best regards,
> 
> Brian
>

---

From: Brian Trammell <[email-masked]>
Date: Wed, 19 Aug 2009 15:32:02 +0200

hi, Gerhard,

Replies inline, redundancy eliminated to conserve electrons. :)

Regards,

Brian

> assume that a CP would process IPFIX Messages in a different order.

If I have a multithreaded application, with one thread receiving, one  
per stream deframing, one thread handling the control side, and  
multiple threads handling deframed data (for example), and I follow  
5101 section 8 paragraph 10, I presume explicitly that the EP will  
hold off on immediate reuse. So I don't necessarily have to lock down  
_the entire CP_ (which could be rather expensive) in order to process  
a withdrawal.

Okay, I would not implement things this way. But, as 5101 does not  
_require_ the immediate processing of templates or withdrawals in  
order, it does not suffice merely to presume an implementation would  
do so just because all the implementations I can imagine writing  
would. And there are a lot of engineers out there younger than I am  
who are a lot thread-happier. :)

>   Observation Domain, and stop decoding IPFIX Messages that use the

The delay is implicit from 5101 section 8 paragraph 10, which reads,  
in full:

     The Template ID from a withdrawn Template MUST NOT be reused
     until sufficient time has elapsed to allow for the Collecting
     Process to receive and process the Template Withdrawal Message.

Or are you arguing that 5101 sections 8 and 9 are not consistent? I  
believe they are.

> between per-stream-EP and 5101-CP.

(imagine a sequence diagram here; my ASCII art skills leave something  
to be desired. :) )

1. A SPS EP connects to a 5101 CP

2. The EP sends a template A tid 257 on stream 1

3. The EP sends a template B tid 258 on stream 2

4. The EP sends data sets simultaneously on streams 1 and 2, according  
to templates A and B respectively

5. The EP sends a Template Withdrawal for 257 followed immediately by  
a template C tid 257 on stream 1, according to SPS section 3.2.2  
paragraph 2 (and in violation of 5101 section 8 paragraph 10)

6. The EP starts sending data according to template C on stream 1

7. The CP, not expecting immediate retransmit (because 5101 says it  
doesn't have to), does... what? with the template C data:

7a. it decodes it correctly?

7b. it buffers it, waiting for the template to be processed?

7c. it decodes it according to template A?

Is case 7c always detectable? What if templates A and C have the same  
length?

> application that uses IPFIX.

Example: YAF (tools.netsa.cert.org/yaf) uses a stack of... I think 64  
templates (it's been a year since I've hacked it), depending on  
whether the flow is a uniflow or biflow, IPv4 or v6, TCP or not, has  
payload entropy calculation or not, and so on... The relationship  
among these templates may or may not be "application" related.  
Following the per-stream specification for YAF would result in a far  
more complicated implementation (not to mention requiring people to  
recompile SCTP to handle 64 streams).

There's an implicit assumption in SPS that application <-> template,  
or application <-> set of templates. It does not hold in the general  
case.

> I think, this is a MAY, not a MUST.

Hm, true.

> have to be implemented if the benefits of the extension are not  

Excellent. Make it clear in the draft. It doesn't read as such to me,  
yet.

>> (I'm also not entirely convinced that das taRecordsReliability  
>> should be
>> some

Of course. Typo. s/Template/Data Sets described by a Template/. The  
point stands...

---

From: Gerhard Muenz <[email-masked]>
Date: Wed, 19 Aug 2009 16:29:12 +0200

Hi Brian,

Brian Trammell wrote:
> per stream deframing, one thread handling the control side, and multiple
> threads handling deframed data (for example), and I follow 5101 section
> 8 paragraph 10, I presume explicitly that the EP will hold off on
> immediate reuse. So I don't necessarily have to lock down _the entire
> CP_ (which could be rather expensive) in order to process a withdrawal.

There is no problem to process incoming IPFIX Messages in multiple
threads as long as one SCTP stream is processed by a single thread.

Not sure what you mean by "control side". Do you suggest that Set ID 2
and 3 are processed by different threads than other Set IDs?
This would be non-sense because RFC5101 does not specify a time gap
between Template and associated Data Sets.

The only reason for having specified the gap between withdrawal message
and template id reuse in RFC5101 is the out-of-order problem which
occurs if the messages are sent on different streams. This problem does
not occur with per-stream any more.

> _require_ the immediate processing of templates or withdrawals in order,
> it does not suffice merely to presume an implementation would do so just
> because all the implementations I can imagine writing would. And there
> are a lot of engineers out there younger than I am who are a lot
> thread-happier. :)

It seems very strange to me that you think that we cannot expect
protocol implementations to process incoming messages in order unless
this is explicitly specified in the corresponding RFC. It would be worth
checking if this behavior has been specified for other protocols where
this is necessary.

> The delay is implicit from 5101 section 8 paragraph 10, which reads, in
> full:
> 

Still, I do not see any hint that allows the CP to process IPFIX
Messages in arbitrary order.

> (imagine a sequence diagram here; my ASCII art skills leave something to
> be desired. :) )
> 
> 5. The EP sends a Template Withdrawal for 257 followed immediately by a
> template C tid 257 on stream 1, according to SPS section 3.2.2 paragraph
> 2 (and in violation of 5101 section 8 paragraph 10)
> 
> templates (it's been a year since I've hacked it), depending on whether
> the flow is a uniflow or biflow, IPv4 or v6, TCP or not, has payload
> entropy calculation or not, and so on... The relationship among these
> templates may or may not be "application" related. Following the
> per-stream specification for YAF would result in a far more complicated
> implementation (not to mention requiring people to recompile SCTP to
> handle 64 streams).
> 
> There's an implicit assumption in SPS that application <-> template, or
> application <-> set of templates. It does not hold in the general case.

per-stream is not talking about applications, and we do not assume such
a mapping.
per-stream allows calculating loss per Template ID or group of Template
IDs. If this does not make sense for the application, then it does not
have to use per-stream.

> Excellent. Make it clear in the draft. It doesn't read as such to me, yet.

Yes, this was also discussed in the IESG evaluation.

>>> (I'm also not entirely convinced that das taRecordsReliability should be

Ah, ok. Now, I understand this point. In this case, the EP should start
a new Template Session if it does not want to use per-stream any more.

Regards,
Gerhard

---

From: Nevil Brownlee <[email-masked]>
Date: Fri, 21 Aug 2009 09:58:34 +1200

Hi Brian:

My goodness, seeing your detailed comments on this, I feel that
I rather under-estimated the difficulties involved with this.
I guess that's because I haven't tried to implement it myself,
sigh.  Still, my post prompted your elucidation, which can only
be good.  And yes, clearly these details do need to be worked
through before we can decide whether changing the draft's
intended status.

Cheers, Nevil

Brian Trammell wrote:
> applicability outside SPS. It's also unclear what an EP should do if it

---

From: Brian Trammell <[email-masked]>
Date: Fri, 21 Aug 2009 13:51:47 +0200

hi, Nevil,

Indeed... though I believe this is quite doable, as the problems  
enumerated are easily resolvable (1. addition of clearer guidelines  
for applicability, 2. a clear procedure for ensuring SCTP per-stream  
export interoperates with "vanilla" 5101 while maximizing its  
advantages, and 3. a fix for the fast template reuse contradiction),  
and we have a fair amount of time in which to get them done given the  
SCTP-RESET normative.

Cheers,

Brian

On Aug 20, 2009, at 11:58 PM, Nevil Brownlee wrote:

>> errors at the CP side. Or rather, _may_ cause errors at the CP  
>> side: 4.5 states what to do for TW messages, but it is not clear  
>> what the CP should do in case of an unanticipated data set.
>> More troublingly, SPS-compliant EPs that use the Template  
>> Withdrawal mechanism specified in Section 4.4 will _probably_  
>> interoperate with 5101-compliant CPs in most cases, but may fail in  
>> a very difficult to detect way. Specifically, since 5101 section 8  
>> paragraph 10 specifies that "[t]he Template ID from a withdrawn  
>> Template MUST NOT be reused until sufficient time has elapsed to  
>> allow for the Collecting Process to receive and process the  
>> Template Withdrawal Message.", 5101-compliant CPs are _not_ obliged  
>> to immediately process template withdrawals in order on the same  
>> stream. Indeed, section 4.4 paragraph 5 _directly_ contradicts this  
>> MUST NOT, as "[t]he Template ID from a withdrawn Template MAY be  
>> reused on the same stream immediately after the Template Withdrawal  
>> Message is sent".
>> SPS-compliant or not, and so cannot know whether or not immediate  
>> reuse is safe; 2. assuming immediate reuse is safe when it isn't  
>> may cause data to be unframed improperly, in a way not detectable  
>> until later analysis; and 3. it is unclear whether we can say this  
>> _at all_ in a Standards Track draft without updating 5101 to strike  
>> the MUST NOT. This is a terrible idea, as that MUST NOT is there  
>> for a reason: namely, to make Template Withdrawals work at all in  
>> the general case.
>> situations interoperable, the draft needs a very clear  
>> specification of the handshake between the EP and CP, by which they  
>> agree they are operating by the SPS rules. The minimum requirements  
>> of this handshake would seem to be:
>> 4. Any receipt of a SPS-illegal template withdrawal or data set  
>> (i.e., on the wrong stream) causes the Collecting Process to log an  
>> error and set the SPS mode to cancelled. Cancelled SPS mode cannot  
>> be reset without a Transport Session reset.
>> 5. An SPS-compatible Exporting Process MUST provide a way to  
>> disable SPS mode on a per-Collecting Process basis. Otherwise, an  
>> administrator has no way to make that Exporting Process work with  
>> 5101 Collecting Processes that cannot handle fast Template reuse.
>> Complicated? Yes, very. But I can't see another way to guarantee  
>> SPS interoperability with 5101.
>> A second problem is one of applicability. Presently, the draft is  
>> not particularly clear that it is recommended for use only when ALL  
>> of the following conditions hold:
>> efficiency within the same logical application, with sharing of  
>> common templates across logical applications, for example, breaks  
>> the assumptions in perstream).
>> 2. There are few enough applications to fit in the number of  
>> streams supported at each endpoint.
>> Therefore, as a new implementor coming upon IPFIX for the first  
>> time, seeing this as a Standards Track draft, I might think that I  
>> _must_ implement it in order to be compliant, even if it has no  
>> benefits for my application (or actively complicates it, as with  
>> the example 1. above). The draft needs a very clear, _exclusive_  
>> statement of the situations in which it applies, and guidance that  
>> it should not be applied where not applicable.
>> (I'm also not entirely convinced that dataRecordsReliability should  
>> be used as an indication that SPS is in effect, as it appears to  
>> have some applicability outside SPS. It's also unclear what an EP  
>> should do if it wants to reserve the right to send a Template  
>> unreliably in the future, or what a CP should to if it detects  
>> inconsistency in the use of the IE. But these are side issues.)
>> Once these issues are addressed, then perhaps it will be  
>> appropriate to discuss an intended status change.
>> Best regards,
>> Brian

---

From: Benoit Claise <[email-masked]>
Date: Fri, 21 Aug 2009 15:27:13 +0200

Dear all,

A couple of points (some of which have been discussed already):
- the per-stream functionality should be considered as an protocol 
extension, i.e. a more constraint implementation, that you might use if 
you see a benefit. This should be made clear in the draft.
So a couple of specifications will be to modify. For example: if this 
per-sctp extension is supported, then CP SHOULD treat a Template 
Withdrawal on a different stream as an error and log it as such
- The Data Record Reliability Option Template, which contains the 
following Information Elements:
    SCOPE: Template ID
    NOT-SCOPE: dataRecordsReliability
... is the condition for the CP to know that EP support the per-stream
I realize it's not clearly mentioned in the draft
- The different compliance cases and the different fall back  conditions 
from per-stream to RFC5101 will have to specified:
    EP per-stream compliant with CP per-stream compliant
    EP not per-stream compliant with CP per-stream compliant
    EP per-stream compliant with CP not per-stream compliant
- A mechanism for disabling the per-stream extension might be needed. 
Potentially with the dataRecordsReliability = FALSE. Note this is not 
described in the draft right now.
- If we go the Intended Status change, this implies another WG last 
call. Therefore, everybody's feedback will have to take into account.
Brian's concerns will obviously have to be addressed.
- Practically, when I look at 
https://datatracker.ietf.org/idtracker/draft-ietf-ipfix-export-per-sctp-stream/, 
I believe that the standard track is the way to go. However, before 
re-applying the changes (see 
http://tools.ietf.org/rfcdiff?url2=draft-ietf-ipfix-export-per-sctp-stream-02.txt), 
of putting [SCTP-RESET] as a normative reference, I would like to get a 
kind of consensus that this whole approach is the way to go.

Regards, Benoit.

>>> 4.5 states what to do for TW messages, but it is not clear what the 
>>> CP should do in case of an unanticipated data set.
>>> 5101-compliant CPs in most cases, but may fail in a very difficult 
>>> to detect way. Specifically, since 5101 section 8 paragraph 10 
>>> specifies that "[t]he Template ID from a withdrawn Template MUST NOT 
>>> be reused until sufficient time has elapsed to allow for the 
>>> Collecting Process to receive and process the Template Withdrawal 
>>> Message.", 5101-compliant CPs are _not_ obliged to immediately 
>>> process template withdrawals in order on the same stream. Indeed, 
>>> section 4.4 paragraph 5 _directly_ contradicts this MUST NOT, as 
>>> "[t]he Template ID from a withdrawn Template MAY be reused on the 
>>> same stream immediately after the Template Withdrawal Message is sent".
>>> reuse is safe; 2. assuming immediate reuse is safe when it isn't may 
>>> cause data to be unframed improperly, in a way not detectable until 
>>> later analysis; and 3. it is unclear whether we can say this _at 
>>> all_ in a Standards Track draft without updating 5101 to strike the 
>>> MUST NOT. This is a terrible idea, as that MUST NOT is there for a 
>>> reason: namely, to make Template Withdrawals work at all in the 
>>> general case.
>>> operating by the SPS rules. The minimum requirements of this 
>>> handshake would seem to be:
>>> benefits for my application (or actively complicates it, as with the 
>>> example 1. above). The draft needs a very clear, _exclusive_