Subject: Re: [IPFIX] WG Last Call for draft-ietf-ipfix-testing-01.txt

From: kobayashi atsushi <[email-masked]>
Date: Tue, 03 Jul 2007 18:42:07 +0900

Dear all,

I had a short review about IPFIX test draft and I have several questions.
I am just trying to implement IPFIX Meidator, it is very useful
as reference document.

- 3.2.3.  Set Padding and 3.2.4.  Record Padding

The document describes the Collecting Process should shut down the SCTP
or TCP, when it recognizes the padding field is non-zero value.

Generally, Exporting Process must put the paddeing with zero. But, on
the collecting process side, it simply decodes IPFIX message, it seems
not to need to check the value of the padding.

I think that "shut down" is too severe for Collecting Process.

- 3.4.1 Using any Information Elements as Scope

The following description seems to define the handling, when the
collecting process receives  option template with Scope Field Count of
zero. In that case, should the collecting process shut down the
transport session?

   The Scope Field Count MAY NOT be zero.  The tester MUST cause the
   export of an Options Template Record containing a Scope Field Count
   of zero.

   The tester MUST ensure that the Collecting Process shuts down the
   SCTP association and discards the IPFIX Message.  The tester MUST
   check that the Collecting Process logged the error.

- 3.4.3.  Metering Process Statistics Option Template

It is not clear as following description. It seems to be condition that
several Metering Process on the same Observation Domain use the same
Exporting Process. Is it correct?

   If several Metering Processes use the same Exporting Process, the
   tester MUST create a Metering Process Statistics Option Template
   containing multiple scopes and an associated Data Record, MUST cause
   the Option Template and associated Data Record to be exported, and

--- 
Atsushi KOBAYASHI  <[email-masked]>
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637

On Wed, 27 Jun 2007 17:15:30 +1200
Nevil Brownlee <[email-masked]> wrote:

> 
> Hi all:
> 
> A revised verison of the IPFIX Testing draft has been published, i.e.
> draft-ietf-ipfix-testing-01.txt, and it's now ready for a Working Group
> last Call.
> 
> That WG LC starts now, and will finish on Saturday, 14 July 07.
> 
> Please read the draft, and send any comments to the IPFIX list.
> 
> Cheers, Nevil
> 
> -----------------------------------------------------------------------
>   Nevil Brownlee                    Computer Science Department | ITS
>   Phone: +64 9 373 7599 x88941             The University of Auckland
>   FAX: +64 9 373 7453   Private Bag 92019, Auckland 1142, New Zealand
> 
> -----------------------------------------------------------------------
> This mail sent through University of Auckland http://www.auckland.ac.nz
> 
> 
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ipfix
> 

--- 
Atsushi KOBAYASHI  <[email-masked]>
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix

---

From: kobayashi atsushi <[email-masked]>
Date: Tue, 03 Jul 2007 18:42:07 +0900

Dear all,

--- 

---

---

From: Paul Aitken <[email-masked]>
Date: Tue, 03 Jul 2007 11:20:54 +0100

Kobayashi-san,

Thanks for reviewing the draft so promptly!

> I am just trying to implement IPFIX Meidator, it is very useful

I'm glad it's helpful to you.

> I think that "shut down" is too severe for Collecting Process.

While [IPFIX-PROTO] doesn't provide a specific instruction on what to do 
in this situation, its general rule (from section 9, "The Collecting 
Process's Side") is:

    If the Collecting Process receives a malformed IPFIX Message, it
    MUST reset the SCTP association, discard the IPFIX Message, and
    SHOULD log the error.

Since [IPFIX-PROTO] section 3.3.1 "Set Format" says:

       Padding
           For
           security reasons, the padding octet(s) MUST be composed of
           zero (0) valued octets.

then non-zero padding constitutes a malformed message, and the shutdown, 
discard and log process must be followed.

> The following description seems to define the handling,

Well, it's defined in IPFIX-PROTO. In [IPFIX-TESTING] we show how to 
ensure that you're following the requirement :-)

> when the

Again, [IPFIX-PROTO] section 3.4.2.1, "Scope", says:

    Finally, note that the Scope Field Count MAY NOT be zero.

So a scope field count of zero constitutes a malformed IPFIX Message, 
and the shutdown, discard and log process must be followed.

> Exporting Process. Is it correct?

I think it's quite possible, since I don't see any exclusion which says 
each MP must use a different EP.

P.

-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

---

From: Paul Aitken <[email-masked]>
Date: Tue, 03 Jul 2007 11:20:54 +0100

Kobayashi-san,

       Padding
           For

P.

---

From: kobayashi atsushi <[email-masked]>
Date: Tue, 03 Jul 2007 20:47:33 +0900

Hi Paul,

Thank you for reply.
Please see in-line.

On Tue, 03 Jul 2007 11:20:54 +0100

> discard and log process must be followed.

At least, checking the value of "paddingOctets" seems not to be task for
the collecting process.
I think that checking the validation of each field in each Flow Record
 seems to be done by next process after decoding in the collecting process.

Even if the value of paddingOctets is not zero, I think that it is not
malformed as IPFIX message.
"paddingOctets" is one of value the Information Elements in Data Record.

>     Finally, note that the Scope Field Count MAY NOT be zero.

IPFIX-PROTOCOL draft says "MAY NOT". I understood it has a possibility
of scope field zero.

> each MP must use a different EP.

Why are multiple scopes needed in following cases?
It indicates the metering process id and observation domain id?

---

---

From: kobayashi atsushi <[email-masked]>
Date: Tue, 03 Jul 2007 20:47:33 +0900

Hi Paul,

Thank you for reply.
Please see in-line.

---

---

From: Paul Aitken <[email-masked]>
Date: Tue, 03 Jul 2007 14:39:18 +0100

Kobayashi-san,

>  seems to be done by next process after decoding in the collecting process.

For paddingOctets in a record, then yes, I could agree.

However, padding between sets will probably never be indicated to any 
other process; they're more about the structure and format of the export 
- so I believe they should be checked by the Collecting Process.

> "paddingOctets" is one of value the Information Elements in Data Record.

OK, I could agree with you. Mainly I want the point to be clear, to 
ensure that everyone implements the same functionality.

So let's see what other people have to say.

> IPFIX-PROTOCOL draft says "MAY NOT". I understood it has a possibility

I recall discussing this point with Benoit and Stewart, and I believe 
our intention was that IPFIX options should always include some scope 
information (unlike NetFlow v9) - else they are just like ordinary data 
records.

And I believe that's how most IPFIX implementors have interpreted this.

However, you're right - the text says "MAY NOT" rather than "MUST NOT". 
Unfortunately "MAY NOT" is not defined in RFC 2119.

I believe the text should have said "MUST NOT", and should be corrected.

(And also, we should be more stringent about checking RFC 2119 keywords 
in drafts!)

> Why are multiple scopes needed in following cases?

Because section 4.1 of [IPFIX-PROTO] ("The Metering Process Statistics 
Option Template") says:

    Note that if several Metering Processes are available on the
    Exporter Observation Domain, the Information Element
    meteringProcessId MUST be specified as an additional Scope Field.

Therefore we included a test that ensures the Collecting Process is able 
to decode multiple scope elements in the MPSO.

> It indicates the metering process id and observation domain id?

Yes, the meteringProcessId.

P.

---

From: Paul Aitken <[email-masked]>
Date: Tue, 03 Jul 2007 14:39:18 +0100

Kobayashi-san,

P.

---

From: kobayashi atsushi <[email-masked]>
Date: Wed, 04 Jul 2007 11:14:08 +0900

Hi Paul,

Thank you for reply.
Please see in-line.

On Tue, 03 Jul 2007 14:39:18 +0100

> - so I believe they should be checked by the Collecting Process.

I understood the Collecting Process checks whether IPFIX messages are
malformed and suspicious like padding between sets non-zero.

> (And also, we should be more stringent about checking RFC 2119 keywords 

I understood what you said. But, the Collecting Process should be allowed
to receive the scope field count of zero, as long as IPFIX PROTOCOL draft
says "MAY NOT".

We might interpret that the observation domain id in IPFIX header
implicitly indicates a scope of whole message, in that case scope field might
be considered needless.

>     meteringProcessId MUST be specified as an additional Scope Field.

I think that it is a little bit different condition.

Above mention indicates that several Metering Processes are available on
the Exporter Observation Domain. Test draft said that several Metering
Processes use the same Exporting Process.

I think that metering process id seems to be needless if several
Metering Process in different Observation Domain use same Exporting
Process. I think that it could be better adding description related to
the Observation Domain.

Thanks,
Atsushi KOBAYASHI

---

---

From: kobayashi atsushi <[email-masked]>
Date: Wed, 04 Jul 2007 11:14:08 +0900

Hi Paul,

Thank you for reply.
Please see in-line.

Thanks,
Atsushi KOBAYASHI

---

---

From: Paul Aitken <[email-masked]>
Date: Wed, 04 Jul 2007 11:00:59 +0100

Kobayashi-san,

> malformed and suspicious like padding between sets non-zero.

Good, we agree.

What about checking the value of "paddingOctets" within a Data Record?

There are at least two places which specify that paddingOctets must 
consist of zero valued octets:

[IPFIX-PROTO] 3.3.1   Set Format

       Padding
           The Exporting Process MAY insert some padding octets, so that
           the subsequent Set starts at an aligned boundary.  For
           security reasons, the padding octet(s) MUST be composed of
           zero (0) valued octets.

[IPFIX-INFO] 5.12.1. paddingOctets

    Description:
       The value of this Information Element is always a sequence of 0x00
       values.

I believe we should include a test for that. However, with only an 
Exporting Process and a Collecting Process, it may be quite hard to 
ensure that the Exporting Process is sending 0x00 octets. Whereas it's a 
lot easier to ensure that the Collecting Process resets the connection 
when non-0x00 octets are sent.

>> So let's see what other people have to say.

Nothing so far, apparently :-(

> to receive the scope field count of zero, as long as IPFIX PROTOCOL draft

Correct. However, as I already pointed out, "MAY NOT" is incorrect 
terminology (not in RFC 2119). I believe the intention was to say "MUST 
NOT".

I've already contacted the IETF tools team and the author of idnits, who 
agree this term is misleading and are now thinking how to modify the 
tool to highlight such incorrect usage in future drafts.

> implicitly indicates a scope of whole message, in that case scope field might

What is the difference between an Options Template with no scope and an 
ordinary Data Template?

How do you think it would be useful to send Options with no scope?

> Process. I think that it could be better adding description related to

OK, I'll ensure the IPIFX-TESTING draft is corrected.

Thanks.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 04 Jul 2007 11:00:59 +0100

Kobayashi-san,

Good, we agree.

       Padding

    Description:

Thanks.

---

From: kobayashi atsushi <[email-masked]>
Date: Fri, 06 Jul 2007 01:04:53 +0900

Paul-san,

On Wed, 04 Jul 2007 11:00:59 +0100

> when non-0x00 octets are sent.

I think  the test for the "paddingOctets" is need, too.

But, I think that checking value of each field in Data Record is not
task of the Collecting Process. If it is so, the Collecting Process
needs to know the attribute of all Information Elements, and check the
validation for received Data Records. It seems to be too far. 

> tool to highlight such incorrect usage in future drafts.

OK, I hope that "MAY NOT" will be changed in the near future.

> How do you think it would be useful to send Options with no scope?

I don't have strong opinion. I am afraid of misleading about IPFIX
protocol draft and IPFIX Testing draft.

---

---

From: kobayashi atsushi <[email-masked]>
Date: Fri, 06 Jul 2007 01:04:53 +0900

Paul-san,

---

---

From: Benoit Claise <[email-masked]>
Date: Mon, 23 Jul 2007 03:52:20 +0200

Paul, Kobayashi-san,
>>> While [IPFIX-PROTO] doesn't provide a specific instruction on what 
>>> to do in this situation, its general rule (from section 9, "The 
>>> Collecting Process's Side") is:
>>>
>>> then non-zero padding constitutes a malformed message, and the 
>>> shutdown, discard and log process must be followed.
>>>
>>>
>>
> other process; they're more about the structure and format of the 
> export - so I believe they should be checked by the Collecting Process.
>
>
> information (unlike NetFlow v9) - else they are just like ordinary 
> data records.
>
> However, you're right - the text says "MAY NOT" rather than "MUST 
> NOT". Unfortunately "MAY NOT" is not defined in RFC 2119.
>
I agree.

Regards, Benoit

---

From: Benoit Claise <[email-masked]>
Date: Mon, 23 Jul 2007 03:52:20 +0200

Paul, Kobayashi-san,
I agree.

Regards, Benoit