Subject: [IPFIX] multiple observation domains transmitted over the same stream

From: Benoit Claise <[email-masked]>
Date: Wed, 20 Sep 2006 17:13:24 +0200

Dear all,

I received the following comment from Michael Behringer

- 10.2.4.3: "Data Sets from multiple Observation Domains MUST NOT be transmitted over the same stream; the Collecting Process SHOULD however verify that the Observation Domain ID values are the expected values."   
What is the collecting process supposed to do with other ODID values then? Drop them? --> Specify.

Here is the proposal: 

Data Sets from multiple Observation Domains MUST NOT be transmitted over 
the same stream; the Collecting Process SHOULD however verify that the 
Observation Domain ID values are the expected values and log an error 
should the occasion arise.

Any objections on this clarifying sentence?

Regards, Benoit. 

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix

---

From: Lutz Mark <[email-masked]>
Date: Thu, 21 Sep 2006 10:57:05 +0200

> Any objections on this clarifying sentence?

I don't see any motivation for the restrictions in 10.2.4.3.

Since an exporting process should be able to export data
of multiple observation domains to the same collector, it
may be advantageous to share the output streams.

What about removing the restrictions:

10.2.4.3 Stream

    An Exporting Process MUST request at least two outbound streams per
    association.  The first stream (referred to as stream zero in the
    rest of this document), is used to send the Template Set and the
    Options Template Set.  Stream zero MUST be fully reliable.  Data
    Sets MUST NOT be sent on stream zero.

    Depending on the application requirement, the Exporting Process
    selects the mode (unreliable, partially reliable, or fully reliable)
    of the stream, used to send the Data Sets.  Unreliable mode MAY be
    used where the application does not require reliable transmission
    and the use of a retransmission queue is impractical.  Only fully
    reliable bidirectional streams are available if TLS [TLS] is used,
    as required by RFC 3436 [RFC3436].

    When Data Sets are exported over a partially reliable stream, they
    SHOULD be marked for retransmission as long as there is room in the
    SCTP send queues.  However, if the queue overflows during times of
    congestion or other retransmission events, the oldest Data Record
    that has been transmitted and marked as partially reliable should be
    freed and marked to be skipped per the PR-SCTP [RFC3758]
    specification.  The freed buffer space should then be re-used for
    the new Data Sets being exported.

Best regards,
Lutz

---

From: Lutz Mark <[email-masked]>
Date: Thu, 21 Sep 2006 10:57:05 +0200

10.2.4.3 Stream

Best regards,
Lutz

---

From: Brian Trammell <[email-masked]>
Date: Thu, 21 Sep 2006 10:12:29 -0400

Lutz,

I concur. I was under the impression the entire purpose of the odId  
field in the message header was to allow observation domain  
multiplexing within a session, be that a single SCTP stream, a single  
TCP connection, or a UDP "session" however it may be defined.

Note that there is no such restriction on odId multiplexing with TCP  
in section 10.4.

However, I'd add back the third paragraph of section 10.2.4.3,  
because it enforces the fact that an Observation Domain should use  
the same ID for all streams in an association, which I don't think we  
want to get rid of. Only the fourth paragraph adds the unnecessary  
restriction.

Regards,

Brian

On Sep 21, 2006, at 4:57 AM, Lutz Mark wrote:

>> - 10.2.4.3: "Data Sets from multiple Observation Domains MUST NOT  
>> be transmitted over the same stream; the Collecting Process SHOULD  
>> however verify that the Observation Domain ID values are the  
>> expected values."   What is the collecting process supposed to do  
>> with other ODID values then? Drop them? --> Specify.
>> Here is the proposal: Data Sets from multiple Observation Domains  
>> MUST NOT be transmitted over the same stream; the Collecting  
>> Process SHOULD however verify that the Observation Domain ID  
>> values are the expected values and log an error should the  
>> occasion arise.

---

From: Brian Trammell <[email-masked]>
Date: Thu, 21 Sep 2006 10:12:29 -0400

Lutz,

Regards,

Brian

---

From: Lutz Mark <[email-masked]>
Date: Fri, 22 Sep 2006 10:31:46 +0200

Brian,

> However, I'd add back the third paragraph of section 10.2.4.3, because 
> it enforces the fact that an Observation Domain should use the same ID 
> for all streams in an association, which I don't think we want to get 
> rid of. Only the fourth paragraph adds the unnecessary restriction.

you are right.

But reading the third paragraph again, it seems to me that this
information is redundant. Since the template management is done
per Observation Domain and an Observation Domain is identified
via its Observation Domain ID. This implies that there must be
only _one_ Observation Domain ID per Observation Domain.

Is this correct? Can anyone shed some light on this.

Best regards,
Lutz

---

From: Lutz Mark <[email-masked]>
Date: Fri, 22 Sep 2006 10:31:46 +0200

Brian,

you are right.

Best regards,
Lutz

---

From: Paul Aitken <[email-masked]>
Date: Fri, 22 Sep 2006 10:05:01 +0100

Lutz

> This implies that there must be only _one_ Observation Domain ID per
> Observation Domain.
> 

If the same OD is observed by multiple devices, we can't expect them all 
to agree on the same OD Id to describe it.

Inside a single device, I don't think it helps to have multiple OD Ids 
to describe the same OD. That would make data reconcilliation at the 
collector almost impossible.

-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

---

From: Paul Aitken <[email-masked]>
Date: Fri, 22 Sep 2006 10:05:01 +0100

Lutz

---

From: Benoit Claise <[email-masked]>
Date: Fri, 22 Sep 2006 13:56:54 +0200

Dear all,

Before answering, I've been waiting for others to comment on this 
restriction:

    "Data Sets from multiple Observation Domains MUST NOT be transmitted
    over the same stream; the Collecting Process SHOULD however verify
    that the Observation Domain ID values are the expected values."

Because, to be honest, I don't recall the justification of its 
existence. I searched through my archive, and this sentence is there 
since [IPFIX-PROTO] version 6.

Anyway, the following paragraph has been removed:

Regards, Benoit.

> If the same OD is observed by multiple devices, we can't expect them 
> all to agree on the same OD Id to describe it.
>

---

From: Benoit Claise <[email-masked]>
Date: Fri, 22 Sep 2006 13:56:54 +0200

Dear all,

Regards, Benoit.