Subject: [IPFIX] Re: [ipfix-protocol] IPFIX-INFO last call comment: Observation Domains, Templates, and Sessions

From: Benoit Claise <[email-masked]>
Date: Fri, 22 Sep 2006 18:35:05 +0200

Brian,

Sorry it took so long to reply.
Note: I understand that the definitions in [IPFIX-INFO] version 12 are 
not in line with [IPFIX-PROTO] version 22, and I will base this 
discussion on the definitions of [IPFIX-PROTO] version 22.

I'm not sure what you are proposing is actually such a good idea at this 
stage of the protocol review (IESG review), even if your model offers 
some more flexibility. Let me explain why:

- to understand from which observation domain id a flow is exporter, you 
need to know what was the exporting process that exported the flow 
(since Observation Domain ID = A 32-bit identifier of the Observation 
Domain that is locally unique to the communication between the Exporting 
Process and the Collecting Process.). However, we don't export the 
exporting process Id in the IPFIX Message. That would require a (not so 
minor) change to the protocol

- Once we have the exporting process id associated with the flow 
records, if we want to know from which physical line card the flow was 
exported, we must have the mapping: (exporting process id, observation 
domain id) -> physical line card. This is true it could be done with the 
option template record, but then it would be a mandatory one (like in 
PSAMP): I think this is overloading the IPFIX.
However, with the definitions currently in [IPFIX-PROTO], the mapping is 
way easier: (observation domain id) -> physical line card. And if you 
chose the observation domain id to be the slot id, like it's done in the 
ENTITY-MIB, then the collector knows exactly the physical line card.

- With your solution, we should specify in [IPIFX-PROTO] how to do the 
Exporter Id management, like we do right now with the template 
management, to answer questions such as: if the exporting process 
reloads, can it take the same Id? Is UDP different in this case? etc...

- in case of backup of exporting processes, the same template Id would 
potentially not be used. So the netflow collector(s) have to reconcile 
the data based on template definitions (key/non key) as opposed to the 
template Id: this is more complex.

Note: As Andrew noted, I agree that the sentence  "Template IDs are 
unique per Exporting Process and per Observation   Domain." - IPFIX 
Protocol Specification, section 9, page 36.  is not in sync with the 
current [IPFIX-PROTO] definition and should be changed to: "Template IDs 
are unique per Observation   Domain."

With the current definitions in [IPFIX-PROTO], as the template Ids are 
unique per observation domain, different independent exporting processes 
will have to make sure that they choose the same template Id for the 
same observation domain. I understand that this might be a constraint, 
but I think this is a small constraint compare to what you propose.

Regards, Benoit.

> I have the following last-call comments for the Information Model.
>
> In the -12 revision if the Information Model, we have:
>
>>> Template ID:
>>>     An identifier of a Template that is unique within the
>>>      communication between an Exporting Process and a Collecting
>>>      Process.  Typically, this Information Element is used for limiting
>>>      the scope of other Information Elements.  Note that after a re-
>>>      start of the Exporting Process Template identifiers may be re-
>>>      assigned.
>
> This scoping is still not quite right, as it limits the ability for a 
> device exporting messages from multiple Observation Domains to use 
> different template namespaces for each Domain; specifically, it 
> unnecessarily breaks conceptual compatibility with the model presently 
> used for multiple-domain export in V9.
>
> That said, templates _should_ be unique to the session (or, here, the 
> communication between EP and CP). They should just be unique to the 
> session through the odId; so odId will have to be session scoped.
>
> The following replacements for sections 5.1.8 and 5.1.9 in INFO 12 
> will make this the case:
>
> in IPFIX-INFO -12 (starting on Page 21):
> 5.1.8.  templateId
>    Description:
>       An identifier of a Template that is unique within an Observation
>       Domain, that is, scoped to the Observation Domain ID of the
>       containing IPFIX Message. Typically, this Information Element
>       is used for limiting the scope of other Information Elements.
>       Note that after a restart of the communication between the
>       Exporting Process and Collecting Process, Template identifiers
>       may be reassigned; this information element SHOULD therefore not
>       be used by the Collecting Process as a persistent identifier.
>    Abstract Data Type: unsigned16
>    Data Type Semantics: identifier
>    ElementId: 145
>    Status: current
>
> 5.1.9.  observationDomainId
>    Description:
>       An identifier of an Observation Domain that is unique within
>       the communication between the Exporting Process and Collecting
>       Process. This identifier SHOULD also be unique per Exporting
>       Process and per IPFIX Device; it is assumed that Observation
>       Domains will be assigned identifiers according to the
>       configuration of each IPFIX Device. Typically, this Information
>       Element is used for limiting the scope of other Information
>       Elements.
>    Abstract Data Type: unsigned32
>    Data Type Semantics: identifier
>    ElementId: 149
>    Status: current
>
> Since templateID and observationDomainID are information element 
> versions of protocol header fields, we'd also have to make the 
> following changes to PROTO 22 to keep the documents parallel:
>
> 3.1 Message Header Format
> <snip to middle of page 13>
>    Observation Domain ID
>            A 32-bit identifier of the Observation Domain that is
>            locally unique to the communication between the Exporting
>            Process and the Collecting Process.  The Exporting
>            Process uses the Observation Domain ID to uniquely identify
>            to the Collecting Process the Observation Domain that
>            metered the Flows.  This identifier SHOULD also be unique
>            per Exporting Process and per IPFIX Device; it is assumed that
>            Observation Domains will be assigned identifiers according
>            to the configuration of each IPFIX Device. The Observation
>            Domain ID SHOULD be 0 when no specific Observation Domain
>            ID is relevant for the entire IPFIX Message; for example,
>            when exporting the Exporting Process Statistics, or in case
>            of hierarchy of Collector when aggregated data records are
>            exported.
>
> 3.4.1 Template Record Format
> <snip to top of page 18>
>      Template ID
>          Each Template Record has a Template ID that is locally unique
>          to the Observation Domain ID of the message in which the
>          Template is exported. Template IDs 0-255 are reserved for
>          Template Sets, Options Template Sets, and other reserved Sets
>          yet to be created.  Template IDs of Data Sets are numbered
>          from 256 to 65535.  There are no constraints regarding the
>          order of Template ID allocation.
>
> Thoughts and comments?
>
> Thanks,
>
> Brian

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix

---

From: Benoit Claise <[email-masked]>
Date: Fri, 22 Sep 2006 18:35:05 +0200

Brian,

Regards, Benoit.

---

From: Andrew Johnson <[email-masked]>
Date: Fri, 22 Sep 2006 19:07:04 +0100

> minor) change to the protocol

Actually the proposal is to make the Observation Domain unique per
communication session.  A collector has session information for each
connection for TCP and SCTP explicitly.  UDP is a little less clear,
each combination of source IP address and port makes and individual
UDP session.  On a session restart the source port will usually change,
indicating a change of session, only when a source port gets reused
do difficulties in detecting a session restart arise.

I believe, at the very least, we want the template IDs unique per
session.  Since they are currently unique per Observation Domain ID,
then either the Template ID or the Observation Domain ID should be
unique per session.

regards

Andrew

---

From: Andrew Johnson <[email-masked]>
Date: Fri, 22 Sep 2006 19:07:04 +0100

regards

Andrew

---

From: Benoit Claise <[email-masked]>
Date: Mon, 25 Sep 2006 09:07:45 +0200

Andrew Johnson wrote:
>> Note: I understand that the definitions in [IPFIX-INFO] version 12 
>> are not in line with [IPFIX-PROTO] version 22, and I will base this 
>> I'm not sure what you are proposing is actually such a good idea at 
>> this stage of the protocol review (IESG review), even if your model 
>> offers some more flexibility. Let me explain why:
>>
>> - to understand from which observation domain id a flow is exporter, 
>> you need to know what was the exporting process that exported the 
>> flow (since Observation Domain ID = A 32-bit identifier of the 
>> Observation Domain that is locally unique to the communication 
>> between the Exporting Process and the Collecting Process.). However, 
>> we don't export the exporting process Id in the IPFIX Message. That 
>> would require a (not so minor) change to the protocol
>
> UDP session.  
Well, you need to impose that a unique ID such as loopback is used.
By default, UDP will take as source IP address of the IP packet ... the 
IP address of the outgoing interface.
> On a session restart the source port will usually change,
> indicating a change of session, 
How could you impose that?
> only when a source port gets reused
> session.  
Still not convinced. Sorry.
Another example: load-balancing, with a mechanism such as RserPool. It's 
so easy to have the same template Ids to reconcile the data between 
collectors

Regards, Benoit.
> Since they are currently unique per Observation Domain ID,
>> records, if we want to know from which physical line card the flow 
>> was exported, we must have the mapping: (exporting process id, 
>> observation domain id) -> physical line card. This is true it could 
>> be done with the option template record, but then it would be a 
>> mandatory one (like in PSAMP): I think this is overloading the IPFIX.
>> However, with the definitions currently in [IPFIX-PROTO], the mapping 
>> is way easier: (observation domain id) -> physical line card. And if 
>> you chose the observation domain id to be the slot id, like it's done 
>> in the ENTITY-MIB, then the collector knows exactly the physical line 
>> card.
>>
>> - With your solution, we should specify in [IPIFX-PROTO] how to do 
>> the Exporter Id management, like we do right now with the template 
>> - in case of backup of exporting processes, the same template Id 
>> would potentially not be used. So the netflow collector(s) have to 
>> reconcile the data based on template definitions (key/non key) as 
>> opposed to the template Id: this is more complex.
>>
>>
>> current [IPFIX-PROTO] definition and should be changed to: "Template 
>> IDs are unique per Observation   Domain."
>>
>> With the current definitions in [IPFIX-PROTO], as the template Ids 
>> are unique per observation domain, different independent exporting 
>> processes will have to make sure that they choose the same template 
>> Id for the same observation domain. I understand that this might be a 
>> constraint, but I think this is a small constraint compare to what 
>> you propose.
>>
>> Regards, Benoit.
>>
>>> This scoping is still not quite right, as it limits the ability for 
>>> a device exporting messages from multiple Observation Domains to use 
>>> unnecessarily breaks conceptual compatibility with the model 
>>> presently used for multiple-domain export in V9.
>>>
>>> That said, templates _should_ be unique to the session (or, here, 
>>> the communication between EP and CP). They should just be unique to 
>>> the session through the odId; so odId will have to be session scoped.
>>>

---

From: Benoit Claise <[email-masked]>
Date: Mon, 25 Sep 2006 09:07:45 +0200

Regards, Benoit.

---

From: Brian Trammell <[email-masked]>
Date: Mon, 25 Sep 2006 11:05:41 -0400

On Sep 25, 2006, at 3:07 AM, Benoit Claise wrote:

>>> Note: I understand that the definitions in [IPFIX-INFO] version  
>>> 12 are not in line with [IPFIX-PROTO] version 22, and I will base  
>>> this discussion on the definitions of [IPFIX-PROTO] version 22.
>>>
>>> I'm not sure what you are proposing is actually such a good idea  
>>> at this stage of the protocol review (IESG review), even if your  
>>> model offers some more flexibility. Let me explain why:
>>>
>>> - to understand from which observation domain id a flow is  
>>> exporter, you need to know what was the exporting process that  
>>> exported the flow (since Observation Domain ID = A 32-bit  
>>> identifier of the Observation Domain that is locally unique to  
>>> the communication between the Exporting Process and the  
>>> Collecting Process.). However, we don't export the exporting  
>>> process Id in the IPFIX Message. That would require a (not so  

I think we're talking past each other a bit here... To reiterate, you  
don't need to know the exporting process ID at all. All you need to  
do is state that the Observation Domain ID is unique per session. As  
Observation Domain ID is the root scope of all other scoped  
Information Elements and implicitly scoped protocol information  
(i.e., Template ID), this implies that we're stating that long-term  
storage of scope information is itself out of scope for the protocol  
-- we're proposing that by the protocol, observationDomainIDs can  
only be assumed to have validity during a single communication  
session. If operators would like to configure their EPs and CPs to  
use "persistent" odIds, that's an implementation level detail, and  
could be addressed as a recommendation in the guidelines.

> By default, UDP will take as source IP address of the IP packet ...  
> the IP address of the outgoing interface.

You can't. This is yet another indication that UDP as a transport is  
subtly broken in a variety of ways. I think at this point we should  
define "session" within the language already used for the UDP  
transport, and note that this may cause runtime problems, which is  
why SCTP is the REQUIRED transport.

> Another example: load-balancing, with a mechanism such as RserPool.  
> It's so easy to have the same template Ids to reconcile the data  
> between collectors

In my view, load-balancing is just a case of extending the definition  
of "session"... I've not thought out the specifics of defining a  
session in an RserPool balanced environment, but the assertions about  
odId scoping hold if you're willing to extent session to meet this  
case...

Thoughts?

Regards,

Brian

>>> records, if we want to know from which physical line card the  
>>> flow was exported, we must have the mapping: (exporting process  
>>> id, observation domain id) -> physical line card. This is true it  
>>> could be done with the option template record, but then it would  
>>> be a mandatory one (like in PSAMP): I think this is overloading  
>>> the IPFIX.
>>> However, with the definitions currently in [IPFIX-PROTO], the  
>>> mapping is way easier: (observation domain id) -> physical line  
>>> card. And if you chose the observation domain id to be the slot  
>>> id, like it's done in the ENTITY-MIB, then the collector knows  
>>> exactly the physical line card.
>>>
>>> - With your solution, we should specify in [IPIFX-PROTO] how to  
>>> do the Exporter Id management, like we do right now with the  
>>> template management, to answer questions such as: if the  
>>> exporting process reloads, can it take the same Id? Is UDP  
>>> different in this case? etc...
>>>
>>> would potentially not be used. So the netflow collector(s) have  
>>> to reconcile the data based on template definitions (key/non key)  
>>> as opposed to the template Id: this is more complex.
>>>
>>>
>>> Note: As Andrew noted, I agree that the sentence  "Template IDs  
>>> are unique per Exporting Process and per Observation   Domain." -  
>>> IPFIX Protocol Specification, section 9, page 36.  is not in sync  
>>> with the current [IPFIX-PROTO] definition and should be changed  
>>> to: "Template IDs are unique per Observation   Domain."
>>>
>>> With the current definitions in [IPFIX-PROTO], as the template  
>>> Ids are unique per observation domain, different independent  
>>> exporting processes will have to make sure that they choose the  
>>> same template Id for the same observation domain. I understand  
>>> that this might be a constraint, but I think this is a small  
>>> constraint compare to what you propose.
>>>
>>> Regards, Benoit.
>>>
>>>> This scoping is still not quite right, as it limits the ability  
>>>> for a device exporting messages from multiple Observation  
>>>> Domains to use different template namespaces for each Domain;  
>>>> specifically, it unnecessarily breaks conceptual compatibility  
>>>> with the model presently used for multiple-domain export in V9.
>>>>
>>>> That said, templates _should_ be unique to the session (or,  
>>>> here, the communication between EP and CP). They should just be  
>>>> unique to the session through the odId; so odId will have to be  
>>>> session scoped.
>>>>
>>>> The following replacements for sections 5.1.8 and 5.1.9 in INFO  
>>>> 12 will make this the case:
>>>>

---

From: Brian Trammell <[email-masked]>
Date: Mon, 25 Sep 2006 11:05:41 -0400

Thoughts?

Regards,

Brian

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 27 Sep 2006 09:41:52 +0200

Hi Brian,

--On 25.09.2006 11:05 Uhr -0400 Brian Trammell wrote:

> I think we're talking past each other a bit here... To reiterate,
> you  don't need to know the exporting process ID at all. All you
> need to  do is state that the Observation Domain ID is unique per
> session.

I'm not sure I can follow.  If it is just unique per session, then
how would I know what domain is identified.  If I get a different
ID for the same observation domain for every session that is created,
Then how would I know that the exporter is reporting from the same
session as last time?

> As  Observation Domain ID is the root scope of all other
> scoped  Information Elements and implicitly scoped protocol
> information  (i.e., Template ID), this implies that we're stating
> that long-term  storage of scope information is itself out of
> scope for the protocol  -- we're proposing that by the protocol,
> observationDomainIDs can  only be assumed to have validity during
> a single communication  session.

I also do not see a need to have template identifiers to be persistent
beyond the lifetime of a session.  Template identifiers do not serve
as flow property, but for ordering information within the protocol
operation.

In contrast, the observation domain identifier indicates where a flow
was measured. This is an important flow property.  I do not like the
idea that an exporter may change this identifier every time a new
session is created.

Why should we allow an exporting process to change this ID during its
lifetime?  I do not see any benefit, just the potential to spread
confusion.

Thanks,

    Juergen

> If operators would like to
> configure their EPs and CPs to  use "persistent" odIds, that's an
> implementation level detail, and  could be addressed as a
> recommendation in the guidelines.
>

---

From: Juergen Quittek <[email-masked]>
Date: Wed, 27 Sep 2006 09:41:52 +0200

Hi Brian,

Thanks,

    Juergen

---

From: Andrew Johnson <[email-masked]>
Date: Wed, 27 Sep 2006 12:01:55 +0100

Hi Jeurgen

> Then how would I know that the exporter is reporting from the same

If we have two software programs, from two different vendors, running
on a linux based router.  Both these programs are monitoring flows
and exporting using IPFIX.  They are exporting to the same collector
from the same Observation Domain.

As far as the collector is concerned, these are two sessions from
the same IPFIX device.  There is no way of telling different Exporting
Processes apart.  If a network problem caused both sessions to restart
simultaneously, the collector has no way to know how the two old
sessions map to the new ones.

So what is the correct scope for the Observation Domain ID?

If it is unique per IPFIX Device then the two programs in the example
should use the same ID.  If they were observing different ODs then they
would have to use different OD IDs.  For this to work we have to mandate
in the protocol either:
  a) that the Observation Domains are configurable so that the user
     can work it all out;
  b) there is some magical calculation that will result in a unique
     OdId for every combination of configuration;
  c) some interprocess protocol so that all the Exporting Processes
     on an IPFIX Device can negotiate the IDs.

I don't think any of these are acceptable.

The Observation Domain ID can't really be unique per Exporting Process,
since we don't have any way of differentiating between Exporting
Processes.

The only real alternative is that they be considered unique per session.

> as flow property, but for ordering information within the protocol

I'm glad we are agreeing on this point.  As long as Template IDs remain
unique per Observation Domain then they will be subject to the same
problems that the Observation Domain does.  That is, if the OdId is
to be shared between different Exporting Processes, then the Templates
will be common across Exporting Processes.

> idea that an exporter may change this identifier every time a new

I don't think that the Observation Domain ID is a useful property of
the flow.  What can I infer about the flow with an OdId of 3?  I can
tell it was seen on a different Observation Domain than the flow with
an OdId of 4, but so what?

I could use an option to export something useful about the OdId,
such as a mapping of OdId to Line Card slot number, or I could
put the Line Card slot number directly into the flow.  That would
provide a tangible property of the flow.  Even if the OdId changed
with every session and between every Exporting Process, that
tangible property would remain.

> lifetime?  I do not see any benefit, just the potential to spread

I think it is worth looking at the history of the Observation Domain
ID.  In NetFlow Export version 9, we have the Source ID.  In a
distributed router we have several line cards.  Each of them generates
it's own export packets with identical source and destination addresses
and ports.  The only way the collector could differentiate between
the templates sent from one Line Card and another was that the packets
had different Source IDs.

This meant that different Line Cards shared a "session" but had a
different number space for templates.  It certainly didn't constitute
a viable flow property, although numbering convention meant that the
Line Card information could be worked out via SNMP.

In IPFIX we renamed the source ID to Observation Domain ID.  It was
considered to be unique per Exporting Process.  Since we can't
differentiate between Exporting Processes we need to change the
scope to something we can use.

regards

Andrew

---

From: Andrew Johnson <[email-masked]>
Date: Wed, 27 Sep 2006 12:01:55 +0100

Hi Jeurgen

regards

Andrew

---

From: Paul Aitken <[email-masked]>
Date: Wed, 27 Sep 2006 13:51:42 +0100

Andrew,

> The only real alternative is that they be considered unique per session.

It's certainly a convincing argument.

> an OdId of 4, but so what?

No; by your arguments above you cannot, since two different sessions may 
choose to describe the same Observation Domain by different IDs - so IDs 
3 and 4 may in fact describe the same thing.

The only way you can make the inference is if each OD is described by a 
unique OD Id, which you argue against in a,b,c above.

If each OD has one (and only one) ID then it's easy for a collector to 
identify which records pertain to that OD, since they all have the same 
ID. Any records with that ID in the header pertain to that OD.

But if each session can assign some ID to each OD, then the collector 
has somehow to determine what that OD ID means before it can do 
*anything* with the data - eg, if the exported data is being 
load-balanced across two sessions, it'd be important for the collector 
to realise that both streams pertain to the same OD even though the IDs 
are different.

> tangible property would remain.

So the OD ID is a local reference to an OD, and there may be several 
different OD IDs referencing the same OD or same OD IDs (from different 
sessions) referencing different ODs. So we need to agree a mechanism for 
dereferencing the OD Id to discover the underlying OD.

Remind me, is the draft in IESG Evaluation? :-(

-- 
Paul Aitken
Cisco Systems Ltd, Edinburgh, Scotland.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 27 Sep 2006 13:51:42 +0100

Andrew,

---

From: Brian Trammell <[email-masked]>
Date: Wed, 27 Sep 2006 09:34:39 -0400

>  a) that the Observation Domains are configurable so that the user

One point to make here - I agree that mandating in the protocol that  
the odIds and their lifetime be user-configurable is not desirable.  
However, I can see certain arrangements of devices where user- 
assigned odIds would make sense (I work on one; the non-IPFIX based  
collection system we maintain here uses "probes", which are analogous  
to odId, change about as often as network hardware is reconfigured in  
the target network, and are explicitly user-configured). So, I'd  
advocate:

1. Collecting Process MUST NOT assume validity of odId across Session  
boundaries (to be precisely defined) unless explicitly so configured  
by the user; this implies
2. Collecting Process MAY be user-configurable to override 1., and
3. Exporting Process MAY be user-configurable to allow external  
assignment of odIds.

> since we don't have any way of differentiating between Exporting

Unless we stick exporting process information in the message header,  
which we don't want to do at this point, and which doesn't really  
completely solve the issue anyway (i.e., what is the timescale of  
validity of epId, then?)

> The only real alternative is that they be considered unique per  

As before, agreed...

> tangible property would remain.

I could also see the validity of _storing_ an odId-analogue at the CP  
side that is durable across reconfiguration; this, however, is  
something the CP needs to decide for itself after collection, and as  
such is out of scope for the protocol.

Cheers,

Brian

---

From: Brian Trammell <[email-masked]>
Date: Wed, 27 Sep 2006 09:34:39 -0400

As before, agreed...

Cheers,

Brian

---

From: Lutz Mark <[email-masked]>
Date: Wed, 27 Sep 2006 18:24:31 +0200

Andrew,

> sessions map to the new ones.

In your example the two programs should use different OD IDs.
Then the mapping works.

Best regards,
Lutz

---

From: Lutz Mark <[email-masked]>
Date: Wed, 27 Sep 2006 18:24:31 +0200

Andrew,

Best regards,
Lutz

---

From: Paul Aitken <[email-masked]>
Date: Wed, 27 Sep 2006 17:35:31 +0100

Lutz,

> In your example the two programs should use different OD IDs.

How do the two programs arrange to use different IDs? If it's necessary, 
for IPFIX to work, then IPFIX should state how it's done.

Also, how would the collector know whether or not they are monitoring 
the same OPs? ie, whether they are exporting redundant duplicate data 
about the same OPs, load balancing data from the same OPs, or exporting 
quite independent data about different OPs?

Somehow we have to find what the OD ID means before we can interpret the 
data.

---

From: Paul Aitken <[email-masked]>
Date: Wed, 27 Sep 2006 17:35:31 +0100

Lutz,

---

From: Lutz Mark <[email-masked]>
Date: Thu, 28 Sep 2006 11:11:23 +0200

Paul,

> for IPFIX to work, then IPFIX should state how it's done.

This has to be managed out of band by the administrator.

> quite independent data about different OPs?

I don't see that using the same OD ID would help
here without having further arrangements.

e.g. if two Exporting Processes send redundant data the
collector need a means to detect that.

> Somehow we have to find what the OD ID means before we can interpret the 

I think the IPFIX collector does not interpret the data.
The collector can only group it per source and OD ID.
The interpretation and further processing should be done
by the application.
For me this seems to be out of scope of the IPFIX protocol.

Best regards,
Lutz

---

From: Lutz Mark <[email-masked]>
Date: Thu, 28 Sep 2006 11:11:23 +0200

Paul,

Best regards,
Lutz

---

From: Paul Aitken <[email-masked]>
Date: Thu, 28 Sep 2006 13:05:36 +0100

Lutz,

>> How do the two programs arrange to use different IDs? If it's 
>> necessary, for IPFIX to work, then IPFIX should state how it's done.
> 

Then IPFIX should say that, so each implimentation provides such a 
capability rather than hard coding values or deciding for itself.

However, this is contrary to the argument made by Andrew and Brian.

>> about the same OPs, load balancing data from the same OPs, or 
>> exporting quite independent data about different OPs?
> 

If data from an OD was being load balanced across two different EPs to 
one collector, then it seems that we would want to identify that this 
data comes from the same OD so we can put it all together at the collector.

If OD IDs are assigned globally on the box, then both EPs must use the 
same OD ID to identify the OD.

But if OD IDs are assigned per EP or per session then we need a new 
mechanism for the EP to report what the OD ID means.

I'm not sure how we could tell the difference between load balancing and 
redundant (ie, duplicate) export.

>> Somehow we have to find what the OD ID means before we can interpret 
>> the data.
> 

But if the OD IDs are assigned per EP then the collector would have to 
be able to tell which EP the data came from so it can correctly group 
the data. How can the collector tell the difference between EPs? It's 
not possible.

However, if the OD IDs are assigned per device or per export session 
then we can tell where they come from from the source address or source 
port.

> The interpretation and further processing should be done

Which will need to know what the OD ID means, before it can process the 
data.

> For me this seems to be out of scope of the IPFIX protocol.

The protocol must make useful information available!

---

From: Paul Aitken <[email-masked]>
Date: Thu, 28 Sep 2006 13:05:36 +0100

Lutz,

---

From: Benoit Claise <[email-masked]>
Date: Thu, 28 Sep 2006 14:34:39 +0200

Brian,
>>>> identifier of the Observation Domain that is locally unique to the 
>>>> communication between the Exporting Process and the Collecting 
>>>> Process.). However, we don't export the exporting process Id in the 
>>>> IPFIX Message. That would require a (not so minor) change to the 
>>>> protocol
>
> don't need to know the exporting process ID at all. 
Well, when you wrote in your proposal "observationDomainId: ... This 
identifier SHOULD also be unique per Exporting Process and per IPFIX 
Device..."
I think you do.
> All you need to do is state that the Observation Domain ID is unique 
> per session. As Observation Domain ID is the root scope of all other 
> scoped Information Elements and implicitly scoped protocol information 
> -- we're proposing that by the protocol, observationDomainIDs can only 
> be assumed to have validity during a single communication session. If 
> operators would like to configure their EPs and CPs to use 
> "persistent" odIds, that's an implementation level detail, and could 
> be addressed as a recommendation in the guidelines.
>
> transport, and note that this may cause runtime problems, which is why 
> SCTP is the REQUIRED transport.
>
The reason why I was mentioning in my previous email in this thread that 
it would be a little too late for that proposal is that (on the top of 
not being fully convinced):
    - definitions had to be changed
    - your definition of session works for TCP and SCTP, not for UDP. So 
this implicitly excludes UDP
    - we should extend the definition of session to include load-balancing.
So I'm not convinced this is such a minor change for a draft in IESG 
last-call.

Regards, Benoit.
>>>> card. And if you chose the observation domain id to be the slot id, 
>>>> like it's done in the ENTITY-MIB, then the collector knows exactly 
>>>> the physical line card.
>>>>
>>>> Protocol Specification, section 9, page 36.  is not in sync with 
>>>> the current [IPFIX-PROTO] definition and should be changed to: 
>>>> "Template IDs are unique per Observation   Domain."
>>>>
>>>> Id for the same observation domain. I understand that this might be 
>>>> a constraint, but I think this is a small constraint compare to 
>>>> what you propose.
>>>>
>>>> Regards, Benoit.
>>>>
>>>>> for a device exporting messages from multiple Observation Domains 
>>>>> to use different template namespaces for each Domain; 
>>>>> the communication between EP and CP). They should just be unique 
>>>>> to the session through the odId; so odId will have to be session 
>>>>> scoped.
>>>>>

---

From: Benoit Claise <[email-masked]>
Date: Thu, 28 Sep 2006 14:34:39 +0200

Brian,

Regards, Benoit.

---

From: Lutz Mark <[email-masked]>
Date: Thu, 28 Sep 2006 18:20:56 +0200

Paul and Brian,

> capability rather than hard coding values or deciding for itself.

agreed.

> However, this is contrary to the argument made by Andrew and Brian.

If I sum up correctly, in his initial mail Brian discovered
that template IDs should not be unique per session, but be unique
within a session per OD ID. I agree completely.

I do not follow that this change implies that OD IDs
should/may/must/whatever change during session restarts.

> same OD ID to identify the OD.

load balancing is a special case where it makes sense that
two EPs use the same OD ID.

> mechanism for the EP to report what the OD ID means.

To recommend that EPs should not share OD IDs if they run
on the same device does not mean that these OD IDs
should be assigned (randomly) by the EPs.
OD ID assignment should be user configurable.
I agree with Juergen that the OD IDs should remain
unchanged during session restarts.

> Which will need to know what the OD ID means, before it can process the 

of course. See above.

Best regards,
Lutz

---

From: Lutz Mark <[email-masked]>
Date: Thu, 28 Sep 2006 18:20:56 +0200

Paul and Brian,

agreed.

of course. See above.

Best regards,
Lutz

---

From: Brian Trammell <[email-masked]>
Date: Thu, 28 Sep 2006 12:36:22 -0400

Benoit,

On Sep 28, 2006, at 8:34 AM, Benoit Claise wrote:

>> you don't need to know the exporting process ID at all.
> Well, when you wrote in your proposal "observationDomainId: ...  
> This identifier SHOULD also be unique per Exporting Process and per  
> IPFIX Device..."
> I think you do.

Sorry - imprecise use of terminology on my part. The EP SHOULD keep  
track of its active odIds, and ensure that no two odIds on the EP at  
any given time describe different domains. By "you don't need", I  
mean the CP doesn't need to know the epId; the EP involved is  
implicit within the Session.

>>
> The reason why I was mentioning in my previous email in this thread  
> that it would be a little too late for that proposal is that (on  
> the top of not being fully convinced):
>    - your definition of session works for TCP and SCTP, not for  
> UDP. So this implicitly excludes UDP
> So I'm not convinced this is such a minor change for a draft in  
> IESG last-call.

I appreciate that... However, I'm not sure what we have now is  
sufficient:

1. It does not address the fact that a CP may not always be able to  
determine an EP's identity, given SCTP multihoming and the fact that  
the epId mechanism (depending as it does on being exported in an  
IPFIX Message, which makes epId depend on odId and vice versa)  
doesn't work. This is admittedly a bit of a corner case, but I do  
believe it is important that the protocol be self-consistent on this  
point.

2. It does not address temporal uniqueness of odIds within an EP. The  
document as it is written, and the argument for EP scoping of odId,  
would seem to imply that odIds are usable _as-is_ at the CP side to  
differentiate observation domains in the data without limitation as  
to time. This would then imply that EPs must never reuse odIds, which  
seems wrong. Otherwise, we need some sort of odId advertise/withdraw  
protocol feature, which also seems wrong.

Addressing issue 2 without session scoping and explicit allowance for  
configurable odIds would be possible by forbidding the CP from doing  
anything with the odId (much as templateId is intended for the  
internal use of the protocol alone):

"The Collecting Process MUST NOT store or interpret the Observation  
Domain ID in the IPFIX Message header, or associate it with stored  
flow data or data derived from stored flow data. The Observation  
Domain ID is intended solely for the internal use of the IPFIX  
protocol, as the root scope for template management and IPFIX Options."

Not elegant, but at least it keeps people from going down the wrong  
path for too long before realizing odId management breaks their  
analysis.

With respect to your concerns with session scoping:

Yes, it does change definitions. I believe the changes are fairly  
minor in their impact on implementations, but if the changes are  
deemed too large to make at this stage, so be it.

The proposal does not specifically exclude UDP, it just recognizes  
the fact that it's difficult to define "session" within a  
connectionless protocol. However, we have the beginnings of a session  
definition already: the time windows specified by section 10.3.6 for  
template management.

The session definition must be by necessity constructed differently  
at the EP and CP side.

A UDP session at the EP side is defined by 1. the four-tuple of  
addresses and ports,  within a time window between 2. the first  
packet sent to a given CP (identified by IPv[46] address) and 3. n  
seconds after last periodic template resend, where n is the  
configured template resend period.

A UDP session at the CP side may be defined by 1. the four-tuple of  
addresses and ports,  within a time window between 2. the first  
packet received from a given EP (identified by IPv[46] address) and  
3. n seconds after last periodic template resend was received, where  
n is the configured template expiry period.

I will admit to not having a satisfactory answer as yet with respect  
to load balancing or redundant export, mainly as an artifact of not  
being deeply familiar with either.

Regards,

Brian

---

From: Brian Trammell <[email-masked]>
Date: Thu, 28 Sep 2006 12:36:22 -0400

Benoit,

Regards,

Brian

---

From: Andrew Johnson <[email-masked]>
Date: Thu, 28 Sep 2006 17:38:24 +0100

> In your example the two programs should use different OD IDs.

What mapping are you referring to?

IPFIX INFO rev 12 the OD IDs were considered unique per Exporting
Process.  That means that each Exporting Process could export
using it's own set of Observation Domain IDs and that the number
space is not common between them.

Now we don't use the OD ID as an Exporting Process Identifier.
In short, it is agreed that the EP of a connection cannot be
identified with the mandatory information.  So Brain and I are
suggesting that the OD ID is considered unique per connection
session.

This isn't really a big change, we already mandate that we resend
all the template and option data if the session restarts.  It's
already assumed that the collector shouldn't retain certain info
between sessions, it's just not explicit in the protocol.

The alternative is to scope the Observation Domain ID to be
entire IPFIX device.  This means that different Exporting
Processes will have to negotiate (or be configured) to use the
Observation Domains correctly.  Furthermore, since template IDs
and other IEs are scoped to the OD ID, then if you wanted two
Exporters to report on the same Observation Domain (with the
same ID) then you would have to negotiate (or configure) use
of Template IDs, flow IDs, common properties IDs, ...

I really don't see this as practical.

Part of the problem here is that the Observation Domain ID seems
to be seen as something useful. Imagine two flows that had
different common properties IDs.  You wouldn't expect that to
be useful information without the options to explain what that
ID means.  You can't even be assured that the two different IDs
actually map to different properties.  The same is true for the
Observation Domain IDs.

Originally the Observation Domain ID was called the Source ID.
This only changed in version 12 of the INFO draft.  It's main
purpose is and always has been (see NetFlow version 9) to be
used to differentiate between templates seen on different Line
Cards of a distributed system.

Let's examine the c7500 router from Cisco.  In this example
we have a system of three line cards and we configure it to
monitor flows that are egressing from a single interface on
one of the Line Cards.

Now the interface being monitored is present on Line Card 3,
however all the line cards and the Routing Processor could
observe and generate flows that are destined for that Interface.
That is because on a c7500 the output features are run on the
input Line Card.

This means that the collector will see flow records from up
to four different sources:

  +----------------------------------------------------------+
  | Routing Processor                                        |
  +----------------------------------------------------------+
           |             |             |            |
           |        +---------+   +---------+   +---------+
           |        |  LC 1   |   |  LC 2   |   |  LC 3   |
           |        +---------+   +---------+   +---------+
           |             |             |            |
           v             v             v            v
  +----------------------------------------------------------+
  | Collector                                                |
  +----------------------------------------------------------+

The collector will see a single UDP "session" from a single
Exporting Process, using 4 different source IDs (i.e. Observation
Domain IDs).

It has been said that you need to know what Observation Domain
a flow was observed in order to aggregate the information
together.  In the above case a single flow that has been load
balanced across interfaces on LCs 1 and 2 will generate two
flow records from different Observation Domains.  Can you
aggregate?  Of course you can!

What is important is getting the right set of flow keys.  In
this case the index of the output interface is important to
the user, so that must be included in the flow.

If the Line Card that the flow was observed on is important
then put it in the flow (element 141).

regards

Andrew

---

From: Andrew Johnson <[email-masked]>
Date: Thu, 28 Sep 2006 17:38:24 +0100

regards

Andrew

---

From: Andrew Johnson <[email-masked]>
Date: Thu, 28 Sep 2006 17:38:58 +0100

> identifier SHOULD also be unique per Exporting Process and per IPFIX 

Perhaps the proposal could be better worded, although you have misquoted
the text which originally read:
      "A 32-bit identifier of the Observation Domain that is
       locally unique to the communication between the Exporting
       Process and the Collecting Process."

The idea is that the communication is explicitly known as each
communication uses a different set of source and destination IP
addresses and ports.  For SCTP and TCP this is even better handled
with a session being explicitly opened and closed.  These sessions
represent a "communication between the EP and CP" without the need
for the collector to know which EP each one is connected to.

>    - definitions had to be changed

We cannot realistically scope things to an EP, so definitions should
change anyway.  I really think this is the smallest change.  See my
other mail to Lutz.

> this implicitly excludes UDP

This is where the "communication between..." text came from.  Even if
session restarts can't be reliably detected with UDP at the collecting
side, the Exporter knows and will resend all the templates etc.

>    - we should extend the definition of session to include load-balancing.

I don't see why that is required.

> So I'm not convinced this is such a minor change for a draft in IESG 

Well if you forget the idea that you have to identify the Exporting
Process then I think it's a smaller change than changing the scope of
the Observation Domain to be the IPFIX Device.  Especially when you
consider the implications of how this will change the scope of all
the other Information Elements that are considered unique to the
Observation Domain.

regards

Andrew

---

From: Andrew Johnson <[email-masked]>
Date: Thu, 28 Sep 2006 17:38:58 +0100

regards

Andrew