Subject: Re: [IPFIX] IPFIX observationPointId uniqueness

From: Paul Aitken <[email-masked]>
Date: Wed, 13 Mar 2013 10:26:57 +0000

Gerhard,

We are monitoring traffic in multiple places - at the ingress and egress 
interfaces, at processes, in ACLs, in policies.

Each of these has a unique 32-bit ID (ie, there may be up to N x 32-bit 
IDs).

However, these IDs are in unique number spaces. eg, the interface ID 
could happen to be the same as the process ID or the same as the policy 
ID. So it's not possible to say what the ID represents without knowing 
the type.

You would meet this issue if you monitor interfaces and also want to 
report PSAMP stats for each selection process. Somehow you need to 
report distinct IDs for the interfaces versus the processes.

Several possibilities come to mind:

1. use multiple different IEs, one per observation point type, with 
multiple templates. This is not ideal, because we have to request new 
IEs for each new observation point type, and store / export multiple 
templates. ie, it adds a layer of unnecessary complexity rather than 
rather than simplifying the export.

2. implement a conversion function, mapping the N x 32-bit IDs into the 
1 x 32-bit observationDomainID space. Obviously only a limited number of 
conversions (1/N) are possible, and there'll be a trade-off between the 
table size and lookup time.

3. associate an observationPointType with each observationPointID, and 
not require any conversion at all. Each internal ID is mapped 1:1 with 
an observationPointId, with uniqueness being determined by the 
associated observationPointType. This is the simplest and cleanest solution.

So to answer your questions directly:

- Do you really need OPID for your purpose? Have you thought about using 
IEs like ingressPhysicalInterface or ingressInterface? Or, if these IEs 
are not appropriate, about defining new IEs?

     - per (1) above, this makes for an unnecessarily complex solution.

- If you need to use OPIDs and if you want to avoid any mapping, have 
you thought about assigning the different sources (interface cards, 
VLANs) to different ODs?

     - they are all within a single OD.
     - Example use case: report packet counts as traffic progresses 
through selection and filtering within the router. The report will be 
exported in a single message with a single OD in the header.
         It's not possible to export a single cohesive report using 
multiple ODs.

P.

On 15/02/13 09:25, Gerhard Muenz wrote:
>
> Benoit, Paul,
>
> Like Benoit, I do not have a good feeling about this proposal. On the 
> other hand, I have not found any RFC that really requires the use of 
> OPIDs. Even RFC5476 leaves it open whether the OP is identified by 
> OPID or any other IE. Therefore, I have been silent until now.
>
> IPFIX-CONFIG assumes that the OPID is assigned by the IPFIX device, 
> i.e. it is not a configuration parameter. I think that we agree here.
>
> Up to now, the IPFIX RFCs do not impose any restriction on the OPID 
> except for the uniqueness per OD. This means that various 
> implementations are possible right now:
> 1) OPID is equal to an internal identifier (e.g. internal interface 
> number) or a non-IPFIX MIB index (e.g. ifIndex, entPhysicalIndex).
> 2) OPID is equal to the IPFIX-MIB index ipfixObservationPointIndex
> 3) OPID is unrelated to any other index, the IPFIX device provides the 
> internal mapping
>
> If IPFIX-MIB is supported by the device, the natural choice for me 
> would be 2). This would allow easy mapping of MIB entries and IPFIX 
> export.
>
> As Paul points out, 1) only works as long as the internal identifiers 
> are unique per OD.
>
> My questions to Paul are:
> - Do you really need OPID for your purpose? Have you thought about 
> using IEs like ingressPhysicalInterface or ingressInterface? Or, if 
> these IEs are not appropriate, about defining new IEs?
> - If you need to use OPIDs and if you want to avoid any mapping, have 
> you thought about assigning the different sources (interface cards, 
> VLANs) to different ODs?
>
> Regards,
> Gerhard
>
>
> On 15.02.2013 01:23, Benoit Claise wrote:
>> Paul,
>>
>> At first glance, it looks reasonable.
>> But what would be the consequence on the Selection Sequence Report
>> Interpretation(see http://tools.ietf.org/html/rfc5476#section-6.5.1). If
>> we use the observationPointId and it's not unique, then we need to
>> include the observationPointType?
>> Note that the sentence "It is RECOMMENDED that this identifier is also
>> unique per IPFIX Device." was inserted specifically for this.
>>
>> Regards, Benot
>>> Dear IPFIX experts,
>>>
>>> IPFIX observationPointId (#138) is defined as:
>>>
>>>          An identifier of an Observation Point that is unique per
>>>          Observation Domain.  It is RECOMMENDED that this identifier is
>>>          also unique per IPFIX Device.  Typically, this Information
>>>          Element is used for limiting the scope of other Information
>>>          Elements.
>>>
>>> We now also have observationPointType (#277), defined as:
>>>
>>>           Type of observation point. Values assigned to date are:
>>>
>>>           1. Physical port
>>>           2. Port channel
>>>           3. Vlan.
>>>
>>>
>>> Could we relax the uniqueness requirements of observationPointId when
>>> an observationPointType is also exported, such that the {
>>> observationPointType, observationPointId } pair must be unique, though
>>> not the observationPointId itself.
>>>
>>> The reason being that observation points of different types already
>>> have IDs, though these are not unique. eg, we have interface 1, 2, 3,
>>> ...; vlan 1, 2, 3, ...; port 1, 2, 3, ...
>>>
>>> So an extra mediation layer is required to convert values from each of
>>> these number spaces into unique IDs in the observationPointId number
>>> space - which becomes harder as we add more places that observations
>>> can be made, ie more observationPointTypes.
>>>
>>> Whereas prefixing with the observationPointType achieves the required
>>> uniqueness in a faster, simpler, and more reliable way.
>>>
>>> To achieve this, I'd propose the following definition change:
>>>
>>>          An identifier of an Observation Point that is unique per
>>>          Observation Domain and per observationPointType, if specified.
>>>          It is RECOMMENDED that this identifier is
>>>          also unique per IPFIX Device.  Typically, this Information
>>>          Element is used for limiting the scope of other Information
>>>          Elements.
>>>
>>> For backwards compatibility, we could define observationPointType = 0
>>> to be "unspecified".
>>>
>>> Thanks,
>>> P.
>>>
>>>
>>> _______________________________________________
>>> IPFIX mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/ipfix
>>
>>
>>
>> _______________________________________________
>> IPFIX mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/ipfix
>>

---

From: Gerhard Muenz <[email-masked]>
Date: Wed, 13 Mar 2013 22:42:57 +0100

Paul,

I always thought of OPID, ODID, MPID, EPID etc. as being abstract 
identifiers. If an implementation can map them directly to 
implementation-specific internal parameters, that's fine. If not, a 
mapping needs to be provided by the device. That's the same for other 
data models.

For example, you would also need to assign ipfixObservationPointIndex 
for each OP if you implement IPFIX-MIB. You could use the same value for 
the OPID IE.

If there is no other solution than map 32 bit directly into OPID, I 
would prefer changing OPID to unsigned64.

Regards,
Gerhard

---

From: Paul Aitken <[email-masked]>
Date: Tue, 26 Mar 2013 12:20:48 +0000

Gerhard,

My problem is mapping multiple discrete 32-bit number spaces into a 
single 32-bit Observation Point ID.

Keeping track of the mapping could be impossibly expensive: in the worst 
case, 2^32 IDs x { (8-bit source type) + (32-bit source ID) + (32-bit 
OPID) } = 36G of memory :-(

This suggests that the number space should be carved, so source 1 uses 
OP IDs 0 - 999, source 2 uses OP IDs 1,000 - 1,999 etc. However I can't 
do that because there are only 32-bits in the destination (OP ID) number 
space. Each of the source number spaces is also 32-bits; I can't know 
whether they'll be linearly allocated or sparsely populated (eg, by a 
hashing algorithm), so I can't impose that certain bits (eg the topmost 
bits) be used for the carving.

A 64-bit OP ID would really help me.

Thanks,
P.

On 13/03/13 21:42, Gerhard Muenz wrote:
> for each OP if you implement IPFIX-MIB. You could use the same value 
> for the OPID IE.
>
>>>> Interpretation(see 
>>>> http://tools.ietf.org/html/rfc5476#section-6.5.1). If

---

From: Gerhard Muenz <[email-masked]>
Date: Wed, 27 Mar 2013 21:54:03 +0100

Paul,

I do not see any problem with 64bit OP ID.

I think, we had a similar motivation for having 64bit selector 
IDs/selection sequence IDs.

Will it confuse existing collectors? (hopefully not)

Regards,
Gerhard

---

From: Paul Aitken <[email-masked]>
Date: Thu, 28 Mar 2013 09:52:07 +0000

Gerhard,

This question needs input from collector experts on this list.

P.

On 27/03/13 20:54, Gerhard Muenz wrote:
>>>> using

---

From: Andrew Feren <[email-masked]>
Date: Thu, 28 Mar 2013 08:01:02 -0400

Hi Paul, Gerhard,

I can't speak for other collectors, but I don't see a problem with
increasing the length of observationPointId.

I never saw any upside to doing anything other than respecting the length
in a template when that length in the template differs from the nominal
type size of the IE.  For reduced size encoding this is a specified
behavior disallowing increased size encoding seemed like extra work with
no return.

Also I think Gerhard made a good case for increasing the size being a
better option than the original proposal.  I didn't have an problem with
the original proposal, but after considering Gerhard's arguments I think
increasing the length of observationPointId is less likely to have any
negative consequences.

-Andrew

---

From: Paul Aitken <[email-masked]>
Date: Thu, 28 Mar 2013 12:16:58 +0000

Thanks Andrew and Gerhard.

I'll forward the request to IE-doctors and IANA.

P.