Subject: Re: [IPFIX] Cascading Common Properties

From: Andrew Johnson <[email-masked]>
Date: Thu, 07 Dec 2006 19:35:58 +0000

Repeated email with correction to the example:
  ipv4DestinationAddress of 192.168.0.1 -> 192.168.0.2

Thanks to Danny Llewallyn for pointing it out.

Cheers

Andrew

-------- Original Message --------

Hi all

May I propose a relatively simple solution.  I think cascades of
identifiers is valid in IPFIX and useful.  I also think that they
are a lot simpler than people imagine.

NORMAL OPTIONS
==============
An option can be thought of as a substitution.  When a Data Record
is received containing Flow information then any identifiers in
that Record can be immediately substituted for the data associated
with it.

For example, a Data Record containing:
 commonPropertiesID = 5
 ipv4SourceAddress  = 192.168.0.1

If we have a definition of cpID 5, say ipv4DestinationAddress =
192.168.0.2) then the collector can immediately expand that into:
 ipv4DestinationAddress = 192.168.0.2
 ipv4SourceAddress      = 192.168.0.1

If cpId 5 is withdrawn or redefined at a later date then that
does NOT redefine this expanded Record.

If cpId 5 is not defined, the collector can hold off decoding
the Data Record for a short while before discarding it.

CASCADING OPTIONS
=================
If a Data Record is received which is definition then any
identifiers in that Record an be immediately substituted for
the data associated with it.  For example:
 commonProperties ID = 6   (scope)
 commonPropertiesID  = 5
 ipv4SourceAddress   = 192.168.0.1

Can be immediately expanded into:
 commonProperties ID    = 6   (scope)
 ipv4DestinationAddress = 192.168.0.2
 ipv4SourceAddress      = 192.168.0.1

If cpId 5 is withdrawn or redefined at a later date then that
does NOT redefine this expanded Record.

If cpId 5 is not defined, the collector can hold off decoding
the Data Record for a short while before discarding it.

A circular definition will sit on the waiting queue until it
is discarded.  A "collision" will immediately expand out to
contain the IE twice.  Today we can send Data Records with
the same IE twice so why not.  It should be noted that
ordering should be preserved.

Andrew

Brian Trammell wrote:
> All,
> 
> Agreed that common properties cascades 1. should be allowed, and 2. 
> addressed explicitly in the document.
> 
> To keep the discussion going, I'd like to propose solutions (or at least 
> choices of solutions) for the points Elisa raised:
> 
>>> duplication (as also written by Andrew)
>>>
>>> cpID A --> src addr A, src port A
>>> cpID B --> src addr A, cpID A
>>>
>>> collision
>>>
>>> cpID A --> src addr A, src port A
>>> cpID A --> src addr B, cpID A
> 
> I can think of two possible simple ways to address these concerns:
> 
> 1. The principles in 3.1.2 Multiple Data Reduction could be readily 
> extended to handle cascades: simply disallow any sort of collision, of 
> which duplication is merely a subset.
> 
> 2. Allow collision or duplication in the cascade, but ONLY to allow a 
> common property "lower" in the hierarchy to override values of one 
> "higher". This makes cascades potentially much more useful (giving us 
> workable inheritance) at the expense of a fair amount of complexity.
> 
> I don't personally have a strong opinion in either direction.
> 
>>> withdrawal
>>>
>>> what happens to a node in the tree if one of its children (or a node 
>>> in its subtree) is a cpID and is withdrawn? Does the collecting 
>>> process need to search for all the CP that are defined in terms of it 
>>> and invalidate them too?
> 
> Yes... it seems to me the only real difficulty with cascading common 
> properties (indeed, with cascading _anything_ really) is handling 
> withdrawal. We could of course allow any of the various policies for 
> cascading withdrawal and even provide a mechanism for setting the common 
> properties withdrawal cascade policy via options (and even allow those 
> options to be bound to common properties themselves, if we were feeling 
> particularly uncharitable toward CP implementors :) ) but I don't think 
> we want to do that for the sake of simplicity. So I think we want to 
> pick a single withdrawal cascade policy.
> 
> There is a question to answer explicitly before doing so: When a common 
> property is withdrawn or invalid, is the record referencing it 1. 
> invalid (as is implied by the document currently, and by most of the 
> messages in these threads) or 2. valid but simply missing data (as has 
> been proposed at least once, I forget by whom, but not particularly 
> deeply discussed yet). Note that if inheritance and overrides are 
> available in cascades as above, this second case is _not_ a valid choice.
> 
> In the first case, I would propose that 1. an Exporting Process SHOULD 
> withdraw all common properties "lower" in the cascade before withdrawing 
> the common properties they reference; 2. a Collecting Process MUST 
> invalidate all common properties defined in terms of a withdrawn common 
> property; and 3. a Collecting Process MUST ignore common properties 
> withdrawals sent by an Exporting Process that would withdraw common 
> properties previously invalidated by a cascade withdrawal as in 2. This 
> proposal is not particularly simple, but I think it is as simple as it 
> can be while holding to the principle of being liberal in what you 
> accept and conservative in what you send, and to the IPFIX design 
> requirement to minimize complexity at the EP.
> 
> In the second case, cascade withdrawal becomes simpler: simply ignore 
> the cascade. The Exporting Process MUST explicitly withdraw any common 
> property that is no longer valid, regardless of whether it's defined in 
> terms of others. Partially withdrawn cascades would then merely be 
> partially valid.
> 
> (For what it's worth, I personally lean toward the first case here.)
> 
> Thoughts?
> 
> Regards,
> 
> Brian
> 
> 
> _______________________________________________
> IPFIX mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ipfix