Subject: RE: ABNF and Versioning (or why Versioning isn't needed)

From: "Ryan Moats" <[email-masked]>
Date: Tue, 27 Jul 1999 10:05:35 -0500

| Please see comments inline.
|
| Yoram Snir
| Cisco Systems
| mailto:\\[email-masked]
| Tel. 972-9-9700085

[More editing to cut down B/W]

| > [Some editing to cut down B/W]
| >
| > | From: Yoram Ramberg [mailto:[email-masked]]
| > |
| > | On the switch from SDN to RDN containment:
| > |
| > | This may be tricky. See if I got this one right.
| > | Once you impose strict containment on the DIT you're also presenting a
| > | restriction on relationships between rules and conditions
relationships.
| > | What happens is that your forcing me to place all conditions related
to a
| > | given rule under this rule in the DIT. What if another rule "wants" to
use
| > | one of those conditions? It must copy that condition (if it knows how
to
| > | find it) and place the copy unde the new rule. This makes that
condition
| > | non-reusable. Two identical conditions must be maintained now and
could be
| > | modified independently.
| >
| > Yes and No (and now let me explain).  What the ABNF specifies
| > (and what is the correct way, I might add) is that
| > policyRuleConditionItem and policyRuleActionItem (two new
| > object classes) are restricted to be contained under the
| > policyRule object class.  For policyRuleConditionItem,
| > there are four attributes defined: policyRuleItemGroupNumber,
| > policyRuleItemNegated, policyRuleItemDN, and
| > policyRuleConditionItemName.  The last is a friendly name that
| > MAY be used as an RDN.  The other three are separate attributes
| > that replace the structures of the SDN as separate attributes.
| > Now, while a tuple (group number, negation, DN) [(group number, DN)
| > for policyRuleActionItem] is bound under a single policyRule in
| > the DIT, other policyRules may reuse the DN pointer to build
| > there own policy items.  While it is possible to restructure the
| > ABNF to allow for entire policyRuleConditionItem and
| > policyRuleActionItem
| > reuse, I'm not yet convinced that this is a good thing.  Reusing
| > the DN's achieves 80-90% (my estimate) of full object class reuse
| > without cluttering up the schema with extra DN pointers.
|
| Reusable objects are a very strong mechanism, not only for easier
| definition of policies but also for data integrity, encapsulation,
consistency and
| maintenance of complex policies. Reusable objects are not only conditions
| but variables  and constant values such as the value of HTTP port
| number in a specific domain.
|
| Reusability is used by DiffServ Policers to share bandwidth for example,
| meaning the packets that meet Rule1 and Rule 2 conditions share are marked
| with a certain DSCP value and policed to 64KB/s (together).
| Other examples are in available at
|
ftp://ftpeng.cisco.com/ftp/public/policy/draft-snir-policy-qos-infomodel-00.
txt

Yes, that's the draft I was thinking of in my previous comment "I'm not yet
convinced..."
and yes, that draft talks about doing full reuse of object class instances
through DN
pointers. However, DN pointers are not the only way to go, DIT aliasing is
another
concept that allows for object reuse while maintain the DIT structure [which
I'm
still thinking about].  The point is that using DNs "unless absolutely
necessary"
is adding gratuitous complexity for little or no gain.

| > | (I used conditions in my example because we deem conditions to be
prime
| > | candidate for reusability. Everyone wishing to create a reusable
object
| > | class and place it in the repository will have this problem exactly)
| > |
| > | Furthermore: SDN provide syntax for specifying the intra-expression
| > | relations, i.e., c1 AND c2 and NOT c3 or (NOT c4 abd C7). How would
you
| > | express this if you drop SDNs altogether? We, then, must add
| > | syntax for this kind of expressions. RDN doesn't do the trick
| > | but atructured RDN still may still doesn't solve reusability).
| >
| > The attribute policyRuleItemNegated handles the negation on
| > an object by object basis.
|
| But you need the ability to define a complex arbitrary Boolean expression
| (including parenthesis).

I think there is some confusion.  The original SDN for conditions had
"group:negation:DN".
All that has been done is to replace that attribute with a set of objects
that have
attributes that store "group", "negation" and "DN".  If the original SDN was
rich
enough, the object classes as currently defined are rich enough.

| > SDNs (or sRDNs) are a bad idea because they require the
| > client to process the returned string BEFORE they can
| > determine if they need to make another LDAP query to retrieve
| > the pointed to object.  This means that a client wanting to
| > make only a single LDAP query would be required to pull the ENTIRE
| > policy tree [this is a BAD thing]. Also, by separating out the DN,
| > you are setting up to take advantage of work that is going on
| > in LDAPEXT.  If you use SDNs, you are limited to "rolling your own".
|
| SDN is something that was defined in the CORE policy information model as
a
| tool to define arbitrary, complex boolean expressions. Every alternative
| that may substitude the SDN mechanism will have other complexities since
it
| must support the basic Boolean operators, so a simple lDAP query for
complex
| conditions is problematic. Simple rules can be created by attachment and
| simple LDAP queries could retrieve them in a single query.

While I normally bow to the collect wisdom of a working group, this is an
exception.  I do not see how SDNs "as they are defined" support arbitrary
complex boolean expressions in a way that using the proposed object classes
don't.  Since I have yet to see how the transformation is lossy, if you say
"object classes don't work", I say "then SDNs don't work either".

Ryan

---

From: "Yoram Ramberg" <[email-masked]>
Date: Wed, 28 Jul 1999 14:18:29 -0500

(I hope this is, indeed, a general interest subject...)

Ryan, I see your point(s) and it is a good one. We must be held accountable
to the burden our schema design puts on readers, in terms the LDAP roundtrip
count. It is a major optimization issue and we have to resolve this. I'm
glad you're pointing this out at this stage so the schema design may benefit
from this proposed improvement.

The direction toward minimizing DN dereferencing is understood. You're
pointing to a possible confusion in interpreting the expressive power of the
replacement of SDN with three attributes. I'm not confused. In terms of
being able to define an arbitrarily complex Boolean expression, I see no
difference between SDNs and the 3-tuple  (group nr., negation, DN) method.
After all, I can reconstruct all SDNs from their respective 3-tuples, no?
This is mighty fine with me.

However, I still don't understand (shame on me!) how we're going to support
reusability of, say, simple conditions. Let's assume, for the purpose of
this discussion, that reusability is a virtue and we'd like to support it
("bow to the collect wisdom of a working group..."). Please explain how DIT
aliasing allows us to support it. Can we still support the notion of global
repositories? Doesn't this defeat the cause of collecting all related object
in a single subtree so you can pick'em up in one LDAP hit? Don't we end up
paying the same price as in the case of SDNs (if our reusable objects do
reside in a global repository)?

Now, let's see if we can calculate the maximum number of LDAP roundtrips
necessary to fully fetch a complex condition, using the different methods.

SDN (call it Core method):

I think that the number of LDAP queries you'll have to execute to get the
whole
condition depends on the number of instances of simple conditions it is made
of.
It's probably something like:
nr. of simple conditions + 1 (for the Rule)
What does this mean?
A complex condition we'd like to represent is the infamous five tuple IP
header
condition. At a maximum, again, it costs five simple conditions "fetch"
operations to fully gather the darn thing. This added to the hit for the
Rule,
it takes six roundtrips to satisfy us. Remember I said "maximum" twice. I'm
not
convinced this can't be optimized somewhat...

Raju's bulky conditions:

Other methods for modeling conditions in LDAP do get a much better rating on
this. If we follow Raju's schema we find that  same condition (quite
restricted,
though) can be fetched with a single sniff. What the schema pays for that is
measured in its expressive power. Conditions are "bulky" and very
restricted.
To accommodate a slight deviation from the prescribed model you'd have to 1.
Derive your own mechanism (existing classes are too specific to derive
from), 2.
Accept the fact that your rules would be opaque to readers, other than
specially
adapted clients -- your own.

Ryan's 3-tuple method:

In this case the whole complex condition (and, in fact, all complex
conditions!) may be fetched with a single LDAP roundtrip. What's really
attractive here is that we pay no price in terms of expressive power:
Arbitrarily complex conditions can still be specified. What does this method
costs us? I'm yet to fully understand this but I have a hunch that
reusability is the victim, or at least, some reusability. I already asked
Ryan to elaborate on this.

A few words about reusability:

Note that we aren't really talking about object class reusability -- this is
*instance reusability*: The ability to use the same simple condition in
several rules. One reason we want this is pointed out in Yoram Snir's latest
entry in this thread. It's very clear that reusing named condition helps --
we want it (but not at all costs!). Let see if we can clarify this issue by
using another example: Policers.

In the QoS schema there's a concept called "Policer". A Policer models a
certain aggregate treatment to network traffic. Based on our understanding
of QoS Policy requirements and of the DiffServ model we specify three types
of aggregations:

1. Filter aggregation: Treatment given to all traffic that "passes" a given
filter. A rule that has filter made out of 2 simple conditions such as
'(SourcePort == 80) && (sourceIP == 1.2.3.4)' may specify a Policer to set
limits on all traffic from a given WEB server, say, 3Mbps.

2. Class aggregation: DiffServe class, that is. A service class corresponds
to a given DSCP value. If one defines a service class policer, all flows
"colored" by the DSCP of this service class may be aggregated into a single
class-level policer treatment. (We think this is defined in the PHB policy
block)

3. Arbitrary aggregation: We may want to collect several flows into a single
aggregate and police it together. In our current definition we'd just flag a
policer as "shared" and place it in the repository (only repository resident
policers may be "shared"). We now place a reference to this policer in all
rules that contain policy for members of the aggregate. Suppose, for
example, you have two rules that together specify policy for non-urgent
traffic. We want all traffic not related to our help desk application and
our support extranet to be policed together to not ever surpass the 2mbps BW
(under congestion conditions, of course). There are other policies related
to this traffic so it takes several rules to put this thing together:

R1: If (TelnetTraffic) then color = DS1 and Policer is P1;
R2: If (DestinationIP == BigBoss) then color = DS2 and Policer is P1;
R3: If (NOT (HelpDeskApp or SuppExtranetGateway)) then color = DONT_CHANGE
and Policer is P1;

I know this is not perfect but y'all get the point: We may still want to
color flows belonging to the aggregate differently but still want to police
them together.

Using a reusable Policers repository (where policers may be marked 'shared'
(for #3 type aggregation) or 'non-shared' (for #1 type aggregation) allows
us to specify the three behaviors. (The class level policer needs not be
reusable)

I'm assuming that the other benefits of reusable objects are known and
understood -- see Yoram Snir's email for more)

Ok, this is it for now...
*Yoram

----- Original Message -----
From: Ryan Moats <[email-masked]>
To: <[email-masked]>; 'Yoram Ramberg' <[email-masked]>;
<[email-masked]>
Sent: Tuesday, July 27, 1999 10:05 AM
Subject: RE: ABNF and Versioning (or why Versioning isn't needed)

---

From: "Ryan Moats" <[email-masked]>
Date: Wed, 28 Jul 1999 09:38:34 -0500

| (I hope this is, indeed, a general interest subject...)

If it isn't somebody should say so ;-)  On the other hand, the
LDAP schema is a charter item, so it should be ;-).

I'm editing [again] to save BW.  If I leave out an important
point of Yoram's discussion, my apologies in advance.  Anything
I don't reference here, I either agree with or am in small
enough disagreement to not hold it up.

| The direction toward minimizing DN dereferencing is understood. You're
| pointing to a possible confusion in interpreting the expressive
| power of the replacement of SDN with three attributes. I'm not
| confused. In terms of being able to define an arbitrarily complex
| Boolean expression, I see no difference between SDNs and the 3-tuple
| (group nr., negation, DN) method. After all, I can reconstruct all
| SDNs from their respective 3-tuples, no? This is mighty fine with me.

Excellent!  Agreement is a good thing.

|
| However, I still don't understand (shame on me!) how we're going
| to support reusability of, say, simple conditions. Let's assume,
| for the purpose of this discussion, that reusability is a virtue
| and we'd like to support it "bow to the collect wisdom of a working
| group..."). Please explain how DIT aliasing allows us to support it.
| Can we still support the notion of global repositories? Doesn't
| this defeat the cause of collecting all related object in a single
| subtree so you can pick'em up in one LDAP hit? Don't we end up paying
| the same price as in the case of SDNs (if our reusable objects do
| reside in a global repository)?

DIT aliasing (in this discussion) can be viewed as an implementation
detail, so I'd like to take it off the table for the time being.

Well, shame on me for not explaining this clearly (or at all).
Here's my view: the classes policyContion and policyAction are still part
of the schema (as auxiliary classes [hmm, maybe these should be renamed
to conform to typical LDAP schema designs 8-0]) so that we still have
the ability to attach simple conditions or actions to the policyRule class
itself.  Because of this, I don't see where we've lost anything.  In
the case of a simple policy rule, we still attach the aux classes to
the policyRule class [and retrieve in a single LDAP operation].  Note
that this is not reusable [and wasn't in the original schema] in the
sense that an auxiliary class can be attached, but doesn't bring any
default values with it.  The data has to be instantiated separately for
each object in the directory.

This brings up another idea.  If we want, we can consider attaching
the policyCondition and policyAction items to policyRuleConditionItem
and policyRuleActionItem classes (this is currently not allowed in the
ABNF).  This would mean that DNs are only used in the case of very
complex rules that are to be reused.

| Now, let's see if we can calculate the maximum number of LDAP roundtrips
| necessary to fully fetch a complex condition, using the different methods.
|
| SDN (call it Core method):
|
| I think that the number of LDAP queries you'll have to execute to get the
| whole condition depends on the number of instances of simple conditions
| it is made of.
|
| It's probably something like: nr. of simple conditions + 1 (for the Rule)
| What does this mean? A complex condition we'd like to represent is the
| infamous five tuple IP header condition. At a maximum, again, it costs
| five simple conditions "fetch" operations to fully gather the darn thing.
| This added to the hit for the Rule, it takes six roundtrips to satisfy us.
| Remember I said "maximum" twice. I'm not convinced this can't be optimized
| somewhat...
|
| Raju's bulky conditions:
|
| Other methods for modeling conditions in LDAP do get a much better rating
on
| this. If we follow Raju's schema we find that  same condition (quite
| restricted, though) can be fetched with a single sniff. What the schema
pays
| for that is measured in its expressive power. Conditions are "bulky" and
very
| restricted. To accommodate a slight deviation from the prescribed model
you'd
| have to
|
| 1. Derive your own mechanism (existing classes are too specific to derive
| from),
| 2. Accept the fact that your rules would be opaque to readers, other than
| specially adapted clients -- your own.
|
| Ryan's 3-tuple method:
|
| In this case the whole complex condition (and, in fact, all complex
| conditions!) may be fetched with a single LDAP roundtrip. What's really
| attractive here is that we pay no price in terms of expressive power:
| Arbitrarily complex conditions can still be specified. What does
| this method costs us? I'm yet to fully understand this but I have a hunch
that
| reusability is the victim, or at least, some reusability. I already asked
| Ryan to elaborate on this.

This isn't quite right.  While you get all of the conditions in a single
fetch, you don't get the rules pointed to by the DN unless you either

(a) make fetches for each of those rules.  This requires the same number
of fetches as using SDNs.
(b) plan to make use of something like the extended matching rule discussed
in
draft-moats-ldap-dereference-match-00.txt.  This can return everything in
one fetch, but requires support for the extension.
(c) allow for simple policy conditions to be attached to the
policyRuleActionItem
and policyRuleConditionItem classes as discussed above.  This gets
everything
in a single fetch, but sacrifices reusability in the sense that the data
stored
in the auxiliary is not reusable.

My opinion is that the only reusability is the instantiation of the
tuple is not reusable.  The conditions pointed to with the DN are.  By going
with DIT containment and aux classes, you give the policy administrator
freedom:
They can optimize storage to minimize LDAP queries (aux classes) or optimize
for data reuse (DN pointers) or anywhere in between on a policy by policy
basis.
I think this is a good thing.

|
| A few words about reusability:
|
| Note that we aren't really talking about object class reusability
| -- this is *instance reusability*: The ability to use the same simple
| condition in several rules. One reason we want this is pointed out in
Yoram
| Snir's latest entry in this thread. It's very clear that reusing named
| condition helps -- we want it (but not at all costs!). Let see if we can
| clarify this issue by using another example: Policers.

Since, I won't argue the positives of "reusability" and I agree that having
it
"(but not at all costs)" is a good thing, I dropped the rest of the mail
to save BW.  I think that the method (c) above [which will require a delta
to the ABNF schema] is the way to get the best of both worlds in that you
let the administrator make there own decision. I think that is "the best
of both worlds" and I (thinking as a policy administrator) think this is
what I would want.

Seriously though, from my reading, I think we are getting pretty darn
close to some consensus on this (yeah!)

Ryan

---

From: Yoram Ramberg <[email-masked]>
Date: Wed, 28 Jul 1999 19:00:30 +0300

Following the BW frugality trend, I'll do some cruel editing of my own...

> close to some consensus on this (yeah!)

Yes, I think so. I have some evidence for that in the sense that we wish to
allow the freedom of optimization to the user. Note that the QoS schema does
allow for optimization in the case of simple policy - attach and attach and be
done with it. It is not reusable and was not intended to. This is the excuse we
have for most-compact representation. It goes like this:

condition and action are attached to rule.
variable and constant are attached to the condition.
value is attached to constant (always)
possibly: policer & policer are attached to action
Total: One object. Even LDAP requires but a single operation to bring this baby
home.
So we're fine on that end of the stick.

It's arguably the wrong optimization. It makes it very efficient to define
conditions that are *simple*, i.e., an expression of the format '<variable>
<operator> <constant | value>'. It doesn't allow for that arbitrary aggregation
treatment I alluded to before. Obviously not all would agree that this is the
common case. See Raju's schema, for one very different approach to optimization.
We may have to bite this bullet and provide an optimization for some very common
"template" conditions, like that 5-tuple bitch. (sorry)

Are we fine also on the other end? I need to chew on your latest email a bit
more to fully understand it. Maybe Yoram Snir (RDN for the other Yoram) would
have some fresh thoughts...

Thanx,
*Yoram

---

From: "Ryan Moats" <[email-masked]>
Date: Wed, 28 Jul 1999 11:06:59 -0500

| Following the BW frugality trend, I'll do some cruel editing of my own...
|
| Ryan Moats wrote:
|
| > Seriously though, from my reading, I think we are getting pretty darn
| > close to some consensus on this (yeah!)
|
| Yes, I think so. I have some evidence for that in the sense that
| we wish to allow the freedom of optimization to the user. Note
| that the QoS schema does allow for optimization in the case of
| simple policy - attach and attach and be done with it. It is not
| reusable and was not intended to. This is the excuse we have for
| most-compact representation. It goes like this:
| condition and action are attached to rule. variable and constant
| are attached to the condition. value is attached to constant (always)
| possibly: policer & policer are attached to action Total: One object.
| Even LDAP requires but a single operation to bring this baby home.
| So we're fine on that end of the stick.
|
| It's arguably the wrong optimization. It makes it very efficient
| to define conditions that are *simple*, i.e., an expression of the
| format '<variable> <operator> <constant | value>'. It doesn't allow
| for that arbitrary aggregation treatment I alluded to before. Obviously
| not all would agree that this is the common case. See Raju's schema,
| for one very different approach to optimization. We may have to bite
| this bullet and provide an optimization for some very common
| "template" conditions, like that 5-tuple bitch. (sorry)

Well, I think that my suggestion (in previous email) of allowing
policyCondition
and  policyAction auxiliary classes to be attached to
policyRuleConditionItem
and policyRuleActionItem, respectively solves this.  But definitely
chew on it some more (cause I could be mistaken)

| Are we fine also on the other end? I need to chew on your latest
| email a bit more to fully understand it. Maybe Yoram Snir (RDN for
| the other Yoram) would have some fresh thoughts...

I think we are REAL close (the more I think about it, the more
I think that the aux class attachment method to the child
classes is a GOOD thing and should be added).

Ryan