Subject: Re: draft-ietf-ldapbis-protocol - controls

From: Howard Chu <[email-masked]>
Date: Thu, 31 Mar 2005 21:08:26 -0800

Jim Sermersheim wrote:
> I believe the broad interpretation of "appropriate" leads to far more 
> interoperability problems than the narrow view.
>  
> I'm glad to see that you don't believe a control can be partially applied.
>  
> The fact is, every implementation is going to behave quite differently 
> when it is allowed to either fail-over to ignoring a control or return 
> an error which was caused by the application of that control.
>  
> It will be extremely difficult to evaluate all error paths in code to 
> determine whether the error could be allieviated by ignoring any 
> non-critical controls that might be present. If one cooses not to do 
> that, one could instead, upon encountering any error condition, ignore 
> one non-critical control. If the operation still fails, one could ignore 
> another non-critical error (and probably should un-ignore that first 
> one). One could keep doing this until one has ignored enough 
> non-critical controls (rolling back the transaction each time), 
> until one determines that the operation is just going to fail no matter 
> what, or until the the operation succeeds (one hopes one hasn't 
> accidentally ignored a few non-critical controls that really wouldn't 
> have caused the error)

I am not suggesting that servers do any back-tracking to mitigate the 
failures encountered while processing a request accompanied by 
non-critical controls. Only stating that the presence of a non-critical 
control should not cause a server to choose not to *begin* executing the 
accompanying request. Once the server has decided to begin executing the 
request, any incidental errors should of course be reported back to the 
client as usual.

-- 
   -- Howard Chu
   Chief Architect, Symas Corp.       Director, Highland Sun
   http://www.symas.com               http://highlandsun.com/hyc
   Symas: Premier OpenSource Development and Support

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 31 Mar 2005 21:43:16 -0800

At 08:49 PM 3/31/2005, Jim Sermersheim wrote:
>I believe the broad interpretation of "appropriate" leads to far more interoperability problems than the narrow view. 
> 

This, I think, needs to be made crystal clear.

>The fact is, every implementation is going to behave quite differently when it is allowed to either fail-over to ignoring a control or return an error which was caused by the application of that control.

The specification don't prescribe same behavior.  The
specification prescribes syntaxes and semantics aimed
at allowing implementations to communicate requests
and responses allowing clients to receive service from
the server.  If the client communicates that portions of
the request are not critical, the server should not only
be free to ignore those portions when responding, but
the server should be encouraged to ignore those portions
would lead to responses that deny service to the client.
Deny service will certainly lead to interoperability
problems as the protocol provides no mechanism allowing
the server to communicate which portions of the request it
couldn't deal with.  Fallbacking is problematic as the
protocol has no mechanism to express how far to fallback,
or whether fallbacking would allow service to be provided.

>It will be extremely difficult to evaluate all error paths in code to determine whether the error could be allieviated by ignoring any non-critical controls that might be present. If one cooses not to do that, one could instead, upon encountering any error condition, ignore one non-critical control. If the operation still fails, one could ignore another non-critical error (and probably should un-ignore that first one). One could keep doing this until one has ignored enough non-critical controls (rolling back the transaction each time), until one determines that the operation is just going to fail no matter what, or until the the operation succeeds (one hopes one hasn't accidentally ignored a few non-critical controls that really wouldn't have caused the error)
> 
>If the implementation instead, just performs some arbitrary pre-checks to make sure it's fairly certain that the control won't cause the operation to fail, it will never perform the same as another implementation.

The issue is not whether the operation will perform the same
on another implementation.  The issue is whether the service
provided is within the prescribed possibilities.  If the
client requested an operation with two non-critical controls
X and Y, then the service provided should be consistent with one
of the following prescribed semantics:
        a) operation
        b) operation+X
        c) operation+Y
        d) operation+X+Y

I would hope that servers recognizing both X and Y would try
to provide the "best" service possible.  If they know that
they cannot provide operation+X+Y in a particular scenario,
and are able to detect that scenario up front, they should
by all means ignore one or both of the controls if by doing
so they can provide service.  Since the client did not mark
X or Y critical, the server should assume that service
with a) semantics is better than service with b) or c) than
no d) service.

>Maybe I'm over-complicating or over-simplifying or over-thinking this?
> 
>I rather doubt anyone is confused as to whether a control may be partially applied — that's just silly, so I don't think we need to add language to that effect.
> 
>I think the confusion has to do with what "appropriate" means.
>If the WG wants it to be more precicely defined, then let's come up with some wording. As it is, it just says appropriate. My only fear is that it will ultimately lead to worse interoperability.

I think s/appropriate/appropriate in the server's determination/
would be appropriately vague.

Now, maybe what we need is to clarify is servers should,
where feasible and consistent with semantics choices
of the request (e.g., no partial application of controls),
favor providing some service over providing no service.

>Jim
> 
>>>> "Kurt D. Zeilenga" <[email-masked]> 3/31/05 9:09:46 PM >>>
>I find this comment from Steve Kille interesting:
>Some extensions can safely be ignored without loss of service.
>Others will simply not work if they are not understood and
>need to be rejected. The criticality mechanism allows this
>to be communicated. This is particularly important in a
>store and forward messaging environment, or where
>directory requests are chained.
>< <http://www.openldap.org/lists/ietf-ldapext/199802/msg00088.html>http://www.openldap.org/lists/ietf-ldapext/199802/msg00088.html >
>
>I think the key point I and Howard are making is that by not
>ignoring a recognized and applicable (per the specification)
>non-critical control one cannot service, one causes an
>unnecessary and inappropriate loss of service.
>
>The language in RFC 2251 was:
>If the server recognizes the control type and it is appropriate for
>the operation, the server will make use of the control when 
>performing the operation.
>
>I think it natural for implementors interpret "appropriate
>for the operation" in a broad manner, and I think it clear that
>many client and server implementors have interpreted "appropriate
>for the operation" in a broad manner. Narrow interpretations,
>such as viewing the language as a mandatory requirement
>for servers to "make use" of applicable controls they recognize
>leads to idiotic behavior.
>
>For instance, I could code an LDAP server to "recognize"
>every known request control but not support any of them.
>Under the narrow interpretation taken by the WG, my server would
>be obligated to turn unwillingToPerform (or some other
>appropriate result code). I do not believe this is what was
>intended by designers of LDAP. But regardless of their intent,
>the interpretation taken by the WG will lead to significant
>interoperability problems and hinder development/deployment
>of distributed directory services.
>
>What I think needs to be clarified is the above text is that
>the server is to perform the operation, the whole operation,
>as extended by the control. It cannot perform only part of
>the operation as extended by the control. This, of course,
>has some implications on distributed systems, but I argue
>that the server ability to ignore applicable non-critical
>controls is key to making such systems work in face of
>extensions... which is what I think Steve was hinting at
>in is 1998 ASID/LDAPEXT post.
>
>-- Kurt
>
>
>
>At 07:01 PM 3/31/2005, Howard Chu wrote:
>>Jim Sermersheim wrote:
>>>Sorry, if the operation is chained before being applied locally, don't change criticality.
>>> 
>>>And yes, other options are to test the capabilities of the other agents/backends, and to perform the behavior locally when not supported (a server or frontend could do this with the sorting or vlv control, but would have to gather all entries first.
>>
>>The idea of testing the capabilities of the other servers/backends in advance really doesn't scale well. If you have a large distributed set of directory servers, where any server may be the frontend for yet another arbitrarily large set of servers, you may spend more time querying the capabilities (and getting unreliable answers) than actually processing the client's original request. Nor is this sort of thing properly cachable, since different servers with different capabilities may enter and leave the network at arbitrary times.
>>
>>Following this line of attack is futile. It imposes additional overhead on the servers to collect a type of information in which the client has absolutely no interest.
>>
>>There is nothing in RFC2251 that suggests the current LDAPbis interpretation is either intended or correct. Call me crazy, but I'd say a behavior that is fundamentally useless doesn't belong in the spec.
>>
>>> >>> Mark Ennis < <mailto:[email-masked]>[email-masked] > 3/30/05 12:13:20 AM >>>
>>>Jim Sermersheim wrote:
>>> > I believe this is what our chaining server does as well, though we
>>> > have talked about changing chained controls to be critical in the
>>> > past.
>>> >
>>> > I think your work around could be implemented and render (somewhat)
>>> > expected effects:
>>> >
>>> > Say a search operation has attached to it a non-critical control
>>> > (which the DSA supports). When the local DSA chains, it forces it to
>>> > be critical.
>>> >
>>> > If the operation was chained before the control was used in
>>> > processing the operation, then any resultCode returned from the
>>> > chained-to DSA would be preserved.
>>>Such as an unavailableCriticalExtension resultCode when the original
>>>request had no critical controls?
>>> >
>>> > If the operation was chained during operation evaluation after being
>>> > partially supported by the local DSA, and if the chained-to DSA
>>> > returned unavailableCriticalExtension, the operation could return a
>>> > different error (which would indicate that the operation could not be
>>> > completed as specified).
>>>Might work, although this strategy would lead to a difference in the
>>>results obtained depending on whether the server handles the chaining
>>>versus returning referrals and the client handling followup of the
>>>referrals.
>>>Possibly the question of "what result would a client get if it handled
>>>requests to each individual context?" is worth exploring. I would be
>>>inclined to think a client would send the same request to each context
>>>with the control marked non-critical (or not bother to send it to the
>>>contexts which do not advertise support for the control). This leads me
>>>to be inclined to have a server replicate this behaviour rather than
>>>return errors for non-critical controls.
>>> >
>>> >
>>> >
>>> >
>>> >>>> Mark Ennis <_ <mailto:[email-masked]>[email-masked] < mailto:[email-masked] >_ > 3/29/05 6:11:00 PM >>>
>>> >
>>> > As indicated in the message Jim referenced, our server will ignore
>>> > the criticality flag of a control it recognises. However, this does
>>> > not entirely address the issue Howard raised.
>>> >
>>> > In a distributed environment, a server receiving a request may need
>>> > to chain this request to a number of other servers, where the support
>>> > of various controls in the different servers may be different. Where
>>> > a given server receiving a chained request does not support a
>>> > control, it will base its processing decision on the criticality of
>>> > the control. So even if the server which received the request from
>>> > the client can support a given control, the control will potentially
>>> > be ignored for some part of the result set. This results in the
>>> > effective behaviour of the server, which received the original
>>> > request from the client, ignoring the non-critical control for some
>>> > contexts, even though it does support the control. This is the way
>>> > our server effectively behaves.
>>> >
>>> > The only way around this, at present, would seem to be to force all
>>> > the controls in a chained request to be critical, which would seem to
>>> > be counter to the client's request that the processing of a
>>> > non-critical control is optional.
>>> >
>>> > Our policy is to handle controls, in general, in the way John
>>> > McMeeking described, which is to ignore unsupported and inapropriate
>>> > non-critical controls.
>>> >
>>> > - Mark.
>>> >
>>> > Jim Sermersheim wrote:
>>> >
>>> >>> From another vendor:
>>> >> _http://www.openldap.org/lists/ietf-ldapbis/200403/msg00055.html_
>>> >>
>>> >>
>>> >>
>>> >>>>> "Jim Sermersheim" < <mailto:[email-masked]>[email-masked] < <mailto:[email-masked]>mailto:[email-masked] >_ > 3/29/05 5:08:37 PM >>>
>>> >>>>>
>>> >>
>>> >>
>>> >> Our implementations do this:
>>> >>
>>> >> Is the control recognized and appropriate (as specified) for the
>>> >> operation? Yes, process the operation No, Is it marked critical?
>>> >> Yes, fail with unavailable critical extension No, Ignore the
>>> >> control and process the operation
>>> >>
>>> >> This has been discussed at length in the past, and consensus was
>>> >> that this was the original intent and that this is what should be
>>> >> conveyed in the new document.
>>> >>
>>> >> IIRC, there were interoperability side effects of allowing
>>> >> "appropriate" to mean anything the server wishes it to mean.
>>> >>
>>> >> Jim
>>
>>-- 
>> -- Howard Chu
>> <http://www.symas.com/>http://www.symas.com http://highlandsun.com/hyc

---

From: Mark Ennis <[email-masked]>
Date: Fri, 01 Apr 2005 16:14:15 +1000

Kurt D. Zeilenga wrote:
>> I believe the broad interpretation of "appropriate" leads to far
>> more interoperability problems than the narrow view.
>> 
A control which results in modification of multiple objects must not be 
partially applied, but what about a control that modifies the behaviour 
of a query? A control requesting extra information from each entry in a 
search result, for example a control to reproduce the behaviour of the 
modifyRightsRequest in a DAP read operation, may be partially applied, 
without having any serious implications. To what extent should the 
allowance to partially apply a control be regulated by the specification 
controlling the infrastructure, i.e. [protocols], versus the control 
specification? For example, a control specification for a tree delete 
control should include a discussion of considerations relating to 
application of the control in a distributed environment or across 
different contexts, regardless of what [protocols] may say.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 31 Mar 2005 22:35:42 -0800

At 10:14 PM 3/31/2005, Mark Ennis wrote:
>A control which results in modification of multiple objects must not be partially applied, but what about a control that modifies the behaviour of a query?

Meaning that given a request+control where the control is non-critical,
the server is to perform either the operation indicated by the
request OR to perform the operation indicated by request+control.

The server cannot perform a portion of the operation as indicated
by request+control and a portion of the operation as indicated
by request.  For instance, it would be inappropriate for a
server, in response to a search+manageDsaIt request, to only
apply the semantics indicated by the manageDsaIt to a subset
of the results.  The server is obligated to apply those semantics
across all results or no results.

>A control requesting extra information from each entry in a search result, for example a control to reproduce the behaviour of the modifyRightsRequest in a DAP read operation, may be partially applied, without having any serious implications.

Different kind of "apply".  If the prescribed semantics of the
operation indicated by request+control allow for some of these
modifications to be applied to the DIT and others not, that fine.
But if the prescribed semantics of this operation required that
all or none of these modifications to be applied to the DIT, then
that's what has to be done in performance of that operation.

>To what extent should the allowance to partially apply a control be regulated by the specification controlling the infrastructure, i.e. [protocols], versus the control specification?

The control specification defines what operation to perform
in response to a request+control message.

Where the control is non-critical, the server has a choice of
perform either:
        the operation indicated by the request as extended by the control
or
        the operation indicated by the request.

Performing some other operation is not an option.

>For example, a control specification for a tree delete control should include a discussion of considerations relating to application of the control in a distributed environment or across different contexts, regardless of what [protocols] may say.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 31 Mar 2005 23:04:33 -0800

At 10:14 PM 3/31/2005, Mark Ennis wrote:
>A control which results in modification of multiple objects must not be partially applied,

You're using "applied" here differently that how I, and I think
Jim, used it.

My point was that that a server has to apply the semantics
of the request+control to the entire operation OR apply
the semantics of the request (less the control) to the
entire operation.  It cannot invent an operation that
applies applies semantics request and control not consistent
with these two options.

The semantics of a request+control that prescribe an operation
which modified multiple DIT objects could require that all or
none of these modifications are to be applied to the DIT,
or it could allow that some of the modifications apply and
others not.  That's up to the control specification.  That
is NOT what I was referring to here.

>but what about a control that modifies the behaviour of a query?

>A control requesting extra information from each entry in a search result, for example a control to reproduce the behaviour of the modifyRightsRequest in a DAP read operation, may be partially appli…

If the control specification says that the operation indicated
by the request+control allows the actions to be performed on
a subset of the objects, then a server is free to do that.

The key here is that server is applying the semantics of
request+control throughout operation.  It is the semantics
of the request+control which say whether or not certain
actions are to be performed.

>To what extent should the allowance to partially apply a control be regulated by the specification controlling the infrastructure, i.e. [protocols], versus the control specification?

The control specifications defines the semantics of the operation
requested by request+control.  The protocol specification states
how the client can request
        a) the operation indicated by request be performed,
        b) the operation indicated by request+control be performed,
        c) either operation be performed.
When the client requests c), the server then has the choice to:
        1) perform the operation indicated by the request
        2) perform the operation indicated by the request+control

>For example, a control specification for a tree delete control should include a discussion of considerations relating to application of the control in a distributed environment or across different c…

Likely so.

---

From: Mark Ennis <[email-masked]>
Date: Fri, 01 Apr 2005 17:09:08 +1000

Kurt D. Zeilenga wrote:
>>>> I believe the broad interpretation of "appropriate" leads to
>>>> far more interoperability problems than the narrow view. I'm
>>>> glad to see that you don't believe a control can be partially 
>>>> applied.
>>> 
>> A control which results in modification of multiple objects must
>> not be partially applied, but what about a control that modifies
>> the behaviour of a query?
> 
> 
> Meaning that given a request+control where the control is
> non-critical, the server is to perform either the operation indicated
> by the request OR to perform the operation indicated by
> request+control.
> 
> The server cannot perform a portion of the operation as indicated by
> request+control and a portion of the operation as indicated by
> request.  For instance, it would be inappropriate for a server, in
> response to a search+manageDsaIt request, to only apply the semantics
> indicated by the manageDsaIt to a subset of the results.  The server
> is obligated to apply those semantics across all results or no
> results.
But this does not address a distributed environment where a server 
chaining a request may not know the capabilities of the servers it may 
chain a request to. A search request or a tree delete request may be 
chained to multiple servers, each with different capabilities. The only 
way to enforce the requirements you have stated is to never apply the 
control.

> 
> 
>> A control requesting extra information from each entry in a search
>> result, for example a control to reproduce the behaviour of the
>> modifyRightsRequest in a DAP read operation, may be partially
>> applied, without having any serious implications.
> 
> 
> modifications to be applied to the DIT and others not, that fine. But
> if the prescribed semantics of this operation required that all or
> none of these modifications to be applied to the DIT, then that's
> what has to be done in performance of that operation.
The example I quote is a request for information about what parts of 
entries are modifiable, not a request to modify anything. Again, in a 
distributed environment, servers supporting the control may return a 
control response indicating the information as requested. Servers not 
supporting the control may ignore the control. The application of the 
control in this case would be "partial" and quite safe.

> 
> 
>> To what extent should the allowance to partially apply a control be
>> regulated by the specification controlling the infrastructure, i.e.
>> [protocols], versus the control specification?
> 
> 
> The control specification defines what operation to perform in
> response to a request+control message.
> 
> Where the control is non-critical, the server has a choice of perform
> either: the operation indicated by the request as extended by the
> control or the operation indicated by the request.
> 
This assumes complete knowledge by the server of how the control will be 
applied over the entire area affected by the operation. In a distributed 
environment, this knowledge cannot be guaranteed, meaning the above 
requirements can only be met by never doing request+control in these 
circumstances.

> 
> 
>> For example, a control specification for a tree delete control
>> should include a discussion of considerations relating to

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 01 Apr 2005 00:43:50 -0800

At 11:09 PM 3/31/2005, Mark Ennis wrote:
>>>>>glad to see that you don't believe a control can be partially applied.
>But this does not address a distributed environment where a server chaining a request may not know the capabilities of the servers it may chain a request to.

Some operations cannot be chained, or easily chained.  That's
a issue with their specification.  That is, if the semantics
of that operation (whether defined solely by a request, or
defined by a request plus some number of controls) allow
different actions to be applied to entries of interest,
then that's that.  If the specification of that operation
requires that the same actions be all entries, then that's
that.

The server cannot do the former when specification prescribes
the latter.

But where the client has requested that either operation
X or Y be performed, the server has a choice of doing either
X or Y.  But it cannot do Z.

>A search request or a tree delete request may be chained to multiple servers, each with different capabilities. The only way to enforce the requirements you have stated is to never apply the control.

Depends on the semantics of the operation (defined by the
request+control, as specified in the control specification).

The protocol specification is not defining the semantics of
operations defined by request+control.  The protocol specification
is defining what operations may choose between to perform.  A
client sending a request with a non-critical control is
requesting one of the following be performed:
        a) operation defined by request+control
        b) operation defined by request (less control)

Howard and I argue that the server choose b) if it unable
or unwilling to do a).  The server does not have the option
(c) to perform operation a) on a fraction of the entries of
interest and operation b) on the another fraction of the
entries of interest.

Note that if a) allowed different actions to be applied to
different fractions of entries of interest, while the server
would have the option to do so, that's part of a).  That
is, options with a) shouldn't be regarded as a new operation
(c), but as options within the operation (a).

>>Different kind of "apply".  If the prescribed semantics of the operation indicated by request+control allow for some of these modifications to be applied to the DIT and others not, that fine. But
>The example I quote is a request for information about what parts of entries are modifiable, not a request to modify anything.

Matters not to my argument.  See below.

>Again, in a distributed environment, servers supporting the control may return a control response indicating the information as requested. Servers not supporting the control may ignore the control.

Depends on the specification of the operation indicated by
the request+control.  If the specification says MUST
provide the requested information for each entry returned,
then the server violates the protocol if it returns an
entry without that information.

If on the other hand, the specification says the information
is OPTIONAL, then the server need not ever return the information.

>The application of the control in this case would be "partial" and quite safe.

If the control specification says the extra information MUST
be returned, it is not safe nor appropriate for the server to
ignore that MUST.

>This assumes complete knowledge by the server of how the control will be applied over the entire area affected by the operation.

I am assuming that the server is providing requested service.
If that service requires knowledge to perform that the server
doesn't have, then the server shouldn't perform that operation.
If it has the choice of performing another operation (by ignoring
a non-critical control), it would be appropriate for it to
give that a go.  That operation might require knowledge the
server doesn't have either, in that case, if there or no
other operations to choose from (no more controls to ignore),
an error will have to be returned.

>In a distributed environment, this knowledge cannot be guaranteed, meaning the above requirements can only be met by never doing request+control in these circumstances.

In a distributed environments, knowledge for any operation, including
base operations, is not guaranteed.