Subject: [ldapext] Fwd: I-D ACTION:draft-zeilenga-ldap-incr-00.txt

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Wed, 20 Oct 2004 18:08:25 -0700

This I-D describes an extension we've found to be quite
useful in provisioning applications, Modify-Increment.
When combined with a read-entry control, one can increment
value(s) of an attribute and read-back the values, in one
atomic operation.  Pretty simple.

The catch is that it extends the Modify operation enumeration.
I'm guessing servers which don't implement this extension
would return protocolError.  If there are servers which
would have a more adverse reaction, I might raise the feature
discover bar a bit to avoid such.

Comments?

Kurt

>To: [email-masked]
>From: [email-masked]
>Date: Wed, 20 Oct 2004 10:36:31 -0400
>Subject: I-D ACTION:draft-zeilenga-ldap-incr-00.txt
>Reply-To: [email-masked]
>
>A New Internet-Draft is available from the on-line Internet-Drafts directories.
>
>
>        Title           : LDAP Modify-Increment Extension
>        Author(s)       : K. Zeilenga
>        Filename        : draft-zeilenga-ldap-incr-00.txt
>        Pages           : 5
>        Date            : 2004-10-19
>        
>This document describes an extension to the Lightweight Directory
>  Access Protocol (LDAP) Modify operation to support an increment
>  capability.  This extension is useful in provisioning applications,
>  especially when combined with the assertion control and/or the
>  pre-read or post-read control extension.
>
>A URL for this Internet-Draft is:
>http://www.ietf.org/internet-drafts/draft-zeilenga-ldap-incr-00.txt
>
>To remove yourself from the I-D Announcement list, send a message to 
>[email-masked] with the word unsubscribe in the body of the message.  
>You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
>to change your subscription settings.
>
>
>Internet-Drafts are also available by anonymous FTP. Login with the username
>"anonymous" and a password of your e-mail address. After logging in,
>type "cd internet-drafts" and then
>        "get draft-zeilenga-ldap-incr-00.txt".
>
>A list of Internet-Drafts directories can be found in
>http://www.ietf.org/shadow.html 
>or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
>
>Internet-Drafts can also be obtained by e-mail.
>
>Send a message to:
>        [email-masked].
>In the body type:
>        "FILE /internet-drafts/draft-zeilenga-ldap-incr-00.txt".
>        
>NOTE:   The mail server at ietf.org can return the document in
>        MIME-encoded form by using the "mpack" utility.  To use this
>        feature, insert the command "ENCODING mime" before the "FILE"
>        command.  To decode the response(s), you will need "munpack" or
>        a MIME-compliant mail reader.  Different MIME-compliant mail readers
>        exhibit different behavior, especially when dealing with
>        "multipart" MIME messages (i.e. documents which have been split
>        up into multiple messages), so check your local documentation on
>        how to manipulate these messages.
>                
>                
>Below is the data which will enable a MIME compliant mail reader
>implementation to automatically retrieve the ASCII version of the
>Internet-Draft.
>Content-Type: text/plain
>Content-ID: <[email-masked]>
>
>ENCODING mime
>FILE /internet-drafts/draft-zeilenga-ldap-incr-00.txt
>
><ftp://ftp.ietf.org/internet-drafts/draft-zeilenga-ldap-incr-00.txt>
>_______________________________________________
>I-D-Announce mailing list
>[email-masked]
>https://www1.ietf.org/mailman/listinfo/i-d-announce

_______________________________________________
Ldapext mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ldapext

---

From: Neil Dunbar <[email-masked]>
Date: Thu, 21 Oct 2004 13:38:24 +0100

> atomic operation.  Pretty simple.

What should a server return if an increment is requested on an attribute
with a non-discrete domain of values? (eg, increment on a string value).

I suppose Unwilling To Perform would be OK; Constraint Violation
perhaps. Undefined Type doesn't seem right.

Cheers,

Neil

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 21 Oct 2004 07:45:17 -0700

>perhaps. Undefined Type doesn't seem right.

Our implementation returns constraintViolation, which I feel
is the more appropriate result code than unwillingToPerform.

---

From: "David Boreham" <[email-masked]>
Date: Thu, 21 Oct 2004 08:13:04 -0700

Is it intended that this feature work with replication ?

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 21 Oct 2004 09:23:43 -0700

>Is it intended that this feature work with replication ?

Yes.  The client can update the master using Modify-Increment.
The master can then replicate the DIT change to shadows
(and these to other shadows).  If the replication protocol
supports expression of an Modify-Increment change, that can
be used directly.  Otherwise, the change can be expressed
as one would a Modify-Replace.

We support two replication protocols in OpenLDAP.  In
one (a log-based push protocol), the Modify-Increment
capability of that protocol is used.  In the other (a
state-based pull protocol), the resulting DIT values
are transferred.

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Thu, 21 Oct 2004 09:44:11 -0700

> state-based pull protocol), the resulting DIT values

I see. So in some cases this feature wouldn't work 
(in that it would no longer guarantee atomicity) ?
In the case that you do use an atomic replication propagation
operation, what happens when two 
clients have independently successfully completed
a supposedly atomic increment operation upon two 
different servers ? Is it sufficient that one client
receives the wrong result (at least wrong in that
another client could receive exactly the same value,
talking to another server) ?

Should there be a way for clients to discover if the
feature will work or not ? (or perhaps it shouldn't be
enabled on servers that are participating in replication
mechanisms that renders its effects non-atomic?).

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 21 Oct 2004 10:31:13 -0700

Like LDAP, this extension to LDAP is designed to be used
in directory systems which act in accordance with X.500.
If one is using this extension in a system which doesn't
adhere single-master aspects of the X.500 data model,
lots of more than this operation will lead to application
problems.  I discuss these problems in
draft-zeilenga-ldup-harmful (currently expired, I will
refresh shortly).

For instance, applications which incrementing values using
Modify-Delete+Add would suffer from the lack of ACID
properties in the multi-master system.  Likewise, applications
which used Modify-Increment would suffer from the lack
of ACID properties in the multi-master systems.

In LDAP, clients may presume server is acting in accordance
with the X.500 data model.  If the server is unable or
unwilling to do that, then it should not provide service.
Clients which do desire an alternative service model
should explicit request that model (through a request
control or other mechanism).  And, as far as how to perform
this operation (or any other operation) within an
alternative service model, I will leave that those who
desire to specify alternative service models.

Kurt

At 09:44 AM 10/21/2004, David Boreham wrote:
>I see. So in some cases this feature wouldn't work (in that it would no longer guarantee atomicity) ?
>operation, what happens when two clients have independently successfully completed
>a supposedly atomic increment operation upon two different servers ? Is it sufficient that one client

---

From: "David Boreham" <[email-masked]>
Date: Thu, 21 Oct 2004 10:45:14 -0700

> unwilling to do that, then it should not provide service.

I see, I think you've answered my question: this
feaure should not be implemented by servers that
are participating in a multimaster replication mechanism
with respect to the entry being modified (unless the
server does something unusual to deliver the
required consistency guarantees such as using 2-phase
commit).

It might be worth noting this in the draft, for those
who might ask this question in the future.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 21 Oct 2004 12:49:11 -0700

>required consistency guarantees such as using 2-phase

It's my view that servers which are 'participating in
a multimaster replication mechanism' are not LDAP servers.
LDAP servers, aside from communicating using LDAP PDUs,
act in accordance with X.500.  While the servers you
refer might communicate using LDAP PDUs, they are not
acting in accordance with X.500 and hence, in my view,
are not LDAP servers.

>who might ask this question in the future.

I think its sufficient to specify that LDAP servers
are to act in accordance with X.500.  This I-D does
so by incorporating [Roadmap].

That said, I think it would be good to note to the
community the harm that truly non-optional introduction
of MMR might have on existing directory applications,
to reiterate to implementors that LDAP servers are to
act in accordance with X.500, and encourage those
who desire a different service model to design and
specify a system that offers it.  That system could
be based on LDAP, but if so, it should engineered
as a truly optional extension to LDAP.

(For those who don't grok the terms "truly optional"
or "truly non-optional", I refer you to discussion
of the "MAY" keyword in RFC 2119).

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Thu, 21 Oct 2004 16:15:12 -0700

> acting in accordance with X.500 and hence, in my view,

Can you cite (or quote) a section from X.500 that backs up 
this assertion ?

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Thu, 21 Oct 2004 20:42:11 -0700

At 04:15 PM 10/21/2004, David Boreham wrote:
>Can you cite (or quote) a section from X.500 that backs up this assertion ?

Though not definitive, you might also consider David
Chadwick's "Understand X.500"
  <http://www.isi.salford.ac.uk/staff/dwc/Version.Web/Contents.htm>.
It gives some rationale as to why the ITU selected the
(single) master copy scheme was for the Directory.  If
you want something definitive, I suggest you read X.525 and
sections of X.501 describing the DSA Model.

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Fri, 22 Oct 2004 10:30:48 -0700

> sections of X.501 describing the DSA Model.

Thanks. I wasn't really looking for tutorial advice,
more a specific reference (like section so and so, subsection
whatever, where it says the following : ...).

Surely you must be able to come up with a hard
reference to back up your assertion that multi master
replication is fundamentally incompatible with X.500 and
hence LDAP ?

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 22 Oct 2004 12:21:35 -0700

>replication is fundamentally incompatible with X.500 and

Yes, but since I don't see any need to rehash prior
discussions regarding this fundamental aspect of X.500,
I choose to offer only general references at this time.

I hope we can avoid rehashing this.  I think consensus is
pretty clear on the matter.  X.500 specifications and
implementations presume that one and only one DSA is
authoritative for an entry held in the Directory.

However, that said, I note that this I-D may not be
subjected to any IETF consensus review.  It may be
directly submitted to the RFC Editor.

So, if folks think the X.500/LDAP specifications are
not adequately clear in this area, I would suggestion
that the matter be raised to the LDAPBIS WG so that
they may establish consensus in this area and reflect
that consensus in the revised LDAP TS.

Kurt

---

From: "David Boreham" <[email-masked]>
Date: Fri, 22 Oct 2004 12:46:22 -0700

> authoritative for an entry held in the Directory.

I see, you're really arguing from the perspective
of how the service is implemented rather than any
specific service semantics as seen by clients.
So it's simply enough to say that MMR isn't allowed,
and not important to say _why_. 
I get your thinking now, thanks.

---

From: John McGarvey <[email-masked]>
Date: Fri, 22 Oct 2004 17:02:04 -0400

It is absurd to exclude multi-master replication from normal LDAP 
configurations.  Every large deployment uses it, because they can't 
tolerate a single point of failure. That being the case, there is no 
consensus not to use it.  Just the opposite: There is a consensus that it 
must be used.

One can support multi-master replication and the increment operation. 
However the increment operation, and all operations on that attribute 
value, must occur on a particular master, and must be deferred if that 
master is not available.

John McGarvey
IBM directory architect
919-877-4892 or t/l 254-4892

"David Boreham" <[email-masked]> 
Sent by: [email-masked]
10/22/2004 03:46 PM

To
<[email-masked]>
cc

Subject
Re: [ldapext] Fwd: I-D ACTION:draft-zeilenga-ldap-incr-00.txt

---

From: "David Boreham" <[email-masked]>
Date: Fri, 22 Oct 2004 15:25:29 -0700

>value,
>must occur on a particular master, and must be deferred if that master is 
>not available.

Which (now that we're back within the realms of reality), brings me
back to the question as to how clients are to know this.

A client might attempt to increment an attribute of an entry
when it isn't safe to do so.

Should the server 'know' not to allow the operation ?
Or should the client discover somehow in advance if it
would be safe to do so ?
Or are the 'rules' expected to be enforced administratively somehow ?

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Fri, 22 Oct 2004 16:02:41 -0700

At 02:02 PM 10/22/2004, John McGarvey wrote:
>One can support multi-master replication and the increment operation.  However the increment operation, and all operations on that attribute value, must occur on a particular master, and must be deferred if that master is not available. 

Yes, if you somehow are able to restrict the applications
which rely on the single-master properties when updating some
fragment of the DIT to a particular master in a multi-master system,
then you have, in effect, a single-master system (in respect to
these applications).

My primary concern with the MMR extensions for LDAP proposed so
far is that they simply were not truly optional [RFC2119].  It
certainly is possible (some would say trivial) to design an truly
optional MMR extension to LDAP.

Kurt