Subject: Summary of issues with the Java LDAP API and resolutions

From: Rob Weltman <[email-masked]>
Date: Wed, 21 Mar 2001 09:15:06 -0800

This is a summary of questions, issues, and reservations raised by Kurt in conjunction with the last call on the Java LDAP API technical submission, and proposed responses and corrective actions.

  The changes proposed below are mostly editorial or consist of adding clarification of already implied behavior. However, items 9 and 12 involve dropping a total of three API methods of LDAPConnection, while item 14 involves adding signatures to the bind() method.

Rob

1. >4.38.1 makeSocket
>   "There may be additional makeSocket methods defined when interfaces to
>   establish TLS and SASL authentication in the Java environment have
>   been standardized."

Remove the sentence

2. > Under what circumstances is an API implementation
> required to send an Unbind request?  with disconnect()?
> if so, how do you drop the connection without sending
> an Unbind (such upon receipt of a notice of disconnect
> or an invalid PDU)?

   An API implementation is to send an Unbind request before disconnecting when the LDAPConnection.disconnect() method is called. If an API implementation receives a notice of disconnect from the server, it disconnects immediately without attempting to send an Unbind request.

> How are notices of disconnect handled?

  The client treats the connection as closed by the server (the server is expected to close the connection after sending the notice, but the client treats the connection as closed even if that does not happen).

> How are invalid PDUs handled?

  State in the draft that the API implementation should throw an exception if an invalid PDU is received. I propose a new result code:

100 INVALID_RESPONSE

3. > Why are RFC 2119 imperatives not used?

  Add the boilerplate text about the meaning of SHOULD, MAY, etc. Check all occurrences of should, may, must and uppercase them if appropriate, or delete them.

4. > It appears that section 1 is borrowed from RFC 2251.

  Not quite, it is borrowed from the C API draft. But add an acknowledgement of that in the acknowledgements section.

5. > The I-D represents a number of simple assertion filters
> in the form "sn=Jensen", an invalid form.  The I-D
> should use "(sn=Jensen)".
> 
> The I-D represents a number of DN in the form:
>   cn=Barbara Jensen, o=University of Michigan, c=US
> 
> An LDAPv3 generated per RFC 2253, Section 2 would
> have the form:
>   cn=Barbara Jensen,o=University of Michigan,c=US
> 
> That is, the more restrictive RFC 2253 form, as
> opposed to the RFC 1779 form, should be used.

OK

6. > I also recommend that examples use domains (example.com),
> DNs (dc=example,dc=com) and IPv4/IPv6 addresses reserved
> for examples. For IPv4, IIRC, 192.0.2.0 can be used
> (though that might be TEST numbers).  For IPv6, you can
> use the IPv6 over IPv4 address (2002::/16) based on the
> example IPv4 addresses you use.

OK

7. > I also note that some examples imports "netscape.ldap", not
> "org.ietf.ldap".  BTW, I couldn't find any packaging, name
> space requirements for this API, nor any guide as to what
> namespaces/packaging extensions to this API should be
> placed in.

  Make it org.ietf.ldap.

8. > I believe the API should support an mechanism to issue
> an Unbind request.

  Not required. Servers are allowed to immediately disconnect a client that issues an Unbind request (according to RFC 2251) and the servers we are familiar with do that. There is no difference from the client perspective between disconnecting on the one hand and disconnecting with Unbind on the other.

  See also the discussion under 2 above.

9. > I reiterate my request that a detailed security analysis should
> be undertaken before the document is progressed to the IESG.  One
> of the areas which concern me is the management of authentication
> secrets by the API.  Also, a discussion of considerations
> implementing API SASL and TLS features should be detailed.  In
> particular, the handling of TLS alerts seems an area needing
> some discussion.  There were security trade-off taken (in
> referral chasing) which should be discussed in the I-D.

  We propose dropping two methods which would allow unscrupulous code, if it was handed a live LDAPConnection, to access credentials:

  LDAPConnection.getAuthenticationPassword()
  LDAPConstraints.getReferralHandler()

  Eliminating the methods means that it will not be possible to completely serialize and then deserialize a live connection, but it will reduce the risks incurred by passing such a connection to untrusted code.

  The C API draft security section contains the following two additional sentences which we could add to the security section of the Java API draft:

"Implementations of this API SHOULD be cautious when handling authentica-
tion credentials.  In particular, keeping long-lived copies of creden-
tials without the application's knowledge is discouraged."

  For SASL and TLS features, the reader is directed to RFCs 2222 and 2246, respectively.

  As for TLS alerts, nothing is required of this draft. A TLS implementation used by an implementation of the API should conform to RFC 2246, including section 7.2 which covers the Alert Protocol (closure alerts and error alerts). Higher level protocols do not need to be aware of the Alert Protocol. If an alert leads to the TLS connection being closed, the API and application become aware of the disruption as with other connection interruptions.

10. > The 9.1 example code seems to use getStringValues() without
> a care in world to whether or not the values are compatible
> with this method.  I think a statement of the limited
> applicability of this and other String-typed value access
> methods should made, something like:
>         Applications should not use methods returning values
>         in String form unless those values are known restricted
>        to UTF-8.  Applications should not assume an attribute,
>         regardless of its syntax, is returned in string form
>         as the server is free to return any attribute using
>        ;binary transfer.

  The latter sentence is not correct, but we will add (in 4.1.4, 4.1.6, and the example):

"getStringValues() should only be called if the attribute values are known to be strings with values restricted to UTF-8. The returned Strings have undefined values if the attribute contains values not restricted to UTF-8."

11. > I would think that an referral/continuation handler provided
> the application would need to distinguish the between a
> search continuation and a referral. I note that chasing
> requirements for referrals and continuations differ, for
> example how the client needs to process an LDAP URL with
> an absent base differs.

  This has been refuted on the ldapext list.

> Also, I note it would be useful if methods of LDAPUrl
> (4.41.1) distinguished between an explicit and default
> parameters of an LDAP URL.  For example getScope() could return
> SCOPE_DEFAULT if an explicit scope wasn't provided.  As
> this behavior is likely only useful to applications
> doing their own referral/reference chasing, likely
> best to have some means to enable/disable this behavior.

  Unnecessary complication.

> What does "explicit URL string" imply?  Does it imply
> that URLs with implicit parameters are not allowed? 
> e.g. LDAPUrl("ldap://")

  Remove "explicit".

> What does "full-blown LDAP URL" imply?

  A fully defined LDAP URL, with defaults assigned where not specified by the client.

> And "nearest X.500/LDAP"?  nearest?  X.500?

  Replace with "localhost".

> Also, I suggest the default port be 389 so that:
>         LDAPUrl("ldap://localhost")
> and
>         LDAPUrl("localhost", null, null);
> are equivalent.

  Yes

12. > The TS depends upon "ldaps" and "LDAP over SSL" which
> have no technical specification of appropriate
> standard track maturity.

  Remove 4.41.13 LDAPUrl.isSecure.

13. > I would assume "thread-safe" is a term defined in the Java
> technical specification.  As this requirement is a MUST, I
> would suggest a normative reference to the specific definition
> of the term would be appropriate.

  The Java Technical Specification at http://java.sun.com/docs/books/jls/second_edition/html/memory.doc.html#30206 does not mention thread-safety, but thread-safe is not a Java term but rather a general programming concept applicable to any language supporting threads. We are not aware of a normative reference, but we could provide one from a dictionary if necessary.

> Also, how do implementations indicate which additional
> classes and methods are thread-safe?
 
  By stating that they are thread-safe in the documentation for the implementation.

14. > The bind passwd is a String.  Besides the implied
> UCS-2 -> UTF-8 conversion, I suspect there are
> password values which cannot be represented as a
> String.  I believe byte[] would be more appropriate
> type as the associated protocol field is an
> octetString (and common attributes for password
> storage hold octetString values).

  I have not heard of a single case so far, in the four years the API has been available and used, where the password could not be represented as a String, but it is legal for an LDAP bind password to contain values not in the UTF-8 character set. The following addition to the API supports those potential cases.

  For each bind() method of LDAPConnection (section 4.6.5 of http://www.ietf.org/internet-drafts/draft-ietf-ldapext-ldap-java-api-13.txt) add a corresponding method that takes a byte[] instead of a String for password.

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Sat, 24 Mar 2001 21:37:29 -0800

At 07:15 AM 3/21/01 -0800, Rob Weltman wrote:
>   An API implementation is to send an Unbind request before disconnecting when the LDAPConnection.disconnect() method is called.

Even if cloned?

Anyways, this behavior should be detailed.

>If an API implementation receives a notice of disconnect from the server, it disconnects immediately without attempting to send an Unbind request.

This is appropriate.  This behavior should be detailed.

Also, is such the notice of disconnect result code and error text made
available to the application only via a unsolicited notification
listener or is also available via LDAPConnection access methods?

>  The client treats the connection as closed by the server (the server is expected to close the connection after sending the notice, but the client treats the connection as closed even if that does…

The term "client" is ambiguous when discussing an API.  Please
distinguished the "application" from the "API."  

>  State in the draft that the API implementation should throw an exception if an invalid PDU is received. I propose a new result code:

I would hope the API would be extensible... allowing someone
to layer support for additional PDU types on top of the
API.  There will be additional PDU types added to LDAP soon.

>  Not required.

What?  I believe the API should provide a mechanisms to
directly issue each protocol PDU.  Though the disconnect()
method is okay... but it provides no means to provide
Unbind controls.

>  Eliminating the methods means that it will not be possible to completely serialize and then deserialize a live connection, but it will reduce the risks incurred by passing such a connection to unt…

And since it was eliminated for security considerations, the
consideration should be detailed in the I-D.

>tials without the application's knowledge is discouraged."

Please!

>  As for TLS alerts, nothing is required of this draft. A TLS implementation used by an implementation of the API should conform to RFC 2246, including section 7.2 which covers the Alert Protocol (c…

I note that LDAP use of TLS (RFC2830, 4.1) allows for LDAP data to
be sent over the TCP stream after a TLS closure.  Application
notification of changes in security services provided is critical.

>  The latter sentence is not correct,

Let me clarify... RFC 2251 does not restrict when a server
may choose to return any attribute described using ASN.1
using ;binary transfer.

>but we will add (in 4.1.4, 4.1.6, and the example):

>  This has been refuted on the ldapext list.

I will review the comments as I am not yet convinced.

>  Unnecessary complication.

Unnecessary unless an application needs to implement their
own chasing...

>  A fully defined LDAP URL, with defaults assigned where not specified by the client.

That seems like a more appropriate description than "full-blown".
I suggest you add it.

>  The Java Technical Specification at http://java.sun.com/docs/books/jls/second_edition/html/memory.doc.html#30206 does not mention thread-safety, but thread-safe is not a Java term but rather a gen…

Because of the MUST, I think you need to be more specific
in what is required by the implementor.

>  By stating that they are thread-safe in the documentation for the implementation.

A documentation provided for an implementation should not
be viewed as part of implementation.  I do not believe
it appropriate for the TS to contain any particular
documentation requirements.

I would assume this API provides source-level interoperability.
If a developer has to read each APIs document, than there
is not source-level interoperability.

---

From: Rob Weltman <[email-masked]>
Date: Sat, 24 Mar 2001 22:17:31 -0800

> >   An API implementation is to send an Unbind request before disconnecting when the LDAPConnection.disconnect() method is called.

  Only before disconnecting. If multiple objects (clones) share the same physical connection, disconnection only happens when the last object sharing the connection calls disconnect().

> listener or is also available via LDAPConnection access methods?

  Only through a listener.

> distinguished the "application" from the "API."

  In the case of the Java LDAP API draft, the client is the API implementation.

> notification of changes in security services provided is critical.

  Hmmm, that's interesting. Are you aware of any TLS implementations that allow this? I had discussions with the inventors of SSL (the precursor to TLS) back at Netscape about just that and they didn't think it was realistic for a connection to be continued to use for non-TLS exchanges on terminating the TLS session. I know that RFC 2246 section 7.2.1 leaves the question open.

  If it is realistic, we'll address it. If not, RFC 2830 should change.

> Unnecessary unless an application needs to implement their

  Unnecessary in that case, too.

> in what is required by the implementor.

  Thread-safe is a well-known and well-understood programming concept, but it is not defined in the Java Technical Specification (that applies to many other programming concepts as well).

> is not source-level interoperability.

  APIs are interoperable even if one is thread-safe and one is not, as long as the user of the APIs is aware of the situation. The users of APIs become aware of the APIs and their characteristics by reading the documentation. It is fully acceptable for the documentation for an API to indicate which methods are thread-safe and which are not.

  In this case, we are talking about value-add to the standard. Implementors will commonly provide additional methods and features beyond the standard. The only way to find about them and how to use them is by reading the documentation for them. You won't be able to find out about them by reading the standards document for the API.

Rob

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Sat, 24 Mar 2001 23:14:25 -0800

>  Only before disconnecting. If multiple objects (clones) share the same physical connection, disconnection only happens when the last object sharing the connection calls disconnect().

I suggest this be clarified in the TS.

>> listener or is also available via LDAPConnection access methods?

I suggest this be noted in the TS.

>  In the case of the Java LDAP API draft, the client is the API implementation.

I believe that an unwise distinction to make as it implies
the API is solely responsible for meeting requirements of
RFC 2251.

>  Hmmm, that's interesting. Are you aware of any TLS implementations that allow this?

I believe OpenSSL does.

>I had discussions with the inventors of SSL (the precursor to TLS) back at Netscape about just that and they didn't think it was realistic for a connection to be continued to use for non-TLS exchanges on terminating the TLS session. I know that RFC 2246 section 7.2.1 leaves the question open.

I believe it should be addressed in the API.

>  Unnecessary in that case, too.

Unnecessary in that the application can implement it's own
LDAP URL parsing because this class doesn't address the needs
of such applications.

>  Thread-safe is a well-known and well-understood programming concept, but it is not defined in the Java Technical Specification (that applies to many other programming concepts as well).

Well I agree that "thread safe" is generally understood concept,
I note it has numerous definitions.  My dictionary has two which
have subtle but significant differences.  Given the MUST, I assume
there are specific requirements which must be met and these
subtleties matter.

>  APIs are interoperable even if one is thread-safe and one is not,
>as long as the user of the APIs is aware of the situation. The users of APIs become aware of the APIs and their characteristics by reading the documentation.

In my opinion, it is inappropriate for a user to have to read
any particular's documentation to use their implementation of
the specification.  The whole point of standardization is to
free the user from having to code to specific implementations.
An API TS which fails in this regard, is not appropriate for
the standard track.

I believe the TS should include a statement as to level
of interoperability which it intends to provide. I presume
the TS aims to provide source-level compatibility.

---

From: Rob Weltman <[email-masked]>
Date: Sun, 25 Mar 2001 15:38:16 -0800

> I believe it should be addressed in the API.

  I have to think about that a little more.

> LDAP URL parsing because this class doesn't address the needs

  An application is always free to do its own LDAP URL parsing, and there will undoubtedly always be applications that choose to do so. But all applications that implement their own chasing of referrals do not need to distinguish between explicit URL parameters and implied URL parameters. As a matter of fact, in the years I've worked with the Java LDAP API and applications that did their own referral processing, I haven't seen a single such case. The effective URL is what matters other than on error, and on error the literal URL is available to present to the user or a log file.

> there are specific requirements which must be met and these

  There are no specific requirements other than that it be safe for more than one thread to invoke the method simultaneously, which it is what thread-safe means.

> An API TS which fails in this regard, is not appropriate for

  The proposed API guarantees that a developer can code to one definition and it will work on any implementation of the API. However, if the developer wants to take advantage of additional features beyond what is specified in the standards document for the API in a particular implementation, the developer will have to read the documention for that implementation. There is no way around it. In the case under discussion, an application which assumes a method is not thread-safe will work also if the method is thread-safe.

> the TS aims to provide source-level compatibility.

  The API provides binary compatibility which is stronger than source-level compatibility (which may require you to recompile an application when using a different implementation of the API). But of course, here as anywhere else, if a particular implementation provides additional features and a developer choose to take advantage of them, the developer can't assume the same features are available in every other implementation.

Rob

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Sun, 25 Mar 2001 20:01:54 -0800

At 03:38 PM 3/25/01 -0800, Rob Weltman wrote:
>  An application is always free to do its own LDAP URL parsing, and there will undoubtedly always be applications that choose to do so. But all applications that implement their own chasing of referrals do not need to distinguish between explicit URL parameters and implied URL parameters.

The scope defaulting is different for search referrals than for
search continuations.  In a referral, no explicit scope implies
the client should reuse the prior scope.  In a continuation, no
explicit scope implies the client should use a base derived from
the prior base (base->base, one->base, subtree->base).

>As a matter of fact, in the years I've worked with the Java LDAP API and applications that did their own referral processing, I haven't seen a single such case.

You'll find such a case detailed in RFC 2251 examples!

>The effective URL is what matters other than on error, and on error the literal URL is available to present to the user or a log file.

As far as the other issues I note (thread-safe,MUST document),
I have no additional words I think I can offer which might get
my point across to you.  Anyways, I think I many object to
the wording and I'll try to get back to the list with a more
specific suggestion or two.

Kurt

---

From: Rob Weltman <[email-masked]>
Date: Sun, 25 Mar 2001 23:12:41 -0800

> the prior base (base->base, one->base, subtree->base).

  That has already been discussed on this list. The API implementation delivers the referral correctly rewritten so the client doesn't need to worry about it, regardless of if using automatic referral following or automatic referral following.

> You'll find such a case detailed in RFC 2251 examples!

  The API we propose (and which has been is use for four years now) handles the rewriting of the scope exactly as per RFC 2251 so that there is no need for a client to do it.

Rob

---

From: Rob Weltman <[email-masked]>
Date: Sun, 25 Mar 2001 23:17:55 -0800

>   That has already been discussed on this list. The API implementation delivers the referral correctly rewritten so the client doesn't need to worry about it, regardless of if using automatic refer…

  Make that "if using automatic referral following or chasing referrals itself".

---

From: "Kurt D. Zeilenga" <[email-masked]>
Date: Mon, 26 Mar 2001 06:36:07 -0800

At 11:12 PM 3/25/01 -0800, Rob Weltman wrote:
>  That has already been discussed on this list. The API implementation delivers the referral correctly rewritten so the client doesn't need to worry about it,

I must have missed this point, I was under the impression that this was
a low-level API which provided direct access to all values transferred
by the protocol.  Though I have concerns with this API not providing
a low-level API, the approach you suggest will work (but will limit
the user as all high-level APIs naturally do).  Anyways, I believe the
TS needs much more clarification as to what is expected of implementations
in this area.

>regardless of if using automatic referral following or automatic referral following.

>  The API we propose (and which has been is use for four years now) handles the rewriting of the scope exactly as per RFC 2251 so that there is no need for a client to do it.

That's not how I read the TS.

---

From: Rob Weltman <[email-masked]>
Date: Mon, 26 Mar 2001 19:57:10 -0800

> TS needs much more clarification as to what is expected of implementations

  If there was ambiguity in how referrals are to be rewritten, or if there was greater value in the freedom to botch the scope field of a referral URL than in the correct rewriting of the URL, we could put the onus on the developer to handle the referral rewriting. But since RFC 2251 is clear on what needs to be done, it is better handled by the API implementation. A determined botcher can still change the URL delivered by the API implementation if doing manual referral chasing, but it requires extra effort on his/her part.

Rob