Subject: Re: X.500, LDAP Considered harmful Was: OCSP/LDAP

From: [email-masked] (Peter Gutmann)
Date: Sat, 25 Jan 2003 17:38:25 +1300

"Hallam-Baker, Phillip" <[email-masked]> writes:

>First let us dispense with the system designed for use by humans. We have a
>system, it is called the Web, everyone else lost, get over it.

Yup.  That's what the motivation was for my certs-over-HTTP RFC.

>The hierachical organization of the directory does not help the email
>application at all. In fact it is exactly the type of annoyance that led for
>the need for HTTP over FTP. The application knows exactly the question it
>wants to answer but that is not the question the directory will accept,
>instead we have to pay proper homage to the X.500 data model by navigating
>arround it a bit.
>
>The directory does not want to answer the question at all, instead it wants
>to play 20 questions with the client. So while what the client wants to ask
>is 'what is the email address of [email-masked]' what the
>directory wants to be asked is 'do you have any american', yes, 'do you have
>any americans working for first data', 'do you have any...' and so on.

I pointed that out in my ACSAC paper a few years ago: You take a simple query
"SELECT cert FROM foo WHERE key = bar" and then the client dresses it up as an
incredibly complex X.500 filter which the server has to do more work to
translate back into an SQL-equivalent query to actually fetch the key.  The
analogy I've used is that it's like watching a friend of mine's four-year-old
eat peas with a fork, she uses her fingers to put the peas on the fork, moves
it up to her mouth, then uses her fingers to move the peas off the fork into
her mouth, and everyone gets to say how clever she is to use a fork at her
age.  Certs in X.500 directories are the same thing.

Peter.

---

From: "Anders Rundgren" <[email-masked]>
Date: Sat, 25 Jan 2003 16:03:44 +0100

>>In conclusion it is time for the PKI world to seriously consider 
>>whether X.500 and LDAP provide the right certificate repository
>>structure for PKI. For myself I consider them harmful.

>"Peter Gutmann" <[email-masked]> writes:

And the very same ideas spurred me to write the Plug-and-Play PKI for
Web Services proposal, augmenting PKI with the naming system used
on the Web (i.e. URLs), instead of waiting another 15 years or so for
ISO to get the global X.500 registry up and running.

By also loosen-up the ties between PKI and the hierarchical X.500
data model, you get a direct link to the relational database systems,
which are powering just about every serious Web-app there is.

<snip>

Guys,
In spite of pretty different backgrounds, targets, and means, I feel that we
share a common view that the PKI world must take some additional steps
towards the Web world in order to ever become an integral part of the it.

                                       Let's Go!

Anders Rundgren

PS Pardon the "plug" DS

---

From: Michael Ströder <[email-masked]>
Date: Sat, 25 Jan 2003 20:53:49 +0100

Peter Gutmann wrote:
> translate back into an SQL-equivalent query to actually fetch the key.

That's simply not true. I wouldn't claim that X.500 or LDAP directories are 
a really simple thing. But neither are relational databases if you want a 
standardized schema, access protocol, etc.

I see no reason why

SELECT cert FROM foo WHERE key = bar

(note that you left out the host connection part)

is less complex than

ldap://host.domain/o=foo?userCertificate?sub?(key=bar)

(including definition of the connection part)

In both examples the application has to be configured to know
- the host to connect to
- foo
- key
- bar

In the DB example the application has additionally to know about table 
column 'cert' and the application has to be programmed to be capable to even 
access the proprietary database product. To me this looks more complex than 
the LDAP example.

Note that certificate retrieval via LDAP works in wide-spread MUAs since 
years. I simply don't understand why companies don't use it. Well, the 
answer is probably that companies do not deploy X.509-based PKI...

My impression is, especially when reviewing the discussion about the draft 
for LDAP PKI schema, that people here are talking things more complex than 
they are. Especially when implementing LDAP PKI schema one would *not* have 
to change the client (in opposite to many false statements on this list).

> The

Peter, normally you're certainly doing better. Sorry, I file your postings 
about this topic under "Peter Gutmann's personal aversions against X.500 
directories". ;-)

Ciao, Michael.

---

From: "Blake Ramsdell" <[email-masked]>
Date: Sat, 25 Jan 2003 16:36:32 -0800

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Michael Ströder
> Sent: Saturday, January 25, 2003 11:54 AM
> To: Peter Gutmann
> Cc: [email-masked]
> Subject: Re: X.500, LDAP Considered harmful Was: OCSP/LDAP
> 
> Peter Gutmann wrote:
> > 

You're missing Peter's point here -- the SELECT is not necessarily meant
to be SQL, it's meant to convey that the single useful problem that a
certificate database (be it X.500, SQL, whatever) solves is to return
some number of certificates that exactly matches some search criteria.
To use S/MIME as an example, this is one of two criteria:

1. SubjectKeyIdentifier
2. IssuerAndSerialNumber

And potentially a third:

3. Email address

And with PKIX as an example for the purpose of certificate path
construction, you need to have:

1. Subject (Name)
2. SubjectKeyIdentifier (overlaps with S/MIME)

And I think that's it.

> (including definition of the connection part)

I'm glad you were able to get the connection part in there, and that's a
great weight off my mind (it's a configuration parameter -- who cares
what the values are or how they're encoded?)  How exactly do you format
the examples that I've given in such a way to query an LDAP server?
This is the essence of Peter's argument:

"I need to get one or more certificates that meet this exact criteria."

Which I don't think can be done with LDAP, and if it can, then it's in
the form of an overly complex query that varies depending on the schema.

How do I search on IssuerAndSerialNumber?  How do I search on
SubjectKeyIdentifier?

> about this topic under "Peter Gutmann's personal aversions 

I seem to have the same aversion.  The only search I've even gotten to
work is based on email address (ironically enough).  I even failed with
the subject DN search (also, ironically enough), since that is typically
searching for intermediate CA certificates which are not found in some
(most?) directories.  The only effective certificate database I've even
been able to manage is one that indexes on the fields that I've given,
and in fact is based on a general-purpose relational database, populated
with certificates that came along with S/MIME messages.

This "never been able to make the queries work" can certainly be chalked
up to Blake is a Chimp in some cases (I suspect that in some universe
the subject search should work, but I have not found that universe yet),
but I'm not sure that I'm alone here.

For eight years I've been working with S/MIME, X.509 and related
standards, and with the exception of adding SubjectKeyIdentifier, the
criteria I've wanted to search on have not changed.  And I still cannot
do those searches today with anything other than my own database.

Corrections / flames / putdowns welcome, especially corrections and
well-written putdowns.

Blake

---

From: Richard Levitte - VMS Whacker <[email-masked]>
Date: Sun, 26 Jan 2003 09:09:54 +0100 (CET)

In message <!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAA[email-masked]> on Sat, 25 Jan 2003 16:36:32 -0800, "Blake Ramsdell" <[email-masked]> said:

blake> For eight years I've been working with S/MIME, X.509 and related
blake> standards, and with the exception of adding SubjectKeyIdentifier, the
blake> criteria I've wanted to search on have not changed.  And I still cannot
blake> do those searches today with anything other than my own database.

I think we're getting to the point that everyone is making.  'I still
cannot do searches today with anything other than my own database'
says it all.  It's quite true, and it applies to relational databases
as much as LDAP directories the way they're handled today.

The real question is, if someone would need to, how do they search in
your database?  What's the schema?  Is it standardised in any way, or
does every database repository come with it's own, making it a
configuration hell to make sure one can work with each of them?

Sounds to me like exactly the same problem as the ones you point out
for repositories in LDAP directories...

And still, with LDAP, there's work going on to create sensible schemas
(that I can see.  If someone has done something similar for relational
databases, I can't remember seeing it) to be used.  Granted, there may
be things missing (I haven't looked, but I can think of
SubjectKetIdentifier as something that may be missing).  What stops
anyone from proposing?

For a few specifics:

searching in IssuerAndSerialNumber: I agree, tricky.  No real standard
as far as I know.  Intuition would make me look at serialNumber=x,{CA-dn}
(The CA subject with the serial number tucket in front of it), but I'm
sure there are all kinds of more or less imaginative ways to do
this...

Failing DN search: I'm sorry, someone didn't populate the directory
properly, and that should be used as a rebuttal to LDAP?  You realise
what's going to happen with unpopulated RDBMs, don't you :-)...

As for me, I don't really care what the access method to repositories
are, as long as there's a schema I can follow and the repository has
usefull content (or referals to useful content).

-- 
Richard Levitte     | http://richard.levitte.org/ | Spannv. 38, I
Levitte Programming | http://www.lp.se/           | S-168 35 Bromma
T: +46-708-26 53 44 |                             | SWEDEN
     "Price, performance, quality...  choose the two you like"

---

From: "Blake Ramsdell" <[email-masked]>
Date: Sun, 26 Jan 2003 00:50:20 -0800

> From: Richard Levitte - VMS Whacker [mailto:[email-masked]] 
> Sent: Sunday, January 26, 2003 12:10 AM
> To: [email-masked]
> Cc: [email-masked]; [email-masked]; [email-masked]

They can't -- I'm just pointing out that since I'm not aware of any
standard, consistent way to present the queries I've listed to a
directory and get back the certificates I need, I gave up and made my
own proprietary solution.  I would absolutely love to have a published,
standard way of looking things up in a directory using these queries.
Once again, this could be something that I'm missing.

My point is not that relational databases are the salvation -- I
couldn't care less what you use on the back end.  I'm saying that in
order to find a certificate that matches a certain criteria, you should
be able to present the criteria, and get back the matching
certificate(s).  I think this is the essence of Peter's position also.

> usefull content (or referals to useful content).

I think we are on exactly the same page -- I don't care what the
protocol is, or what the schema is, or anything else -- I just want my
questions answered consistently.  I am still not convinced that LDAP
provides this today, or if sufficient efforts are underway to modify it
to support these types of queries.

Phill and Peter (and potentially others) have pointed out that LDAP has
a different design philosophy, and we're trying to make it do things
that it wasn't really designed to do.  We seem to be telling ourselves
that it's best to jam it into LDAP because "that's the standard", and
presumably we are deluded enough to believe that maintaining LDAP
compatibility is somehow virtuous.  The queries that have been presented
seem to be contrary to the normal operation and philosophy of LDAP
(specifically IssuerAndSerialNumber and SubjectKeyIdentifier).  If the
best answer is to modify or profile LDAP in some way, then I've got no
complaints -- I just want my queries answered.

Blake

---

From: Michael Ströder <[email-masked]>
Date: Sun, 26 Jan 2003 13:38:43 +0100

Blake Ramsdell wrote:
> 
> You're missing Peter's point here

I don't.

> -- the SELECT is not necessarily meant

Nope. Peter was definitely talking about complexity of X.500 filters vs. SQL 
statements. I tried to explain that
1. it's equally complex
2. Peter ignored important issues
3. LDAP is already implemented in wide-spread application e.g. MUAs (still 
some issues though)

Please note that I don't have a personal preference or aversion against any 
technology. But I have aversions against people arguing against something 
which at least is partially implemented and proposing something new for 
which there's still is not even an I-D claiming to have the right solution 
for all the issues.

> How exactly do you format

Did you read draft-klasen-ldap-x509certificate-schema? Some people here are 
more in favour of combining certificate matching rules and 
draft-ietf-ldapext-matchedval though.

> "I need to get one or more certificates that meet this exact criteria."

Yepp. Who doesn't?

> the form of an overly complex query that varies depending on the schema.

It's equally complex with any sort of database:
- access protocol
- schema
- security considerations, privacy aspects
...

> The only effective certificate database I've even

Well, you're welcome to write an I-D for deploying a relational database as 
certificate store. I guess in the end it will have the same issues like 
directories.

> do those searches today with anything other than my own database.

Yes, since people are talking about instead implementing it. :-(

Maybe the use-cases should be cleanly modeled. Otherwise we won't come to a 
consent.

Ciao, Michael.

---

From: Richard Levitte - VMS Whacker <[email-masked]>
Date: Sun, 26 Jan 2003 17:34:24 +0100 (CET)

In message <!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAA[email-masked]> on Sun, 26 Jan 2003 00:50:20 -0800, "Blake Ramsdell" <[email-masked]> said:

blake> My point is not that relational databases are the salvation -- I
blake> couldn't care less what you use on the back end.  I'm saying that in
blake> order to find a certificate that matches a certain criteria, you should
blake> be able to present the criteria, and get back the matching
blake> certificate(s).  I think this is the essence of Peter's position also.

And I took it as "let's toss LDAP and use RDBMs, because that works,
at least for me".  My appologies for the misunderstanding...

blake> > As for me, I don't really care what the access method to repositories
blake> > are, as long as there's a schema I can follow and the repository has
blake> > usefull content (or referals to useful content).
blake> 
blake> I think we are on exactly the same page -- I don't care what the
blake> protocol is, or what the schema is, or anything else -- I just want my
blake> questions answered consistently.  I am still not convinced that LDAP
blake> provides this today, or if sufficient efforts are underway to modify it
blake> to support these types of queries.

I see two things that could provide this today: HTTP and LDAP.  The
former via draft-ietf-pkix-certstore-http-03, the latter via RFC 2587
and draft-klasen-ldap-x509certificate-schema-01 (which, BTW, contains
the attributes I was looking for), just as examples.  And from that
point of view, since LDAP repositories are searchable on attribute
values, I'm not sure it's that bad.  Of course, proper DNs will help
as well...

For me, the above is more sufficient than anything based on SQL that
I've seen so far...

blake> Phill and Peter (and potentially others) have pointed out that
blake> LDAP has a different design philosophy, and we're trying to
blake> make it do things that it wasn't really designed to do.

I must be missing something.  What exactly is LDAP not designed to do
in this context?

---

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 26 Jan 2003 09:20:47 +0100

Peter,

From the 03-draft I cannot deduct what length-limitations there
are on the "name" and "email" search attributes.

Anders

---

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 26 Jan 2003 19:53:08 +0100

Michael,
My knowledge of LDAP is limited but isn't it designed to support
a tree-shaped directory where the subject DN of a certificate
is supposed to match a directory entry?  Using CertStore you
would not build a tree-shaped directory but a "flat"
attribute-based structure like the following:

TABLE (
  certblob binary(1000),   -- this is it...
  certHash    binary(20),
  email        varchar (255),
  iHash        binary(20),
  iAndSHash    binary(20),
  name         varchar(255),
  sHash       binary(20),
  sKIDHash  binary(20)
)

I believe the tree-directory puts additional requirements on
subject DNs that does not necessarily match the needs of RPs.

How does LDAP cope with DN collisions?  The RDBMS-variant
can optionally cope with arbitrary duplicates.

When you store a cert in the structure above, you "suck" out all the
"goodies" and store it in a prepared state for _blistering_ performance.

The drawback with this scheme is that it is less suitable for
moving certificates from one database (organization) to
another (organization).  But why should you do that?
It only increases the risk that you use an outdated key.

The things we can read about making directory schemas
match between organizations is not exactly encouraging.
CertStore plus dynamic lookup seems to eliminate
this documented mess.

Anders

----- Original Message -----
From: "Michael Ströder" <[email-masked]>
To: <[email-masked]>
Sent: Sunday, January 26, 2003 13:38
Subject: Re: X.500, LDAP Considered harmful Was: OCSP/LDAP

I don't.

Yepp. Who doesn't?

Ciao, Michael.

---

From: Michael Ströder <[email-masked]>
Date: Sun, 26 Jan 2003 20:12:57 +0100

Anders Rundgren wrote:
> is supposed to match a directory entry?

You MAY retrieve an LDAP entry by specifying a search with DN of the entry 
as the search base. But you can also do a key by value search in the whole 
subtree.

>  Using CertStore you

It would be absolutely trivial to implement draft-ietf-pkix-certstore-http 
working against an LDAP backend.

> subject DNs that does not necessarily match the needs of RPs.

Not necessarily. Note that you can also use a flat name space in the 
directory if you give up the pseudo requirement that the subject DN and the 
directory DN has to be the same.

> How does LDAP cope with DN collisions?

Just give each entry another DN. ;-)

> The RDBMS-variant can optionally cope with arbitrary duplicates.

If you want to address a certain table row later you won't let the RDBMS do 
this automagically. You will define some sort of naming rules.

> "goodies" and store it in a prepared state for _blistering_ performance.

Which approach are you talking about? This mainly applies to all approaches, 
I think.

> another (organization).

Inter-server sync processes can work in many ways.
That's not an issue here.

> It only increases the risk that you use an outdated key.

No matter which database technology you use for searching/retrieving the 
certificate each implementation actually doing something with the key has to 
take full care about certificate validation.

> CertStore plus dynamic lookup seems to eliminate

Both technologies address different needs. I don't see any reason why they 
shouldn't coexist.

Note that LDAP is already implemented in wide-spread client applications. 
IMHO draft-klasen-ldap-x509certificate-schema works well with existing 
client and server implementations.

Ciao, Michael.

---

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 26 Jan 2003 20:48:06 +0100

Michael,
How do you explain the often very heated discussions on the PKIX-list
regarding subject DNs as "pointers into DITs" versus "a bunch of attributes"?

Is this a problem in X.500/LDAP or isn't it?   This problem does not
seem to be present in CertStore at least.

Anders

---

From: Michael Ströder <[email-masked]>
Date: Sun, 26 Jan 2003 21:35:57 +0100

> regarding subject DNs as "pointers into DITs" versus "a bunch of attributes"?

It's an unnecessary academic discussion.

> Is this a problem in X.500/LDAP or isn't it?

It isn't provided you don't want to have the problem.

> This problem does not seem to be present in CertStore at least.

Yes.

Ciao, Michael.

---

From: "Blake Ramsdell" <[email-masked]>
Date: Sun, 26 Jan 2003 22:03:01 -0800

> [mailto:[email-masked]] On Behalf Of Richard 
> Levitte - VMS Whacker
> Sent: Sunday, January 26, 2003 8:34 AM

Well, the HTTP seems fairly new, and I have actually already implemented
it in about 15 minutes on top of my proprietary certificate database.
Now, it's not widely deployed yet, but it's certainly easy to implement
;).  Parse query, call database, return results.

The LDAP citation that you (and Michael) point out was something that I
hadn't seen, so that answers my "how do I make this query" question.

Now to find a server that implements either of these.  Well, the HTTP
one I just wrote myself, so an LDAP one would be useful.

> For me, the above is more sufficient than anything based on SQL that

Yeah, but don't tell anyone -- my certstore implementation is SQL
underneath.

I think that the certstore "facade" makes the most sense in the long
run, and you just bridge that as necessary to LDAP or a SQL server, and
hide your shame about which one you use.

> I must be missing something.  What exactly is LDAP not designed to do

My understanding of the philosophy is that it's a hierarchical directory
based on DN attributes, each leaf entry containing some number of
attributes corresponding to the entity represented by that leaf.  You
can search relative to a RDN base, providing search criteria in the form
of attribute=value pairs that are attributes that occur in the node that
you're searching for, and specify which attributes of that node you'd
like returned.

As I understand this LDAP draft, it says that each certificate be in its
own entry in the directory, and these entries will have additional
attributes that are equality-searchable for the queries that we've been
talking about.  So the certificates become the leaves of the directory,
instead of a higher-level logical entity which owns the certificates.
Not sure this is really a problem, and it's not the first such deviation
from the original intent (subject DNs of certificates have nothing to do
in most cases with the DN representing that entity in the directory).
But I'm not sure we can say with a straight face that this isn't beating
LDAP with a hammer to make it more like we want it.

Blake