Subject: [pkix] CA Testing - how to ?

From: Massimiliano Pala <[email-masked]>
Date: Tue, 17 Nov 2009 16:13:29 -0500

Dear PKIX-ers,

I have recently been involved in some discussions about the need to test
the conformance of PKI software in general and, more specifically, CA
software.

One of the biggest problem in testing the software is that CAs would
refuse to generate certificates that are not compliant to their policies
or certificates used for testing purposes only.

The simplest solution would be to have a Policy that is used only for
test certificates, and issue the test certs with that policy OID. Unfortunately,
applications would not understand that those are test certificates and would
allow them to be used also outside test environments - raising a lot of
possible controversies.

My question is: what is the best practices out there ? What do you do to
test your environments ? Do you also have tools to validate your certificates
against your policy documents or are you doing everything by hand ?

To mitigate the problems related to issuing of certificates for test
purposes only, could a critical extension (eg., purpose:test-only) be added
to a certificate in order to allow that to be used in test environments only ?

What I am trying to understand is: shall we try to standardize something that
would allow apps to recognize a ``test'' certificate from ``normal'' certificate ?

-- 

Best Regards,

	Massimiliano Pala

--o------------------------------------------------------------------------
Massimiliano Pala [OpenCA Project Manager]                   [email-masked]
                                                  [email-masked]

Dartmouth Computer Science Dept               Home Phone: +1 (603) 369-9332
PKI/Trust Laboratory                          Work Phone: +1 (603) 646-8734
--o------------------------------------------------------------------------
People who think they know everything are a great annoyance to those of us
who do.
							   -- Isaac Asimov

---

From: Ogle Ron <[email-masked]>
Date: Tue, 17 Nov 2009 16:28:07 -0500

I issue certificates for DOCSIS certified Cable Modems.  I know that there
are several different CA software vendors who support these certificates.
The certs follow a well defined set of extensions and flag definitions.  We
interoperate fairly well.

BTW, for development and testing purposes, DOCSIS defines a separate PKI.
This way there is no problem with knowing what is test and what is
production.

Ron Ogle

---

From: Massimiliano Pala <[email-masked]>
Date: Tue, 17 Nov 2009 16:47:22 -0500

So how do you test that the production PKI works exactly as the test
PKI if they are two separate setup ? You just rely on the fact that the
configurations are the same ? Don't you have to assert the production
environment somehow ?

Thanks for sharing :)

Cheers,
Max

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 17 Nov 2009 13:45:56 -0800

>This way there is no problem with knowing what is test and what is

This seems like the safest way to go: issue all test certificates under a root that no one has, so they can't trust it by accident.

---

From: Stephen Wilson <[email-masked]>
Date: Wed, 18 Nov 2009 09:03:57 +1100

I don't see how separating test artefacts from the real thing is any 
different (or harder) in PKI than it is for any other software.  
Financial services and health software for example need to be tested 
with realistic data, sometimes real life data, and we have to take care 
with that.

I agree with the point that the more you do to make test certificates 
special, the less valid they are in testing.  Surely at some point in an 
end-to-end test you will try out the certificates in some live 
application?  So you cannot make the certificates un-usable.

The best safeguard against test certificates being used accidentally 
would surely be to destroy the Subject private key(s) after testing is 
complete. 

And it would be prudent to use mickey mouse Subject names.  Perhaps 
literally.

Cheers,

Steve Wilson

Lockstep Group

www.lockstep.com.au <http://www.lockstep.com.au>

Lockstep Consulting provides independent specialist advice and analysis
on digital identity and privacy.  Lockstep Technologies develops unique
new smart ID solutions that enhance privacy and prevent identity theft.

Massimiliano Pala wrote:
> ------------------------------------------------------------------------
>
> _______________________________________________
> pkix mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pkix

---

From: Ogle Ron <[email-masked]>
Date: Tue, 17 Nov 2009 17:14:39 -0500

The configurations are the same just different CA root keys.  We can
validate by looking at the CA certificates and end user certificates to
verify that all extensions, policies, and flags are set the same way.

CableLabs provides the root cert and private key.  We use the private key to
sign our subordinate test CA cert.  Then we issue certs and keys as
necessary for testing purposes.  We issue 2 different types of certs.  One
type is issued for the cable modem for authentication and authorization.
The other type is issued for code signing.

When we are testing, the cable modem network is configured to validate using
the test root certificate.  For production, the cable modem network is
configured to use the production root certificate.  It's fairly simple.

With this testing scheme you can correctly test the affect of extensions,
flags, and policies.  The only difference is the public/private key values.
So you really are testing X.509 correctness/compliance.

For chaining and simple verification purposes, I also use the Apple Keychain
and Microsoft CAPI tools.

Ron Ogle

---

From: max pritikin <[email-masked]>
Date: Tue, 17 Nov 2009 17:21:02 -0600

How about code in the (production) client that verifies the retrieved  
certificate against the active profile?

When installing manufacturing certificates onto hardware devices it is  
especially important to block inadvertent CA config changes (resulting  
in incorrect certificates). Having strict profile checks in the  
requesting client would help prevent this.

- max

---

From: [email-masked]
Date: Wed, 18 Nov 2009 09:12:54 +0100

We issue production certs for testing porpuses to be tested in production environments with a limited lifecycle, for example, 1 month or 15 days. All the time, these are controlled and we´re working with this customer together, after that, they expire or we revocate them. Is this a good approach?

Iñigo Barreira
Responsable del Área técnica
[email-masked]
945016127

ERNE! Baliteke mezu honen zatiren bat edo mezu osoa legez babestuta egotea. Mezua badu bere hartzailea. Okerreko helbidera heldu bada (helbidea gaizki idatzi, transmisioak huts egin) eman abisu igorleari, korreo honi erantzuna. KONTUZ!
ATENCION! Este mensaje contiene informacion privilegiada o confidencial a la que solo tiene derecho a acceder el destinatario. Si usted lo recibe por error le agradeceriamos que no hiciera uso de la informacion y que se pusiese en contacto con el remitente.

-----Mensaje original-----
De: [email-masked] [mailto:[email-masked]] En nombre de Massimiliano Pala
Enviado el: martes, 17 de noviembre de 2009 22:13
Para: pkix
Asunto: [pkix] CA Testing - how to ?

Dear PKIX-ers,

I have recently been involved in some discussions about the need to test the conformance of PKI software in general and, more specifically, CA software.

One of the biggest problem in testing the software is that CAs would refuse to generate certificates that are not compliant to their policies or certificates used for testing purposes only.

The simplest solution would be to have a Policy that is used only for test certificates, and issue the test certs with that policy OID. Unfortunately, applications would not understand that those are test certificates and would allow them to be used also outside test environments - raising a lot of possible controversies.

My question is: what is the best practices out there ? What do you do to test your environments ? Do you also have tools to validate your certificates against your policy documents or are you doing everything by hand ?

To mitigate the problems related to issuing of certificates for test purposes only, could a critical extension (eg., purpose:test-only) be added to a certificate in order to allow that to be used in test environments only ?

What I am trying to understand is: shall we try to standardize something that would allow apps to recognize a ``test'' certificate from ``normal'' certificate ?

-- 

Best Regards,

	Massimiliano Pala

Dartmouth Computer Science Dept               Home Phone: +1 (603) 369-9332
PKI/Trust Laboratory                          Work Phone: +1 (603) 646-8734
--o------------------------------------------------------------------------
People who think they know everything are a great annoyance to those of us who do.
							   -- Isaac Asimov

---

From: Stefan Santesson <[email-masked]>
Date: Wed, 18 Nov 2009 10:59:22 +0100

Max,

I agree with the other comments suggesting setup of a separate PKI.
With a separate root key and everything else just as production, you will
get the most realistic test.

E.g. Using a test policy under the production root is IMO both bad practice
and a significant deviation between the test and production environment.

/Stefan

---

From: Massimiliano Pala <[email-masked]>
Date: Wed, 18 Nov 2009 12:42:36 -0500

Hi Stefan,

I understand your point of view. That is probably why many CAs setup
support test environments (and related maintenance costs..).

In my experience this might work well, but you can never be sure that
the setup are exactly the same after a certain amount of time. Test
environments are considered to be "expendable" sometimes. So people
use them for tests.. and they may be not aligned to what is in production
at some point. Many times in complex IT systems, it happens that
something that was perfectly working in the test environment does not
work properly in the production one...

If that happens and an audit is due, you end up auditing something that
is not the real system :(

And.. you can always argue that what you are auditing is not the real
system - so something could always be different and you might miss it.

With a system that would allow auditing the production system at any
time it would reduce the costs associated with auditing and maybe the
need of a complex setting that allow for a test and a production systems
to co-exist (and all the procedures to port the configuration of a
test system to the production one... :D)

Thanks for your point of view :D

Cheers,
Max

-- 

Best Regards,

	Massimiliano Pala

---

From: Massimiliano Pala <[email-masked]>
Date: Wed, 18 Nov 2009 12:54:52 -0500

Hi Max,

that would be a really good tool. Especially when auditing for:
- federating existing PKIs
- periodically testing your PKI
- assert the compliance to the policy of all the members of a hierarchy

For that to happen, though, we need to work on the definition of a
machine parsable policy (well, at least for the technical aspects) that
apps could use to check certificates against.

Today, the format of a policy is not specified. Current RFCs provide
guidance about the structure of a document. Some CAs publish their
policy in English/PDFs some use other languages/other formats (MS DOC),
very few (I know about only one) use XML.

If we provide a FORMAT (e.g., ASN.1) for expressing the technical aspects
of a policy, that would be really helpful in auditing activities. Also,
apps could finally make use of Policies - something that, at the moment,
users do not really care/understand.

Would a Policy Format document be something the list is interested into ?
The proposal is not to change the content of a policy, but specify a
format that would be useful to apps.

Also, (maybe - not sure if practical) if such a format was standardized,
a user could set the minimum requirement for certificates processing in
apps and use those requirements across different apps as well :D

Later,
Max

---

From: "Moudrick M. Dadashov" <[email-masked]>
Date: Wed, 18 Nov 2009 20:15:40 +0200

Having a test CA doesn't cover all cases. I'd vote for your initial
proposal TestOnly extension in certificate.

There are scenarious where the service is expecting EE cert of a real
person issued by real (no test) CA. You can't test this with a test CA
issued EE cert.

Thanks,

M.D.

cell: +370-699-26662

Massimiliano Pala wrote:

Hi
Stefan,

use them for tests.. and they may be not aligned to what is in
production

need of a complex setting that allow for a test and a production
systems

Cheers,

Max

Max,

With a separate root key and everything else just as production, you
will

E.g. Using a test policy under the production root is IMO both bad
practice

and a significant deviation between the test and production
environment.