Subject: RFC 3280 test cases -- Was: On cross-certificates and pathLenConstraint

From: "Stefan Santesson" <[email-masked]>
Date: Thu, 9 Sep 2004 19:27:41 +0100

Steve,

You are absolutely right in your observation. The fact is that only a
minor part (I would guess about 20%) of the deviations we have
discovered can be found through the NIST tests.

But I'm truly divided on what the best course of action is.
The RFC 3280 logic, when you really dig into it, is so enormously
complex that a test constructed to catch all deviations would be huge
and also extremely costly to perform.

I wouldn't rule out though that it might be possible to make the tests a
bit more effective without major increase of complexity.

Stefan Santesson
Microsoft Security Center of Excellence (SCOE)

> -----Original Message-----
> From: Stephen Kent [mailto:[email-masked]]
> Sent: den 9 september 2004 19:32
> To: Stefan Santesson
> Cc: [email-masked]
> Subject: RE: On cross-certificates and pathLenConstraint
> 
> At 6:17 PM +0100 9/9/04, Stefan Santesson wrote:
> >Yes,
> >That is what I'm saying.
> >The NIST test doesn't catch that though.
> >
> >You can say that MSCAPI is almost faithful to RFC 3280 (with all the
> >best intentions) but there are other minor deviations as well that we
> >have on our list.
> >
> >/Stefan
> 
> Well, this suggests that the NIST test suite needs to be more
> comprehensive. Our discussion shows that non-compliance  can be
> determined independent of a test suite, whereas successful execution
> of a test suite only increases confidence in compliance :-)  The
> usual partial testing vs. counter-example issue.
> 
> Steve

---

From: "Santosh Chokhani" <[email-masked]>
Date: Thu, 9 Sep 2004 16:35:28 -0400

Stefan and Steve Kent:

I would not sell NIST test suite short.  From what I know about it, it may
not be testing that aspect.

MS needs to know that MS CAPI continues to fail several of the NIST PKITS
tests, independent of trust anchor issues, in the areas such as:

	Name Constraints
	Policy related constraints (inhibit policy mapping, require explicit
policy, inhibit any policy)
	Unknown critical extension (MS CAPI merrily process certificates
with unknown critical extensions)

There are other areas where it does not provide support. 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Stefan Santesson
Sent: Thursday, September 09, 2004 2:28 PM
To: Stephen Kent
Cc: [email-masked]
Subject: RFC 3280 test cases -- Was: On cross-certificates and
pathLenConstraint

Steve,

You are absolutely right in your observation. The fact is that only a minor
part (I would guess about 20%) of the deviations we have discovered can be
found through the NIST tests.

But I'm truly divided on what the best course of action is.
The RFC 3280 logic, when you really dig into it, is so enormously complex
that a test constructed to catch all deviations would be huge and also
extremely costly to perform.

I wouldn't rule out though that it might be possible to make the tests a bit
more effective without major increase of complexity.

Stefan Santesson

> of a test suite only increases confidence in compliance :-)  The usual 
> partial testing vs. counter-example issue.
> 
> Steve