Subject: Re: DN Encoding by UTF8String

From: [email-masked] (Peter Gutmann)
Date: Mon, 15 Dec 2003 20:36:21 +1300

Jean-Marc Desperrier <[email-masked]> writes:

>The problem is *not* for the low level crypto API to support UTF8String.
>Basically all API around do that.

Only newer ones do.  Some (e.g. Netscape 4.x) will not just fail to process
the string but actually crash the application when they encounter a
UTF8String.

>There's is detailled information about how memcpy() to encode and memcmp() to
>compare don't work as soon as anything more complex than printablestring is
>involved. 

Why not?  The encoding rules are:

  DN originator (e.g. client generating a cert request) encodes the DN in any
    way they see fit;
  subsequent users use memcpy() to "re-encode" the original DN, and memcpy()
    to compare it;

OK, that doesn't work for something odd like a directory lookup, but for
normal cert usage where all you care about is locating a cert or checking for
equality based on a DN, it's fine.

>And don't work also if you take into account anything from the full x520
>comparison rules.

... which don't work in practice (see the X.509 Style Guide), so it's not
really relevant.

>But this kind of comparaison being prevalent in application, the only
>solution to avoid problem is for the CA to normalize strictly every name
>before emitting certificates.

How well do you think that'll work in the presence of implementations that use
memcpy() and memcmp()?  Have you ever tried generating a cert request from
(say) IE and sending it to a CA that "strictly normalizes" the DN before
returning it?

(To save you the effort if you haven't done this, it'll be rejected by the
 client.  I'm just using IE as a common, easy-to-get-to example here, you can
 see the same thing with other software when you fiddle with DN encodings.
 Try signing an S/MIME message with the DN in the SignerInfo canonicalised to
 not match the cert DN).

In my early naivete about the real state of X.500 I actually tried to
implement the X.520 name comparison rules some years ago (that's how the Style
Guide entry on this came about).  I found out very quickly that one thing you
never, ever do is change the DN encoding once it's been created by the
originator (I originally stored the DN in canoncalised form, and had to change
my code to cache the original DN so I could "encode" it with memcpy()).  A
year or two after changing the encoding to memcpy() I #ifdef'd out the full DN
comparison and replaced it with a memcmp() of the encoded form.  A year or two
after that, I removed the X.520 comparison code entirely.  All it was was a
potential source of trouble anyway, it was far too complex to rely on.  I
haven't had any trouble since I switched to memcpy()/memcmp(), and it's been
used in some really odd environments with all sorts of non-European character
sets (cryptlib seems to be quite popular in Asia for some reason, possibly
because it really goes out of its way to handle i18n, which means the i18n
support got a lot of stress-testing).  In fact, exactly *because* of memcpy()
/memcmp() it will work with absolutely anything: BMPString, T61String,
T61String that's actually an 8859-1String, floating diacritics (how much other
software can handle those?), UTFString, string types not even dreamed up yet.

Peter.

---

From: Richard Levitte - VMS Whacker <[email-masked]>
Date: Mon, 15 Dec 2003 11:02:40 +0100 (CET)

In message <[email-masked]> on Mon, 15 Dec 2003 20:36:21 +1300, [email-masked] (Peter Gutmann) said:

pgut001>   subsequent users use memcpy() to "re-encode" the original
pgut001>     DN, and memcpy() to compare it;

I doubt that :-).

pgut001> In my early naivete about the real state of X.500 I actually
pgut001> tried to implement the X.520 name comparison rules some years
pgut001> ago (that's how the Style Guide entry on this came about).  I
pgut001> found out very quickly that one thing you never, ever do is
pgut001> change the DN encoding once it's been created by the
pgut001> originator (I originally stored the DN in canoncalised form,
pgut001> and had to change my code to cache the original DN so I could
pgut001> "encode" it with memcpy()).  A year or two after changing the
pgut001> encoding to memcpy() I #ifdef'd out the full DN comparison
pgut001> and replaced it with a memcmp() of the encoded form.  A year
pgut001> or two after that, I removed the X.520 comparison code
pgut001> entirely.  All it was was a potential source of trouble
pgut001> anyway, it was far too complex to rely on.  I haven't had any
pgut001> trouble since I switched to memcpy()/memcmp(), and it's been
pgut001> used in some really odd environments with all sorts of
pgut001> non-European character sets (cryptlib seems to be quite
pgut001> popular in Asia for some reason, possibly because it really
pgut001> goes out of its way to handle i18n, which means the i18n
pgut001> support got a lot of stress-testing).  In fact, exactly
pgut001> *because* of memcpy()/memcmp() it will work with absolutely
pgut001> anything: BMPString, T61String, T61String that's actually an
pgut001> 8859-1String, floating diacritics (how much other software
pgut001> can handle those?), UTFString, string types not even dreamed
pgut001> up yet.

Hmm.  OpenSSL provides a limit set of rules for comparing DNs: it does
space normalization and a case insensitive compare for PrintableString,
and case insensitive comparison for IA5String if the attribute type is
emailAddress from pkcs#9.  For all other strings, a straight memcmp()
is done.  Would you say we're going overkill?  I'm not sure about
that, since those special cases were added fairly recently, after we
got some user complaint, and possibly after I had a run with the NIST
PKI test bunch...

-----
Please consider sponsoring my work on free software.
See http://www.free.lp.se/sponsoring.html for details.
You don't have to be rich, a $10 donation is appreciated!

-- 
Richard Levitte     | http://richard.levitte.org/ | Tunnlandsv. 3
Levitte Programming | http://www.lp.se/           | S-168 36 Bromma
T: +46-708-26 53 44 |                             | SWEDEN
     "Price, performance, quality...  choose the two you like"