Subject: [certid] weird CN-IDs (subjectCommonName) in SSL Labs Survey Data

From: =JeffH <[email-masked]>
Date: Mon, 18 Oct 2010 17:09:45 -0700

So, I got a couple of bug reports on my original msg in this thread (thanks!).

It turns out (a) my regex was sorta broken, and (b) the subjectCommonName 
column Ivan has in the dbase table has all the "CN" attr values smooshed into 
it. So the certs with a subject with just one CN attr-value-assertion (AVA) 
will (likely) have a reasonable-looking value (but not necessarily a domain 
name) in the subjectCommonName column, but a subject with multiple CN AVAs will 
  have an odd-looking value in subjectCommonName (like the ones I included in 
my orig msg).

Now, as Martin pointed out, the ones that look like this..

   www.cebbank.com+2.5.4.5=#130f313030303030303030303131373438

..seem to be an RDName that's comprised of two AVAs, and in the/this DN string 
representation, they're conjoined by a "+" char. And yes, as Matt noted, it 
seems there's a bug in the parser used for the survey where it apparently 
assumed that an RDN beginning with a "CN=" contains only a CN AVA. Also, upon 
examining subject names more closely, it appears it properly parsed RDNs where 
the CN AVA appears after the first AVA in the RDN. E.g...

   2.5.4.5=#13083431343038353030+CN=secure.capis.com

..yielded "secure.capis.com" in the subjectCommonName column, surprisingly enough.

So, anyway, yes, there are subject name with CN AVAs that have values /other 
than/ dns domain names. Sometimes the cert has both natural-language CN values 
(i.e. one or more) along with domain name values, sometimes just 
natural-language values, but largely the CN values in the overall data are 
domain name values.

With a new regex, I count 622 cert entries with at least one natural-language 
CN value (they may also have CN values containing domain name-like constructs).

e.g...

CN=USMOTIVATION

CN=MONARCH MARKETING SERVICES (MONARCH GRAPHICS, INC)

CN=NORTH CAROLINA INFORMATION DATA, INC.

I hear tell that the regex-like esatz CN-ID values, e.g...

   (www|zeus).asap-solution.com

..are due to at least one browser supporting (but no more?) such constructs.

=JeffH

---

From: =JeffH <[email-masked]>
Date: Tue, 19 Oct 2010 10:07:06 -0700

> There sure are some bizarre things in CNs... is there any chance you could
 > implement what Marcus Ranum calls "artificial stupidity" in your anomaly-
 > detection, create a filter that accepts all standard DN component styles and
 > then kick out certs that don't pass the filter?

that's essentially what I did yesterday tho I refined the regex a bit this 
morning. Here's the (perl) regex..

   /[Cc][Nn]=(?!([\-\w\*]+\.)+[\-\w]+)/

..which appears to also work fine in NEdit. What it does (my intent anyway, I 
am not an awesome regex master) is recognize any CN-ID pattern that is /not/ 
syntatically a dot-separated LDH (letter digit hyphen) DNS domain name.

Employed thus..

while (<>) {
   if ( /[Cc][Nn]=(?!([\-\w\*]+\.)+[\-\w]+)/ ) {
     print "\n"; print;
   };
};

I think it does what you suggest below. What I fed into the above loop is a txt 
file output by querying the raw database table for all contacted domains and 
returned cert subject values (one pair per line, would of course work on a file 
with just subjects in it).

 >  In other words instead of
 > trying to create a regex to detect all the bizarre things that turn up in
 > there, create one to pass normal DNs and treat everything that doesn't pass as
 > an anomaly?

yes, done.

So there appear to (actually) be 433 (I'd mis-counted yesterday (haste makes 
waste)) such subject DNs (622 is an incorrect count AFAICT today).

Note that these subject DNs may /also/ contain syntactically correct CN-ID 
values (many do, some do not).

 > It'd be interesting to see what sort of stuff is floating around
 > out there...

of course. Note that the data is nominally available upon request as Ivan has 
blogged, which I posted here earlier here..

[certid] fyi: Ivan Ristic / Qualsys SSL Labs release raw data from the Internet 
SSL survey
http://www.ietf.org/mail-archive/web/certid/current/msg00484.html

There's a shrink-wrap EULA that Ivan wants folks to agree to before obtaining 
the data and I haven't groveled thru it enough to tell whether it's legit to 
just post all 433 results (or any other full result set) to the a public list & 
archive such as this.

Also note that he/qualsys may re-run the survey periodically. This is also the 
intention of the EFF folks and their "TLS/SSL observatory" 
<https://www.eff.org/observatory> (tho they have yet to publicly release their 
data). (note also that their survey data collection methodologies differ -- EFF 
folks say they "NMAPed Internet for hosts listening on tcp 443", where Ivan 
queried publicly registered domain names in a (large) subset of all TLDs)

HTH,

=JeffH

---

From: =JeffH <[email-masked]>
Date: Tue, 19 Oct 2010 10:24:24 -0700

a further note on the regex..

regex aficionados will note that there are subtle issues with the regex -- e.g. 
"([\-\w\*]+\.)+[\-\w]+)" -- the latter component doesn't /precisely/ describe 
DNS LDH domain name syntax. for instance, \w matches underscore, and thus the 
regex matches labels with underscore appearing anywhere in them, but (current) 
DNS LDH domain name syntax allows for underscore in only the first char 
position of a label. The regex also doesn't check for other first-char-of-label 
restrictions, and also matches wildcards anywhere in labels (which is a 
subtlety we've been debating).

However, I feel that for my present back-of-the-envelope quick looksee in the 
subject name data, that thar regex above is Good Enough.

Of course if others want to be more precise, go for it.

=JeffH