Subject: Re: [pkix] DER encoding in RFC 3161

From: Koichi Sugimoto <[email-masked]>
Date: Mon, 3 Aug 2020 06:05:25 +0000

Hello Phillip and PKIX members,

Thanks for comments.

I understand, ideally better to use strict DER encoding, but we can use BER encoding in practice.

Regards,

Koichi Sugimoto.

From: Phillip Hallam-Baker <[email-masked]> 
Sent: Thursday, July 30, 2020 6:10 AM
To: Koichi Sugimoto <[email-masked]>
Cc: [email-masked]
Subject: Re: [pkix] DER encoding in RFC 3161

I have been doing X.509v3 for 30 years now. I have yet to hear a good answer to the question of why DER encoding is needed in PKIX.

It is a requirement of X.509 but not one that makes the slightest sense in these days when you can store a billion certs on one disk drive and X.500 directory has fewer active users than Edison phonographs. Certificates are moved from point A to point B as an opaque binary blob which is encoded by the signer at signing time and only at signing time. This notion of signing the abstract data rather than the binary bits keeps coming up but I have yet to see anyone work out how to do it reliably, let alone add value by doing so.

A digital signature is over a string of bits. The road to madness is paved by canonicalization schemes.

It is sufficiently possible that there are ASN.1 parsers there that insist on strict DER with definite length encoding throughout that changing PKIX to allow BER encoding would now be a breaking change. And the same argument can be made for cases where BER encoding is indicated. 

So at this point, the best thing we can do is to just accept that the spec is going to be ugly and not try to fix it.

On Tue, Jul 28, 2020 at 3:58 AM Koichi Sugimoto <koichi.sugimoto=[email-masked] <mailto:[email-masked]> > wrote:

Hello PKIX members,

RFC 3161 specifies “The eContent SHALL be the DER-encoded value of TSTInfo.” in “2.4.2. Response Format”

Why RFC 3161 does not require DER-encoded value for full time-stamp token (CMS data)?

On the other hand, following protocol encoding seems to require all DER-encoded for entire time-stamp message.

3..1. Time-Stamp Protocol Using E-mail

3.2. File Based Protocol

3.3. Socket Based Protocol

3.4. Time-Stamp Protocol via HTTP

This seems time-stamp token requires DER-encoded indirectly.

Regards,

Koichi Sugimoto.

_______________________________________________
pkix mailing list
[email-masked] <mailto:[email-masked]> 
https://www.ietf.org/mailman/listinfo/pkix

---

From: [email-masked] (Martin Rex)
Date: Mon, 3 Aug 2020 17:20:55 +0200 (CEST)

Phillip Hallam-Baker <[email-masked]> wrote:
>
> I have been doing X.509v3 for 30 years now. I have yet to hear a good
> answer to the question of why DER encoding is needed in PKIX.
> 
> It is a requirement of X.509 but not one that makes the slightest sense in
> these days when you can store a billion certs on one disk drive and X.500
> directory has fewer active users than Edison phonographs. Certificates are
> moved from point A to point B as an opaque binary blob which is encoded by
> the signer at signing time and only at signing time. This notion of signing
> the abstract data rather than the binary bits keeps coming up but I have
> yet to see anyone work out how to do it reliably, let alone add value by
> doing so.
> 

There seems to be a significant amount of installed base which breaks
horribly if you change the encoding even slightly.  That installed base
maps Certificate based on the certificate fingerprint.

If the outermost encoding of the X.509v3 certificate encoding is changed,
like a proper ASN.1 DER encoding of the PKCS#1 RSA-PSS signature AlgID,
or use of an indefinite length encoding, that installed base implementation
will no longer recognize a TLS client certificate because of the changed
certificate fingerprint.  That the digital signature on that TLS client
certificate verifies just fine, does not seem to matter at all.

That installed base is using a Microsoft Windows 2016 Server, and
was previously using a Microsoft Windows 2008R2 Server.

IIRC there are a number of places in Microsoft CryptoAPI, SChannel and
Microsoft http.sys where the certificate fingerprint is used.

-Martin

---

From: Peter Gutmann <[email-masked]>
Date: Tue, 4 Aug 2020 10:09:21 +0000

Martin Rex <[email-masked]> writes:

>That installed base maps Certificate based on the certificate fingerprint.

Why is this a problem?  In order to make it "break horribly" you'd need to
deliberately re-encode the malleable parts of the certificate in a different
format.  Given the very widespread use of hash fingerprints to uniquely
identify data items I think people know how to work with them by now, the rule
being "don't gratuitously change the data being identified by the hash value".

Peter.

---

From: David Chadwick <[email-masked]>
Date: Wed, 5 Aug 2020 06:49:47 +0100

On 29/07/2020 22:10, Phillip Hallam-Baker wrote:
> It is a requirement of X.509 but not one that makes the slightest 
> sense in these days when you can store a billion certs on one disk 
> drive and X.500 directory has fewer active users than Edison 
> phonographs. Certificates are moved from point A to point B as an 
> opaque binary blob which is encoded by the signer at signing time and 
> only at signing time. This notion of signing the abstract data rather 
> than the binary bits keeps coming up but I have yet to see anyone work 
> out how to do it reliably, let alone add value by doing so.
>

And it is infectious. In the W3C CCG group the canonicalisation issue 
was raised again this week, only this time it is about canonicalising 
the JSON encoding of a public key. I have said it is not needed, but I 
am not convinced that many?anyone? are/is listening.

Kind regards

David

>
>
> It is sufficiently possible that there are ASN.1 parsers there that 
> insist on strict DER with definite length encoding throughout that 
> changing PKIX to allow BER encoding would now be a breaking change. 
> And the same argument can be made for cases where BER encoding is 
> indicated.
>
> So at this point, the best thing we can do is to just accept that the 
> spec is going to be ugly and not try to fix it.
>
>
>
> On Tue, Jul 28, 2020 at 3:58 AM Koichi Sugimoto 
> <koichi.sugimoto=[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hello PKIX members,
>
>     On the other hand, following protocol encoding seems to require
>     all DER-encoded for entire time-stamp message.
>
> [email-masked]

---

From: [email-masked] (Martin Rex)
Date: Mon, 10 Aug 2020 20:10:53 +0200 (CEST)

Peter Gutmann <[email-masked]> wrote:
[ Charset windows-1252 unsupported, converting... ]
> Why is this a problem?

Because it hashes/fingerprints a non-persistent *OUTER* container, rather than ToBeSigned --
the data element for which preproduction in bit-by-bit is clear from the specification.

>
> In order to make it "break horribly" you'd need to deliberately re-encode
> the malleable parts of the certificate in a different format.

No, the breakage occurs when suddenly expecting that malleable parts of an X.509 certificate
(=everyting *OUTSIDE* of ToBeSigned) are guaranteed to be unchanged by whatever storage
and transports a Certificate is going through, including any ASN.1 length encodings on the outside.

> Given the very widespread use of hash fingerprints to uniquely
> identify data items, I think people know how to work with them by now, the rule

Creating a hash over a the outer transport encoding of an object looks like
a terribly dumb idea to me.  In particular when the verification of the certificate shows
that it is still a perfectly valid certificate, and it becomes pretty difficult to determine
why the receipient is failing to recognize it.

This particular server is accepting TLS client certificates, but only when they are conveyed
in a blue wrapper.  It will reject TLS client certificates that are submitted in red, pink
yellow, green or any other perfectly valid wrapping, just because the very first time the
server received this certificate, it had been wrapped in blue.  Unfortunately, the server
will not reveal which particular wrapping color it is stuck on, other than failing for
every other color that the one it is secretly holding.

-Martin

---

From: Peter Gutmann <[email-masked]>
Date: Tue, 11 Aug 2020 02:12:44 +0000

Martin Rex <[email-masked]> writes:

>the breakage occurs when suddenly expecting that malleable parts of an X.509
>certificate (=everyting *OUTSIDE* of ToBeSigned) are guaranteed to be
>unchanged by whatever storage and transports a Certificate is going through,
>including any ASN.1 length encodings on the outside.

Which is the case for everything I know of.  I assume you must have some
special-case situation you've run into where this is an issue, but since
"certificate fingerprints" are pretty much the universal unique identifier for
the things - fire up any cert viewer and you'll see them used, for example -
I'd say that if there is something out there that rewrites certs and breaks
the fingerprint then whatever it is is broken and needs to be fixed.

More to the point, we've been using cert fingerprints for thirty-odd years
without running into any problems, so if something turns up now that breaks
them then that's the problem, not cert fingerprints.

Peter.

---

From: "Manger, James" <[email-masked]>
Date: Tue, 11 Aug 2020 03:45:31 +0000

> since "certificate fingerprints" are pretty much the universal unique identifier for the things - fire up any cert viewer and you'll see them used, for example - I'd say that if there is something out there that rewrites certs and breaks the fingerprint then whatever it is is broken and needs to be fixed.
>
> More to the point, we've been using cert fingerprints for thirty-odd years without running into any problems, so if something turns up now that breaks them then that's the problem, not cert fingerprints.

Such whole-of-cert fingerprints are very widely used, but a slight change of context (from an allowlist to a blocklist) can make them unexpectedly dangerous.
Use whole-of-cert fingerprints in a blocklist ... then watch attackers sail through by tweaking a byte outside the toBeSigned portion.
 
--
James Manger

---

From: Peter Gutmann <[email-masked]>
Date: Tue, 11 Aug 2020 03:58:27 +0000

Manger, James <[email-masked]> writes:

>Such whole-of-cert fingerprints are very widely used, but a slight change of
>context (from an allowlist to a blocklist) can make them unexpectedly
>dangerous.

Sure, but if you're using default-allow (= blocklist), which makes #1 in the
"The Six Dumbest Ideas in Computer Security",
https://www.ranum.com/security/computer_security/editorials/dumb/, then your
security mechanism is broken from the start and you need to fix the mechanism
you're using.

(This was debated years ago on this very list: We can't use cert fingerprints
because they're not bug-compatible with the default-allow brokenness of CRLs
et al, which is one reason why after thirty-odd years we're still permanently
stuck behind attackers who can move faster than our blocklists can catch up).

Peter.