Subject: Re: [TLS] BoringSSL's TLS test suite

From: Henrick Hellström <[email-masked]>
Date: Sun, 25 Sep 2016 23:06:18 +0200

Have you noticed that BoringSSL seems to abort handshakes with an 
illegal_parameter alert, if the server certificate uses the standard 
compliant (albeit highly unusual) DER encoding of NULL OPTIONAL as the 
empty string, instead of the non-standard but ubiquitous 0x05 0x00 encoding?

Is this just a regression bug in BoringSSL, or is it an intentional 
restriction of the TLS protocol that should be propagated to other 
implementations as well?

On 2016-08-16 20:08, David Benjamin wrote:
> Hi folks,
>
> BoringSSL has developed a test harness[1] that consists of a fork of
> Go’s crypto/tls package (recently dubbed “BoGo" at the Berlin hackathon)
> plus a test runner that allows BoGo to be run against the TLS stack
> under test. BoGo can be configured to behave in a number of unexpected
> ways that violate the TLS standard, thus enabling the testing of many
> scenarios that would be otherwise difficult to obtain with a standard
> stack. We (David Benjamin and Eric Rescorla) have been getting it to
> work with NSS and wanted to let others know in case they might find it
> useful.
>
> This system was initially designed to work with BoringSSL, but in
> principle can be used with any stack. The portability is still a little
> rough, and we'll likely make changes as we get more experience here, but
> it has already been used to test NSS[2] and OpenSSL[3]. We've written up
> some notes at [4].
>
> The test suite should be fairly extensive for DTLS and TLS 1.2 (giving
> around 75% line coverage in BoringSSL’s TLS code at last count). It
> tests TLS 1.3 as well, though those tests are still in progress. They
> track BoringSSL’s in-progress TLS 1.3 implementation.
>
> David and Eric
>
> [1] https://boringssl.googlesource.com/boringssl/+/master/ssl/test/
> [2] https://hg.mozilla.org/projects/nss/file/tip/external_tests/nss_bogo_shim
> [3] https://github.com/google/openssl-tests
> [4] https://boringssl.googlesource.com/boringssl/+/master/ssl/test/PORTING.md
>
>
> _______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls
>

---

From: Adam Langley <[email-masked]>
Date: Sun, 25 Sep 2016 14:19:41 -0700

On Sun, Sep 25, 2016 at 2:06 PM, Henrick Hellström <[email-masked]> wrote:
> compliant (albeit highly unusual) DER encoding of NULL OPTIONAL as the empty
> string, instead of the non-standard but ubiquitous 0x05 0x00 encoding?

In what location do you have an omitted NULL? (I suspect that
BoringSSL's behaviour is deliberate, but I can't say for sure without
knowing which case it is.)

Cheers

AGL

---

From: David Benjamin <[email-masked]>
Date: Sun, 25 Sep 2016 21:23:09 +0000

Do you mean in RSA SubjectPublicKeyInfos? For those, such encodings are not
actually standards-compliant. Per RFC 3279, 2.3.1:

   The rsaEncryption OID is intended to be used in the algorithm field
   of a value of type AlgorithmIdentifier.  The parameters field MUST
   have ASN.1 type NULL for this algorithm identifier.

https://tools.ietf.org/html/rfc3279#section-2.3.1

RFC 4055, section 1.2 also says:

   The rsaEncryption object identifier continues to identify the subject
   public key when the RSA private key owner does not wish to limit the
   use of the public key exclusively to either RSASSA-PSS or RSAES-OAEP.
   In this case, the rsaEncryption object identifier MUST be used in the
   algorithm field within the subject public key information, and the
   parameters field MUST contain NULL.

https://tools.ietf.org/html/rfc4055#section-1.2

There are other contexts where (due to historical mistakes) specs declared
both are acceptable, but amazingly not RSA SPKIs. BoringSSL has enforced it
for quite some time now, so it seems this part of the specification matches
reality. If I recall, mozilla::pkix enforces this as well?

David

On Sun, Sep 25, 2016 at 5:07 PM Henrick Hellström <[email-masked]>
wrote:

> > [2]
> https://hg.mozilla.org/projects/nss/file/tip/external_tests/nss_bogo_shim
> > [4]
> https://boringssl.googlesource.com/boringssl/+/master/ssl/test/PORTING.md
> >
> >

---

From: Henrick Hellström <[email-masked]>
Date: Sun, 25 Sep 2016 23:23:35 +0200

> knowing which case it is.)

I am referring to the OPTIONAL parameter of AlgorithmIdentifier, which 
is set to NULL e.g. when algorithm is rsaEncryption, 
sha256WithRsaEncryption or id-sha256.

---

From: Henrick Hellström <[email-masked]>
Date: Sun, 25 Sep 2016 23:35:14 +0200

On 2016-09-25 23:23, David Benjamin wrote:
> Do you mean in RSA SubjectPublicKeyInfos? For those, such encodings are
> not actually standards-compliant. Per RFC 3279, 2.3.1:
>

Then again, the ASN.1 module in 
https://datatracker.ietf.org/doc/rfc5280/ says differently. Strictly 
speaking, RFC 3279 does not override the PKIX specification when it 
comes to X.509 certificates; only for formats such as RSA PUBLIC KEY 
that rely solely on the ASN.1 module in RFC 3279.

> There are other contexts where (due to historical mistakes) specs
> declared both are acceptable, but amazingly not RSA SPKIs. BoringSSL has
> enforced it for quite some time now, so it seems this part of the
> specification matches reality. If I recall, mozilla::pkix enforces this
> as well?

Actually no, just tested Chrome v53 and Firefox v49 for the Windows 
platform. Only Chrome fails when connecting to a server that presents a 
server certificate with the NULL parameters omitted from the DER encoding.

---

From: Adam Langley <[email-masked]>
Date: Sun, 25 Sep 2016 14:49:27 -0700

On Sun, Sep 25, 2016 at 2:35 PM, Henrick Hellström <[email-masked]> wrote:
> Then again, the ASN.1 module in https://datatracker.ietf.org/doc/rfc5280/
> says differently. Strictly speaking, RFC 3279 does not override the PKIX
> specification when it comes to X.509 certificates; only for formats such as
> RSA PUBLIC KEY that rely solely on the ASN.1 module in RFC 3279.

To answer your original question then, this is intentional.

While there are certainly differences of opinion about the
applicability of Postel's law in this space, in practical terms
requiring a NULL in this location empirically has very good
compatibility and we don't like adding flexibility without good
reason.

Cheers

AGL

-- 
Adam Langley [email-masked] https://www.imperialviolet.org

---

From: David Benjamin <[email-masked]>
Date: Sun, 25 Sep 2016 21:55:36 +0000

On Sun, Sep 25, 2016 at 5:49 PM Adam Langley <[email-masked]> wrote:

> https://datatracker.ietf.org/doc/rfc5280/

I believe we are also correct per spec. My interpretation of these
documents is that the general AlgorithmIdentifier structure may or may not
include parameters. However, whether a given parameter value or omitting
parameters altogether is legal is a question for the particular algorithm.
It's not overriding but plugging into the general framework. Specifications
for individual algorithm OIDs say what to put in the parameters field:

For id-rsaEncryption, RFC 3279 (previously cited) says it is not optional
and the only legal value is NULL.
https://tools.ietf.org/html/rfc3279#section-2.3.1

For id-RSASSAS-PSS, RFC 4055 says the parameters are not optional and
specifies the structure.
https://tools.ietf.org/html/rfc4055#section-3.1

For id-ecPublicKey, RFC 5480 says something similar and gives the structure.
https://tools.ietf.org/html/rfc5480#section-2.1.1

For id-dsa, RFC 3279 says they are actually optional. (Although this was a
pretty bad idea since it's used in this bizarre parameter inheritance
thing...)
https://tools.ietf.org/html/rfc3279#section-2.3.2

None of these can be talking about standalone structures like RSAPublicKey.
Those structures usually don't even include AlgorithmIdentifiers (no need
when you already know the key type) and are usually specified in other
documents like RFC 3447.

David

---

From: Jim Schaad <[email-masked]>
Date: Sun, 25 Sep 2016 16:29:59 -0700

> -----Original Message-----
> From: TLS [mailto:[email-masked]] On Behalf Of Henrick Hellström
> Sent: Sunday, September 25, 2016 2:35 PM
> To: David Benjamin <[email-masked]>; [email-masked]
> Subject: Re: [TLS] BoringSSL's TLS test suite
> 
> > Do you mean in RSA SubjectPublicKeyInfos? For those, such encodings
> > are not actually standards-compliant. Per RFC 3279, 2.3.1:
> >
speaking, RFC
> 3279 does not override the PKIX specification when it comes to X.509
> certificates; only for formats such as RSA PUBLIC KEY that rely solely on
the
> ASN.1 module in RFC 3279.

The ASN.1 module in RFC 5280 does not say anything about if the field is
optional for any specific algorithm.  The ASN.1 for algorithm identifier is

AlgorithmIdentifier  ::=  SEQUENCE  {
        algorithm               OBJECT IDENTIFIER,
        parameters              ANY DEFINED BY algorithm OPTIONAL  

This very explicitly says that the value (and hence presence) of the
parameters fields is strictly defined by the algorithm identifier.  The
algorithm identifiers for RSA with the SHA2 algorithms explicitly say they
are required.

RFC 5912 shows that this is required with the way it defines the same
information

  sa-sha256WithRSAEncryption SIGNATURE-ALGORITHM ::= {
       IDENTIFIER sha256WithRSAEncryption
       PARAMS TYPE NULL ARE required
       HASHES { mda-sha256 }
       PUBLIC-KEYS { pk-rsa }
       SMIME-CAPS { IDENTIFIED BY sha256WithRSAEncryption }
   }

You can see that the parameters are required and not optional.

Jim

> > declared both are acceptable, but amazingly not RSA SPKIs. BoringSSL
> > has enforced it for quite some time now, so it seems this part of the
> > specification matches reality. If I recall, mozilla::pkix enforces
> > this as well?
> 
platform.
> Only Chrome fails when connecting to a server that presents a server
certificate
> with the NULL parameters omitted from the DER encoding.
>

---

From: Henrick Hellström <[email-masked]>
Date: Mon, 26 Sep 2016 01:34:37 +0200

On 2016-09-25 23:55, David Benjamin wrote:
> documents is that the general AlgorithmIdentifier structure may or may
> not include parameters. However, whether a given parameter value or
> omitting parameters altogether is legal is a question for the particular
> algorithm. It's not overriding but plugging into the general framework.

THe ITU-T X.690 standard for DER might require some close reading to 
interpret, and this is obviously a topic for debate. I would presume 
that the use of "default" in lower caps in section 11.5, refers to both 
the DEFAULT and the OPTIONAL keyword:

   11.5	Set and sequence components with default value
   The encoding of a set value or sequence value shall not include an
   encoding for any component value which is equal to its default value.

After all, this is the only interpretation that is consistent with the 
description in section 12.5 of DER as unambiguous.

Since NULL is always empty, it should be omitted when OPTIONAL, given 
the above interpretation. But is it optional? The 1988 syntax did not 
feature information objects and classes, hence the use of the ANY keyword.

Luckily the ASN.1 modules were updated in RFC 5912 to modern ASN.1 
syntax. There you have these declarations:

--  SIGNATURE-ALGORITHM
--
--  Describes the basic properties of a signature algorithm
--
--  &id - contains the OID identifying the signature algorithm
--  &Value - contains a type definition for the value structure of
--              the signature; if absent, implies that no ASN.1
--              encoding is performed on the value
--  &Params - if present, contains the type for the algorithm
--               parameters; if absent, implies no parameters
--  &paramPresence - parameter presence requirement
--  &HashSet - The set of hash algorithms used with this
--                  signature algorithm
--  &PublicKeySet - the set of public key algorithms for this
--                  signature algorithm
--  &smimeCaps - contains the object describing how the S/MIME
--              capabilities are presented.
--
--  Example:
--  sig-RSA-PSS SIGNATURE-ALGORITHM ::= {
--     IDENTIFIER id-RSASSA-PSS
--     PARAMS TYPE RSASSA-PSS-params ARE required
--     HASHES { mda-sha1 | mda-md5, ... }
--     PUBLIC-KEYS { pk-rsa | pk-rsa-pss }
-- }

SIGNATURE-ALGORITHM ::= CLASS {
     &id             OBJECT IDENTIFIER UNIQUE,
     &Value          OPTIONAL,
     &Params         OPTIONAL,
     &paramPresence  ParamOptions DEFAULT absent,
     &HashSet        DIGEST-ALGORITHM OPTIONAL,
     &PublicKeySet   PUBLIC-KEY OPTIONAL,
     &smimeCaps      SMIME-CAPS OPTIONAL
} WITH SYNTAX {
     IDENTIFIER &id
     [VALUE &Value]
     [PARAMS [TYPE &Params] ARE &paramPresence ]
     [HASHES &HashSet]
     [PUBLIC-KEYS &PublicKeySet]

AlgorithmIdentifier{ALGORITHM-TYPE, ALGORITHM-TYPE:AlgorithmSet} ::=
         SEQUENCE {
             algorithm   ALGORITHM-TYPE.&id({AlgorithmSet}),
             parameters  ALGORITHM-TYPE.
                    &Params({AlgorithmSet}{@algorithm}) OPTIONAL
         }

    pk-rsa PUBLIC-KEY ::= {
     IDENTIFIER rsaEncryption
     KEY RSAPublicKey
     PARAMS TYPE NULL ARE absent
     -- Private key format not in this module --
     CERT-KEY-USAGE {digitalSignature, nonRepudiation,
     keyEncipherment, dataEncipherment, keyCertSign, cRLSign}
    }

Hence, the correct encoding is for NULL to be absent.

---

From: Henrick Hellström <[email-masked]>
Date: Mon, 26 Sep 2016 01:41:50 +0200

> You can see that the parameters are required and not optional.

Thanks, you are absolutely correct about this, and this crucial for 
getting PKCS #1 v1.5 signatures right (since the algorithm identifier 
encoding is part of the data to be signed), but at the same time, NULL 
should be absent from the RSA public key:

    pk-rsa PUBLIC-KEY ::= {
     IDENTIFIER rsaEncryption
     KEY RSAPublicKey

and this is definitely not common practice.

---

From: Jim Schaad <[email-masked]>
Date: Sun, 25 Sep 2016 17:02:50 -0700

> Sent: Sunday, September 25, 2016 4:35 PM
> To: David Benjamin <[email-masked]>; Adam Langley
> <[email-masked]>
> Cc: [email-masked]
> > omitting parameters altogether is legal is a question for the
> > particular algorithm. It's not overriding but plugging into the general
> framework.
> 
interpret,
> and this is obviously a topic for debate. I would presume that the use of
> "default" in lower caps in section 11.5, refers to both the DEFAULT and
the
> OPTIONAL keyword:

OPTIONAL and DEFAULT are not the same things.  A DEFAULT value is omitted
but not an OPTIONAL value.  A single field cannot be both OPTIONAL and
DEFAULT.

Jim

> After all, this is the only interpretation that is consistent with the
description in
> section 12.5 of DER as unambiguous.
> 
> Since NULL is always empty, it should be omitted when OPTIONAL, given the
> above interpretation. But is it optional? The 1988 syntax did not feature
> information objects and classes, hence the use of the ANY keyword.
> 
> Luckily the ASN.1 modules were updated in RFC 5912 to modern ASN.1 syntax.
> There you have these declarations:
> 
> --  SIGNATURE-ALGORITHM
> --

---

From: Henrick Hellström <[email-masked]>
Date: Mon, 26 Sep 2016 02:10:29 +0200

> but not an OPTIONAL value.  A single field cannot be both OPTIONAL and

My point was that "DEFAULT" is not the same as "default" either, but 
let's leave it there.

You were listed as a co-author on RTC 5912, so maybe you could clarify 
whether the declaration of pk-rsa should still be considered to be correct:

    pk-rsa PUBLIC-KEY ::= {
     IDENTIFIER rsaEncryption
     KEY RSAPublicKey

---

From: Jim Schaad <[email-masked]>
Date: Sun, 25 Sep 2016 17:34:36 -0700

> From: Henrick Hellström [mailto:[email-masked]]
> Sent: Sunday, September 25, 2016 4:42 PM
> To: Jim Schaad <[email-masked]>; 'David Benjamin'
> <[email-masked]>; [email-masked]
> > The ASN.1 module in RFC 5280 does not say anything about if the field
> > is optional for any specific algorithm.  The ASN.1 for algorithm
> > identifier is
> >
> > parameters fields is strictly defined by the algorithm identifier.
> > The algorithm identifiers for RSA with the SHA2 algorithms explicitly
> > say they are required.
> >
> >
> Thanks, you are absolutely correct about this, and this crucial for getting PKCS
> #1 v1.5 signatures right (since the algorithm identifier encoding is part of the
> data to be signed), but at the same time, NULL should be absent from the RSA
> public key:
> 
>     pk-rsa PUBLIC-KEY ::= {
>      IDENTIFIER rsaEncryption
>      KEY RSAPublicKey

No, it appears that I messed this up. (:  It should be required and not absent.

The text in RFC3279 says

The parameters field MUST have ASN.1 type NULL for this algorithm identifier.

Unhappiness.

---

From: Henrick Hellström <[email-masked]>
Date: Mon, 26 Sep 2016 02:46:10 +0200

> No, it appears that I messed this up. (:  It should be required and not absent.

OK, but it is strange. There are older implementations that predate RFC 
5912 by more than a decade that did omit NULL parameters. I know that 
because I encountered them and had to account for them back in 2002, 
iirc. If this is indeed a mistake, you were not the first to make it.

---

From: Jim Schaad <[email-masked]>
Date: Sun, 25 Sep 2016 18:01:21 -0700

> Sent: Sunday, September 25, 2016 5:46 PM
> because I encountered them and had to account for them back in 2002, iirc.
If
> this is indeed a mistake, you were not the first to make it.

Unfortunately true.  The correct value to put at this point is going to be
preferedAbsent or preferedPresent and live with the results because people
and standards have done it both ways for years.

Jim