Subject: [hrpc] Implications of unencrypted metadata in encrypted data/protocols

From: Bryan Ford <[email-masked]>
Date: Mon, 2 Nov 2015 06:25:27 +0900

Hi folks, I have a “where does this belong” type question.

The IETF defines a lot of encrypted protocols and data formats, but currently standard design practices tend to leave unencrypted metadata “around” the encrypted content - such as magic numbers, protocol/format version identifiers, cryptographic scheme identifiers, information about the identities, keys, or certificates of the intended recipient(s) or authentication partner(s), etc.  This leakage of assumed-to-be-innocuous metadata, as we’ve been learning, can have surprising and unexpected implications, including for human rights.

For example, since OpenPGP-encrypted and many other encrypted file formats are basically self-identifying via helpful magic numbers and algorithm-IDs etc, it is trivial for “authorities” of whatever kind - e.g., while flying through an airport - to scan a hard drive, discover that you have some PGP-encrypted documents, and decide that by itself is a reason for suspicion and demand your keys (even if the documents might not even be encrypted for you).

This metadata leakage can have other more subtle implications on identification vs anonymity as well.  For example, as far as I know neither OpenPGP nor other encrypted-file formats/protocols like miniLock (https://github.com/kaepora/miniLock <https://github.com/kaepora/miniLock>) go to any effort to hide the number of recipients a file is encrypted for (i.e., the number of public keys for which an internal session key is encrypted).  But if an encrypted file is found on someone’s hard drive encrypted to exactly 23 recipients, and “the authorities” happen to know that a particular mailing list or dissident group has 23 members, the intended “group” and purpose of the document might be obvious even if the individual recipients’ keys are unknown or blinded (as miniLock does).

This seems like a cross-cutting class of issues potentially affecting many protocols.  For example, TLS currently makes no effort at hiding encrypted chunk header metadata, hence revealing all chunk sizes/boundaries and making fingerprinting-based traffic analysis easier.  Similarly, protocols that negotiate or specify one of several ciphers tend to do that negotiation in cleartext, leading not only to the well-known downgrade-attack issue but also attaching a “crack me!” flag to any file or communication that might for whatever reason happen to use a weak cipher such as RC4.  With all the RC4-encrypted streams out there conveniently labeled “RC4”, attackers don’t even have to invest any computational effort to focus RC4-cracking attention on those RC4 streams; if they weren’t so conveniently labeled, one might envision attackers might at least have to do a not-completely-trivial amount of computation to distinguish (say) a strong AES-encrypted stream from a weak RC4-encrypted stream.

I brought up this metadata leakage issue recently in the context of OpenPGP in section 2.3 of a (very preliminary first-cut) draft I wrote for the OpenPGP WG:

	Title: Modernizing the OpenPGP Message Format
	URL: https://datatracker.ietf.org/doc/draft-ford-openpgp-format/ <https://datatracker.ietf.org/doc/draft-ford-openpgp-format/>

But I’m wondering if it’s worth thinking about and discussing these issues more broadly (or if this is already being done) in the context of HRPC or elsewhere.  Thoughts welcome.

Bryan

---

From: Niels ten Oever <[email-masked]>
Date: Mon, 02 Nov 2015 05:40:31 +0100

Hi Bryan,

Great, I have been looking for exactly something like this: a protocol
transcendent issue, that is not yet addressed in an existing RFC
and/or design principle.

I think this metadata problem is real good example of something we
would want to cover, and it's not covered by RFC7258 on pervasive
monitoring.

The only thing we need before we can take this in is a freedom of
expression, freedom of association, or other rights angle, that
ideally is not (only) privacy or security.

Maybe we can discuss this in the hrpc session tomorrow?

Best,

Niels

Niels ten Oever
Head of Digital

Article 19
www.article19.org

PGP fingerprint    8D9F C567 BEE4 A431 56C4
                   678B 08B5 A0F2 636D 68E9

On 11/01/2015 10:25 PM, Bryan Ford wrote:
> The IETF defines a lot of encrypted protocols and data formats,
> but currently standard design practices tend to leave unencrypted
> metadata “around” the encrypted content - such as magic numbers,
> protocol/format version identifiers, cryptographic scheme
> identifiers, information about the identities, keys, or
> certificates of the intended recipient(s) or authentication
> partner(s), etc.  This leakage of assumed-to-be-innocuous metadata,
> as we’ve been learning, can have surprising and unexpected 
> implications, including for human rights.
> 
> For example, since OpenPGP-encrypted and many other encrypted file 
> formats are basically self-identifying via helpful magic numbers
> and algorithm-IDs etc, it is trivial for “authorities” of whatever
> kind - e.g., while flying through an airport - to scan a hard
> drive, discover that you have some PGP-encrypted documents, and
> decide that by itself is a reason for suspicion and demand your
> keys (even if the documents might not even be encrypted for you).
> 
> This metadata leakage can have other more subtle implications on 
> identification vs anonymity as well.  For example, as far as I
> know neither OpenPGP nor other encrypted-file formats/protocols
> like miniLock (https://github.com/kaepora/miniLock) go to any
> effort to hide the number of recipients a file is encrypted for
> (i.e., the number of public keys for which an internal session key
> is encrypted).  But if an encrypted file is found on someone’s hard
> drive encrypted to exactly 23 recipients, and “the authorities”
> happen to know that a particular mailing list or dissident group
> has 23 members, the intended “group” and purpose of the document
> might be obvious even if the individual recipients’ keys are
> unknown or blinded (as miniLock does).
> 
> This seems like a cross-cutting class of issues potentially
> affecting many protocols.  For example, TLS currently makes no
> effort at hiding encrypted chunk header metadata, hence revealing
> all chunk sizes/boundaries and making fingerprinting-based traffic
> analysis easier.  Similarly, protocols that negotiate or specify
> one of several ciphers tend to do that negotiation in cleartext,
> leading not only to the well-known downgrade-attack issue but also
> attaching a “crack me!” flag to any file or communication that
> might for whatever reason happen to use a weak cipher such as RC4.
> With all the RC4-encrypted streams out there conveniently labeled
> “RC4”, attackers don’t even have to invest any computational effort
> to focus RC4-cracking attention on those RC4 streams; if they
> weren’t so conveniently labeled, one might envision attackers might
> at least have to do a not-completely-trivial amount of computation
> to distinguish (say) a strong AES-encrypted stream from a weak
> RC4-encrypted stream.
> 
> I brought up this metadata leakage issue recently in the context
> of OpenPGP in section 2.3 of a (very preliminary first-cut) draft I
> wrote for the OpenPGP WG:
> 
> Title: Modernizing the OpenPGP Message Format URL:
> https://datatracker.ietf.org/doc/draft-ford-openpgp-format/
> 
> But I’m wondering if it’s worth thinking about and discussing
> these issues more broadly (or if this is already being done) in the
> context of HRPC or elsewhere.  Thoughts welcome.
> 
> Bryan
> 
> 
> 
> 
> _______________________________________________ hrpc mailing list 
> [email-masked] https://www.irtf.org/mailman/listinfo/hrpc
>

---

From: Joseph Lorenzo Hall <[email-masked]>
Date: Tue, 3 Nov 2015 13:51:21 +0900

(unintentional lurker on HRPC here, sorry!)

There are a few works in progress here that I wonder if Bryan would
find meet some of his desires... there's this draft from mnot and a
few of us aimed at SPUD-like metadata extensions and asking for
constrained vocabularies:
https://datatracker.ietf.org/doc/draft-nottingham-transport-metadata-impact/

Ted Hardie and a few of us in the IAB Privacy and Security Program are
writing a "hey, that metadata that was recently encrypted? Don't add
that back outside the unencrypted envelope!":
https://github.com/IAB-PrivSec-program/draft-iab-privsec-metadata-insertion
(I have comments outstanding to Ted on this that I haven't finished
and sent him :/)

Bryan, it sounds like this does some of what you want?

best, Joe

On Mon, Nov 2, 2015 at 1:40 PM, Niels ten Oever <[email-masked]> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Hi Bryan,
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQEcBAEBCAAGBQJWNuk/AAoJEAi1oPJjbWjpvSQH/2GKOS5SovQIPIrTgLqCn9ib
> XcUGBwa436JG1o+KMnf/EdRrCdmYvNSf0y+TqrHb+MSbLb7X1HP3UNMCPH6YHHEe
> YZSpucuOU5vexluoueTlRAaJEnJUHX5hQzrNaPg21DdM0Dx76PmeXWaL2Su47PuW
> dbb1KXmG0WVuuZtfoGx2733z+5kngPsdttYPb+EYFL83fDKIB/do0mI9P3fvgfXE
> 02Gihe8xxRkd3P3MZTl15ayAACng5E7wfLffZJd721X68CJbvdPP0Dg8N8JaAJcj
> 4/g1h0PXzE9FLrGZW0W4XM66VAEHC0uZnAvOt8muhMrfcmArFbjn/nZMtV0q2jk=
> =SuXO
> -----END PGP SIGNATURE-----
>
> [email-masked]
> https://www.irtf.org/mailman/listinfo/hrpc

-- 
Joseph Lorenzo Hall
Chief Technologist
Center for Democracy & Technology
1634 I ST NW STE 1100
Washington DC 20006-4011
(p) 202-407-8825
(f) 202-637-0968
[email-masked]
PGP: https://josephhall.org/gpg-key
fingerprint: 3CA2 8D7B 9F6D DBD3 4B10  1607 5F86 6987 40A9 A871

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 3 Nov 2015 14:04:02 +0900

On Mon, Nov 02, 2015 at 06:25:27AM +0900,
 Bryan Ford <[email-masked]> wrote 
 a message of 243 lines which said:

> For example, as far as I know neither OpenPGP nor other
> encrypted-file formats/protocols like miniLock
> (https://github.com/kaepora/miniLock
> <https://github.com/kaepora/miniLock>) go to any effort to hide the
> number of recipients a file is encrypted for (i.e., the number of
> public keys for which an internal session key is encrypted).

You are not forced to indicate the recipient(s) IDs (RFC 4880, section
5.1 "Zero or more Public-Key Encrypted Session Key packets and/or
Symmetric-Key Encrypted Session Key packets may precede a
Symmetrically Encrypted Data Packet"). If you use GnuPG, the option
throw-keyids won't put the IDs in the encrypted message. It is not the
default and it is labeled as "esoteric option"
<https://www.gnupg.org/documentation/manuals/gnupg-devel/GPG-Esoteric-Options.html>
so few people use it (it has severe usability consequences, see an
example - in french - in
<http://www.bortzmeyer.org/gpg-option-no-keyid.html>).

> analysis easier.

It is a general security/privacy issue, not specific to HR and may be
perpass is the best place to discuss it?

> Similarly, protocols that negotiate or specify one of several
> ciphers tend to do that negotiation in cleartext,

Otherwise, you have an interesting chicken-and-egg problem...