Subject: [dsfjdssdfsd] getentropy(2)

From: Watson Ladd <[email-masked]>
Date: Mon, 14 Jul 2014 19:54:33 -0700

Dear all,

The absence of getentropy(2) on Linux is a major pain point for
everyone. It turns out that chroot jails are not compatible with
/dev/urandom. which doesn't work on linux anyway (because it will
return junk before initialization). As a TLS developer myself
(slowly!) I feel that pain: random number generation is the single
nastiest problem I have to deal with.

Yes, this is different from the usual IETF standard. But application
and library developers need a portable way to get entropy, and that
has to be the same across all platforms, work every time. Nothing
short of a standard system call will work. Perhaps there is a more
appropriate venue like the Open Group or POSIX or the Cxx committee
(no doubt C++ will happily adopt it: a feature not in C++ is always a
bug).

That's all I need: a platform and hardware independent means to get
some random numbers.

Sincerely,
Watson Ladd

---

From: Theodore Ts'o <[email-masked]>
Date: Tue, 15 Jul 2014 04:25:07 -0400

> nastiest problem I have to deal with.

I've been talking to Bob Beck about this; and I'm not really convinced
how important the presense or lack of getneoptry(2) really is.  A
couple of high points about our conversation:

1) The question of what to do if /dev is not properly set up is a bit
of a red herring.  The OpenBSD folks have said that this is not a high
priority for them.  My perspective is that given that the ELF loader
requires that /dev/null be present so it can mmap in zero pages,
worrying about what to do if /dev is not properly set up is probably
not that big of a deal, and aborting if /dev/[u]random is not present is
probably the right thing to do.

2) OpenBSD's primary concern is to protect against the case when the
attacker has consumed all available file descriptors, so the attempt
to open /dev/[u]random fails.  Whether this is something that
justifies a new system call is a bit of a philosophical question.
After all, if the system is under such an attack, it's likely that
many other things will fail, and so failing hard may be better than
failing in a soft fashion.  There may be other portions of the system
that aren't properly checking for these sorts of failure.  For
example, are applications or libraries that try opening files such as
/etc/hosts.deny, if they receive an error, are they distinguishing
between ENOENT and EMFILE or ENFILE?  Or are the applications just
saying, "gee, open("/etc/hosts.deny", O_RDONLY) returned -1, all is
good, let's proceed"?

So points (1) and (2) means that from a philosophical point of view,
you can certainly make the argument that it's actually better to try
opening /dev/[u]random, and if it fails for any reason, it's better to
hard fail the application with a LOG_ERR or LOG_CRIT syslog followed
by an abort() call.

3) OpenBSD's man page for getentropy(2) talks about "seed grade"
entropy, but this is a bit of a misnomer for those people who are
obsessed about the difference between a DRBG and NRBG in SP 800-90A
speak.  Hence, getentropy(2) is non-blocking, and in practice, never
fails (or at least applications are written assuming that it
non-blocks and never fails).

I'm willing in practice to try seeing if I can get consensus for
adding a system call that works like getentropy(2).  I'm probably
going to add a flags field so that the application can select between
a blocking /dev/random read versus a non-blocking /dev/urandom read.
However, it's not clear that enough people will believe that adding a
system call just to justify allowing the userspace arc4random pool to
be able to initialize, only to have some other part of the application
fail due to a file open failing, or worse, silently compromising
security due to an open of some file such as /etc/hosts.deny failing,
is really worth adding a whole new system call.

> (no doubt C++ will happily adopt it: a feature not in C++ is always a

What I'd much rather see is a standard library that *all* applications
can rely upon and use to get entropy.  I don't trust most application
programmers to write a userspace library correctly.  One approach is
to simply advise all applications to use something like getentropy(2)
directly --- even if all that is needed is random padding or IV's.
The problem is that if you do this, for systems that are trying make
some kind of distinction between DRBG and NRBG, you can very quickly
exhaust the supply of hardware-derived entropy.

I accept that there is significant disagreement about whether such
differences should be made, but to the extent that Intel felt it was
necessary to add an RDSEED instruction as well as RDRAND, it's clear
that the difference is something that at least some people care about.
And I suspect it's unfair to say that the only people who care are the
crazies^H^H^H^H^H^H^H engineers who think that FIPS certification
makes any sense other than a purely commercial one (i.e., being able
to sell to the US Government).

But if we get all applications to use the same library, we can
abstract away not only differences in operating system but also
security policies vis-a-vis DRBG/NDRBG blocking/nonblocking.  So what
*I* would prefer is a library interface where the application declares
what it wants the random numbers for:

* Monte carlo simulations
* Padding
* IV
* Session key
* long-term key

etc.  The library can then decide whether or not the overall system
policy should force application to block when generating long-term
keys, ala gpg, or whether it should getting non-blocking entropy
directly from the kernel, or whether using a userspace DRBG which is
initialized from kernel-supplied entropy is the right answer.

Basically, I don't want to leave this choice up to the application
writer, since many application writers won't be competent to make this
choice, and having consistency across different applications which are
conform to the organization's designated security officer seems to be
something that at least some organizations would want.

Cheers,

						- Ted

---

From: Benjamin Kaduk <[email-masked]>
Date: Tue, 15 Jul 2014 08:39:07 -0400 (EDT)

> * Monte carlo simulations
[...]
> something that at least some organizations would want.

I'm not confident that [all] application writers will even be competent to 
correctly choose amongst the 5 listed uses [plus whatever others might be 
added].  If it stays a small number of easily identified things, it might 
still be better than only exposing the blocking/nonblocking-ness directly, 
but it doesn't seem clear-cut to me.

BTW, FreeBSD exposes a sysctl MIB (CTL_KERN.KERN_ARND) to get entropy 
directly, saving syscalls over open("/dev/[u]random")/read()/close().

-Ben

---

From: Theodore Ts'o <[email-masked]>
Date: Tue, 15 Jul 2014 09:13:21 -0400

> but it doesn't seem clear-cut to me.

Well, a large number of these uses should be done in the crypto
library -- since if the application writer can't tell the difference
between an IV and a session key, I'm not sure I want that person
anywhere near crypto code...

> directly, saving syscalls over open("/dev/[u]random")/read()/close().

That only matters if you want to encourage applications to be
constantly asking the kernel to generate numbers, though, yes?  I'm
arguing that it might be better to have a standardized userspace
library, ala OpenBSD's arc4random, which takes care of all of the OS
specific issues, and also provides a crypto-based DRBG.  In that case,
it saves even more syscalls, since it only needs to read entropy from
the kernel at application startup time, and perhaps periodically every
so often when it wants to reseed, but not every single time you need
crypto-sensitive padding, IV's, session keys, etc.

Cheers,

						- Ted

---

From: Watson Ladd <[email-masked]>
Date: Tue, 15 Jul 2014 07:09:05 -0700

> anywhere near crypto code...

What is the difference in how random they should be? There isn't one,
and making it exposable in system policy invites
all sorts of issues.

> crypto-sensitive padding, IV's, session keys, etc.

getentropy(2) could  be getentropy(3). What's more important is what I
want it to do
- Block if the random system is not initialized (or return a return
value indicating failure)
- Return random numbers, even if I've forked or have a multithreaded process
- Do so promptly

Currently on Linux none of the available options work. Forking
requires a post-fork call, /dev/random blocks constantly, and
/dev/urandom
does not block on startup. I'm probably going to open /dev/urandom:
it's the best current alternative.

Perhaps arc4random is better to imitate: I don't really care. I just
need it to work and ship yesterday.

Sincerely,
Watson Ladd
(And don't use RC4 for it!)
>
> Cheers,
>
>                                                 - Ted
>
> _______________________________________________
> dsfjdssdfsd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dsfjdssdfsd

-- 
"Those who would give up Essential Liberty to purchase a little
Temporary Safety deserve neither  Liberty nor Safety."
-- Benjamin Franklin

---

From: Alyssa Rowan <[email-masked]>
Date: Tue, 15 Jul 2014 19:10:07 +0100

On 15/07/2014 14:13, Theodore Ts'o wrote:

> I'm arguing that it might be better to have a standardized
> userspace library, ala OpenBSD's arc4random, which takes care of
> all of the OS specific issues, and also provides a crypto-based
> DRBG.  In that case, it saves even more syscalls, since it only
> needs to read entropy from the kernel at application startup time,
> and perhaps periodically every so often when it wants to reseed,
> but not every single time you need crypto-sensitive padding, IV's,
> session keys, etc.

I agree with that entirely - that should definitely be in the libc (we
need to take special care to ensure threads and forks never share a
context of this), and that is of course how arc4random(3) works.

We then have the thorny issue of how to seed it. Failure to seed
should definitely be considered a fatal error (OpenBSD bails with
SIGKILL).

The existing Linux RNG, though it has stood up well to the test of
time if I may say so, does not quite meet the needs of that use as it
stands.

Linux /dev/random estimates incoming entropy and blocks on warmup, but
cools down on output, assuming entropy is removed from the pool when
used. Many non-interactive systems without hardware TRNGs have
insufficient entropy to keep the pool from blocking frequently: it
would be impractical to make this a fatal error.

Linux /dev/urandom is the same, but never blocks - even if the
generator has not received sufficient initial seeding. This is a
problem which has resulted in many predictable keys in the wild,
especially in embedded devices (which are notoriously challenging
environments for entropy collection in the absence of a TRNG, and
which obviously did not collect enough at startup and just winged it).

What Linux doesn't seem to have is something in the middle; that
blocks until it's safely warmed up, but then doesn't need to block
again. That's the approach used by OpenBSD (as far as I know?). It
stops someone calling it before it's seeded, but also has no concept
of 'entropy exhaustion' as the DRBG is secure (hopefully!).

The fact that reading from Linux's RNG currently requires device file
accesses means access is not atomic and is dependent upon the finite
resources of file descriptors, as the libReSSL developers have clearly
pointed out.

I agree a better approach would be to have a syscall such as
getentropy(2) which:
- - requires no special privileges
- - provides a small amount of data into a provided buffer from a kernel
  CSPRNG (40-256 bytes, ish)
- - iff it has been correctly seeded beforehand with sufficient collected
  entropy
- - returns an error (ENOENT?) if it has never been insufficiently seeded
  to operate securely
- - otherwise does not error (except if an invalid buffer or byte count
  is given) or block
- - does not reduce the entropy estimate in the accumulated buffer (the
  DRBG should be cryptographically secure so this is not a problem)
- - is suitable for seeding arc4random(3) or another CSPRNG, or for
  direct use if only small quantities are required
- - is atomic and uninterruptible

and then libc can use that to seed arc4random(3) using ChaCha20 or
another very good DRBG (i.e. not RC4!) and hopefully everyone can swim
in a sea of quite fast, long-term-key-grade random numbers.

If a blocking/pure-entropy mode analogous to /dev/random is required,
I think the best approach would be to run a second CSPRNG in parallel
which does reduce the entropy count when queried - and its first
secure output could be used to kickstart the main kernel CSPRNG, which
would be a reasonable implementation in practice I think.

I don't think a mode analogous to current /dev/urandom should be
provided, because it can be run completely unseeded. If someone wants
a non-cryptographic PRNG, they can do it much faster in user-space.

Of course, that is just my opinion, this isn't LKML but more of a case
study/general discussion here, and I am aware over the decade the
generator has been in Linux you've seen _plenty_ of opinions. :)

- -- 
/akr

---

From: Theodore Ts'o <[email-masked]>
Date: Tue, 15 Jul 2014 17:06:51 -0400

> and making it exposable in system policy invites

Some people believe, quite passionately, that there should be a
difference in at least some of those use cases.  NIST believes that
there is a difference between what is output by a DRBG and what a NRBG
which is used to seed a DRBG.  Presumably if you need to sell into the
US government market, you'll be forced to care as well, just as
companies who have US government customers are forced to deal with
FIPS whether or not it is actively harmful to security.

And I would *hope* that people would agree there should be a
difference between what is needed for a Monte Carlo simulation and
other various cryptographic use cases.  (I've gotten bug reports from
people who insisted on using /dev/urandom for Monte Carlo simulations,
and who then complained when it was too slow for their purposes....)

So my including Monte Carlo simulations was quite deliberate --- I can
pretty much guarantee some cluess graduate student will come across
the interface, and decide to use it, and if we list "monte carlo
simulations" as one of the options, hopefully they will chose it and be happy.

	     	       	   	    - Ted

---

From: Watson Ladd <[email-masked]>
Date: Tue, 15 Jul 2014 22:03:51 -0700

> FIPS whether or not it is actively harmful to security.

OpenBSD supplies both getentropy(2) and arc4random(3). I need at least
one of these.
Make getentropy(2) the seed, and arc4random(3) the RNG. Also, just
because you have a portable
method not suitable for everyone doesn't mean you can't have a
nonportable one for niche uses like
getting seeds.

The problem is only partly /dev/urandom vanishing in chroots. It's the
fact that every system has
a different way to do things, so cross-platform compatibility is
difficult. Just stick with a good idea,
and use arc4random(3).

> and who then complained when it was too slow for their purposes....)

If the library lives in userspace it can be fast enough, even while
using AES or HMAC to generate the random samples.

Sincerely,
Watson Ladd