Subject: [dsfjdssdfsd] Any plans for drafts or discussions on here?

From: Stephen Farrell <[email-masked]>
Date: Mon, 20 Jan 2014 21:47:27 +0000

Hiya,

Just a health-check on this list. Anyone got plans?

Ta,
S.

---

From: Donald Eastlake <[email-masked]>
Date: Mon, 20 Jan 2014 17:53:55 -0500

Hi Stephen,

I've been a bit snowed under just recently and this week but I have
accumulated some changes and suggestions on the randomness requirement
sod security draft and do plan to do a revision soon.

Thanks,
Donald
=============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 155 Beaver Street, Milford, MA 01757 USA
 [email-masked]

On Mon, Jan 20, 2014 at 4:47 PM, Stephen Farrell
<[email-masked]> wrote:
> _______________________________________________
> dsfjdssdfsd mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dsfjdssdfsd

---

From: "Dan Harkins" <[email-masked]>
Date: Mon, 20 Jan 2014 16:27:11 -0800 (PST)

Hi Stephen,

> Just a health-check on this list. Anyone got plans?

  Thanks for the prod.

  Are HMAC_DRBG and CTR_DRBG suitable as general purpose
DRBGs? If yes, then do people see value in producing an
Informational RFC describing them? If not, what's the problem
and what's a suitable alternative?

  thanks,

  Dan.

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 20 Jan 2014 20:28:26 -0500

> sod security draft and do plan to do a revision soon.

It would be good to see those revisions. It still feels very wrong for us to be suggesting to application developers that they should be doing their own randomness; they should be asking their OS unless they are experts, and those experts don't need an RFC.

--Paul Hoffman

---

From: "Dan Harkins" <[email-masked]>
Date: Wed, 22 Jan 2014 08:13:14 -0800 (PST)

Paul,

On Mon, January 20, 2014 5:28 pm, Paul Hoffman wrote:
> It would be good to see those revisions. It still feels very wrong for us
> to be suggesting to application developers that they should be doing their
> own randomness; they should be asking their OS unless they are experts,
> and those experts don't need an RFC.

  "Ask your OS" is putting faith in the guy that wrote the relevant code
in your OS. It might be a reasonable leap but it's a leap nevertheless.
Recent events should tell us that we should not trust a single source for
these things (even if we are told that this single source is actually the
output of a bunch of uncorrelated sources of entropy being mixed up).

  I see value in draft-eastlake-randomness3 and I also see value in an
Informational RFC on a good DRBG for those who feel the need to have
a belt as well as suspenders.

  Dan.

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 22 Jan 2014 08:28:41 -0800

On Jan 22, 2014, at 8:13 AM, Dan Harkins <[email-masked]> wrote:

> in your OS.

Yes, exactly.

> It might be a reasonable leap but it's a leap nevertheless.

We put faith in the (~85%) guy for all the other crypto code as well, so I don't see the leap.

> output of a bunch of uncorrelated sources of entropy being mixed up).

That's one interpretation. Another is that attackers will look for bad implementations and use those as best they can.

> a belt as well as suspenders.

We disagree here; the chance that the person writing the belt will get it wrong and make their crypto trivial to break for an attacker who knows the weakness seems much higher to me than the change that the OS got it wrong. Yes, we could put some warning at the front of the new document about this, but that warning will be ignored by programmers who are sure they know this stuff.

I could see writing something that forces them to mix in randomness from the OS to their possibly-borked DRBG in the hopes that at least that step will fix their problems. However, if we do that, nearly all the interesting technical stuff in the current document is just confusing fluff. The new document reduces to "use an HMAC with the randomness from your OS as the key and whatever stuff you think is random as the data; done".

--Paul Hoffman

---

From: Krisztián Pintér <[email-masked]>
Date: Wed, 22 Jan 2014 18:51:49 +0100

Paul Hoffman (at Tuesday, January 21, 2014, 2:28:26 AM):
> It still feels very wrong
> for us to be suggesting to application developers that they should
> be doing their own randomness; they should be asking their OS unless
> they are experts, and those experts don't need an RFC.

(new to the list, and i'm not sure about the scope or context, so
forgive me if i'm talking nonsense.)

i agree that the OS should do the work, for multiple reasons, 1, it is
better done in a protected environment (aka ring 0), 2, OS has access
to more entropy, 3, better have one excellent than many good
solutions.

that said, what is the list of OS's today that has sound, reviewed
RNG? openbsd, ...? so this is pretty thin.

---

From: Donald Eastlake <[email-masked]>
Date: Wed, 22 Jan 2014 13:13:33 -0500

Hi Paul,

>> they are experts, and those experts don't need an RFC.

I don't understand why you think having an RFC means that applications
developers are supposed to implement what is described in that RFC.
The IETF does lots of non-application level RFCs. I don't agree that
it is clear who is an expert in this area. I don't agree that any
person believed to be an expert will, in the absence of documentation,
know or take into account all the aspects of what might be called best
current practice in this area. IETF specifications that call for
quantities unpredictable by adversaries need to reference something.
Should they just reference the NIST documents?

Thanks,
Donald

---

From: "Dan Harkins" <[email-masked]>
Date: Wed, 22 Jan 2014 10:40:34 -0800 (PST)

On Wed, January 22, 2014 8:28 am, Paul Hoffman wrote:
> We disagree here; the chance that the person writing the belt will get it
> wrong and make their crypto trivial to break for an attacker who knows the
> weakness seems much higher to me than the change that the OS got it wrong.
> Yes, we could put some warning at the front of the new document about
> this, but that warning will be ignored by programmers who are sure they
> know this stuff.
>
> I could see writing something that forces them to mix in randomness from
> the OS to their possibly-borked DRBG in the hopes that at least that step
> will fix their problems. However, if we do that, nearly all the
> interesting technical stuff in the current document is just confusing
> fluff. The new document reduces to "use an HMAC with the randomness from
> your OS as the key and whatever stuff you think is random as the data;
> done".

  To minimize the possibility of someone borking their DRBG implementation
we can include some test vectors-- instantiating with A produces state of
B; reseeding of state B with C produces state D; generating with state D
produces E, etc.

  The more people that implement anything will increase the probability of
a broken implementation somewhere, I definitely agree. But it is an accepted
economic truth that relying on a single source for anything is a bad idea.

  Dan.

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 22 Jan 2014 11:12:02 -0800

> developers are supposed to implement what is described in that RFC.

Why else write the RFC? Is it for developers who work on /dev/random in various OSs? If so, there is a whole different set of problems with the document, which we discussed during the round before this.

> The IETF does lots of non-application level RFCs.

Sure, and if this is one of those, you need to say that clearly. That's why I said I wanted to see what changes you were making in the -01.

> I don't agree that
> current practice in this area.

Sure. However, I also don't think the document describes what are "best", much less "current", practices in the area. The permathreads on the cryptography mailing list makes it really clear that there is no agreement on what is "best" even among active crypto developers. They also show that many people don't even know what is "current": Ted Ts'o has had to tell folks a few times that their new ideas are in fact already implemented, at least in Linux.

> IETF specifications that call for

Yes.

> Should they just reference the NIST documents?

Definitely not.

--Paul Hoffman

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 22 Jan 2014 11:16:18 -0800

> B; reseeding of state B with C produces state D; generating with state D

If we could be sure that there was no way for the mechanisms needed to test those vectors could possibly appear in an implementation, that would be great. If we can't be sure of that, we are proposing an implementer include something that could cause a catastrophic failure if those inputs are accidentally left in.

Regardless, that's not what Don's draft does.

> economic truth that relying on a single source for anything is a bad idea.

That seems like a strawman: who says that there is a "single source" of ideas for how to implement randomness? Three large server OSs (Linux, FreeBSD, Windows) all do it differently.

--Paul Hoffman

---

From: "Dan Harkins" <[email-masked]>
Date: Wed, 22 Jan 2014 12:29:04 -0800 (PST)

On Wed, January 22, 2014 11:16 am, Paul Hoffman wrote:
> If we could be sure that there was no way for the mechanisms needed to
> test those vectors could possibly appear in an implementation, that would
> be great. If we can't be sure of that, we are proposing an implementer
> include something that could cause a catastrophic failure if those inputs
> are accidentally left in.

  That seems like an absurdly high bar-- that there is no way someone
could so something demonstrably stupid. We use test vectors when
specifying things like HKDF or AES-<pick your mode> and no one
seems to generate a function that only accepts those inputs and only
produces the specified outputs. I really don't think people will suddenly
start  hard-coding test vectors into code now.

> Regardless, that's not what Don's draft does.

  I'm still hoping that an Informational RFC on a good DRBG will be
generated.

> That seems like a strawman: who says that there is a "single source" of
> ideas for how to implement randomness? Three large server OSs (Linux,
> FreeBSD, Windows) all do it differently.

  The single source is "your OS". While there are 3 large server OSs, there
is no single application that can obtain randomness from all three since
there is no application that runs on all of them (and my experience is
that a single linux application won't necessarily work from version X of
the OS to version Y-- hell, sometimes the source won't even compile).

  And you know, the "if we could be sure there was no way…" thing works
both ways, especially regarding a large (monopoly) OS whose track
record on security and code quality is not perfect.

  Dan.

---

From: Paul Hoffman <[email-masked]>
Date: Wed, 22 Jan 2014 14:15:06 -0800

On Jan 22, 2014, at 12:29 PM, Dan Harkins <[email-masked]> wrote:

> produces the specified outputs.

Note the difference with "HKDF or AES-<pick your mode>" and a DBRG. In the former case, if they only accepted the test vectors for input, they would never interoperate. In the latter case, if they only accepted those inputs, they generator would still work just fine; it would also be completely predictable.

> I really don't think people will suddenly

They already frequently do exactly that. I have seen multiple implementations have a FIPS 140 self-test mode built in.

>  I'm still hoping that an Informational RFC on a good DRBG will be

In your mind, who is the intended reader? Implementers of the good DRBG in an OS, or in an application, or both?

>  The single source is "your OS".

Ah, got it. Are you saying "if you are going to re-implement crypto primitives and not trust your OS, and one of the ones you are re-implementing is the DBRG, here is a best practice for how to do it"?

> While there are 3 large server OSs, there

Fully agree. We should definitely not say "X is a best practice for DBRG" where X is a single value.

---

From: "Dan Harkins" <[email-masked]>
Date: Wed, 22 Jan 2014 15:36:01 -0800 (PST)

> an OS, or in an application, or both?

  If an OS developer feels he needs a DRBG then, yes, this would be for
him. And depending on the application, yes this would be for that
developer too.

> Ah, got it. Are you saying "if you are going to re-implement crypto
> primitives and not trust your OS, and one of the ones you are
> re-implementing is the DBRG, here is a best practice for how to do it"?

  It's not necessarily that you don't trust the OS, it's that you don't trust
that to be your sole source.

  Dan.

---

From: Francis Dupont <[email-masked]>
Date: Thu, 23 Jan 2014 00:55:41 +0100

>  We put faith in the (~85%) guy for all the other crypto code as well,

=> another argument is the kernel has access to more resources/entropy
sources than a user mode code. So I tend to agree with Paul: if you have
to trust something the OS is not the worst choice.

Regards

[email-masked]

---

From: Francis Dupont <[email-masked]>
Date: Thu, 23 Jan 2014 01:25:59 +0100

In your previous mail you wrote:

>  there is no application that runs on all of them

=> I know many applications which run on these 3 OSs (and more, i.e.,
Solaris, AIX and HP-UX too if you limit to still alive OSs).
So either I didn't understand you or we are not talking about the same
thing...

>  (and my experience is

=> change your application developer?

Regards

---

From: "Dan Harkins" <[email-masked]>
Date: Wed, 22 Jan 2014 17:36:31 -0800 (PST)

> So either I didn't understand you or we are not talking about the same

  If you take the executable from Solaris and copy it onto your HP-UX
machine it will not run.

  My point was that there were not 3 sources (Paul mentioned 3)
available to an application, there is just one: the one that's part of
OS that the application happened to be compiled for.

> => change your application developer?

  Ha, ha. Funny. I have considered firing myself several times.

  The issue was not the code I was writing, it is the annoying habit of
linux driver developers to change data structures (the one in question
was for an 802.11 device) in gratuitous ways that cause breakage. I
have never had that problem with freeBSD. In fact, the inability of an
application to just compile from one OS release to another is touted
as a feature by some of the more zealous linux people.

  Dan.

---

From: <[email-masked]>
Date: Wed, 22 Jan 2014 19:54:59 -0800

I'm new to the list, so apologies if I go off the track here.

The problem with relying on the OS is that you're not always sure what the
OS is doing.  Example:  If I run Linux under ESXi on a headless server, with
an SSD for a disk, and I use the Ethernet NIC built into the motherboard,
where is the entropy for /dev/random coming from?  Is the driver for that
particular NIC instrumented to contribute to the entropy pool?  Most people
don't know, and further don't know how to find out.

I have seen embedded Linux products before where the sole source of entropy
was the hard disk.  Problem was, the entire system operated off a RAMdisk,
so guess how much disk activity there was during runtime?  

Those of us who deal with FIPS 140 and Common Criteria are now being asked
to document entropy sources, including providing measurements of how much
entropy is actually there and how fast it can be produced.  Looking at one
Common Criteria protection profile specifically - the VPN client PP - the
developer has the choice of generating cryptographic entropy within the
application, or relying on the underlying platform to supply it.  But the
only way you get to rely on the underlying platform is if that platform
itself has been through a PP evaluation and thus quantified the entropy.  So
if you want to have any hope of passing the evaluation, you're almost forced
into doing something yourself.

The various DRBGs are fine IMHO for handing out bits of random to an
application, and lacking anything better today I would certainly encourage
developers to use one.  But the question is how do we seed the DRBG in the
first place?  

-Jon

-----Original Message-----
From: dsfjdssdfsd [mailto:[email-masked]] On Behalf Of
Krisztián Pintér
Sent: Wednesday, January 22, 2014 9:52 AM
To: Paul Hoffman
Cc: Donald Eastlake; [email-masked]; Stephen Farrell
Subject: Re: [dsfjdssdfsd] Any plans for drafts or discussions on here?

---

From: Francis Dupont <[email-masked]>
Date: Thu, 23 Jan 2014 10:34:38 +0100

>    If you take the executable from Solaris and copy it onto your HP-UX

=> it depends: this is true (i.e., won't run) for an application
written in C/C++, but false (i.e., will run) for an application
written in python or ocaml. So there is another level between
binary and source code compatibilities.
 To come back to the main subject, as an application developer I have
the choice between the OS, the crypto library (e.g., OpenSSL) and
home made to provide a (P)RNG/(D/N)RBG to be used to generate
not predictable (BTW it is nearly always prediction resistance,
not backtracking resistance :-) values (e.g., transaction IDs),
and to generate crypto material (I cite the two because constraints
are very different, for instance for not predictable values
I really want a not blocking solution).

>  OS that the application happened to be compiled for.

=> I should have said "to run over" (BTW with virtualization it is
not so trivial...)

>  as a feature by some of the more zealous linux people.

=> I'd like to join you as an old BSD man in Linux zealot bashing
(e.g., it is not a gratuitous change: they just never read the previous
code until they get conflicts from multiple definitions with the same
name :-) but we shall be off topic...

Regards

---

From: Stephen Farrell <[email-masked]>
Date: Thu, 23 Jan 2014 09:57:18 +0000

(Great to see the discussion re-started, but I guess we can
afford more than one subject line:-)

On 01/23/2014 03:54 AM, [email-masked] wrote:
> to document entropy sources,

First, my sympathies for having to deal with that.

But I do wonder to what extent we're finding such evaluations
really useful. I know they are formal form-filling requirements
in various contexts, but I'm not so sure I'm that comfortable
treating them as a first order requirement when it comes to
things we do in the IETF.

I have seen a number of credible arguments that such schemes,
as applied to crypto implementations, are actually counter-
productive.

So - how important is it that any new work in the IETF on
this topic be consistent with a requirement for implementations
to be evaluated via such schemes?

My take would be that that's not hugely important and should
lose out to "doing the right thing," but given that some folks
do need to suffer such evaluations, we should think about 'em
but treat any evaluation-scheme-specific requirements only as
nice-to-have level requirements.

I expect vendors who are forced into doing it might disagree
though.

S.

---

From: <[email-masked]>
Date: Thu, 23 Jan 2014 08:06:16 -0800

What I thought the topic was originally about was providing guidance to
developers on dealing with randomness, should they choose to do that.  My
point was only that there are valid reasons a developer might be forced to
deal with randomness rather than depend on the OS, and public-sector
certification is one such reason.

I know it sounds like paper pushing, but the people writing Common Criteria
profiles really are trying to get vendors to do the right thing.  They are
also open to feedback from the vendor and developer community, and within
the last year the CC community has started "technical communities" which are
open to participation from anyone - for just that purpose.  So if they are
doing the wrong thing, there is an opportunity to correct them.

In the case of entropy specifically, if you believe what is written here:
https://www.niap-ccevs.org/pp/pp_nd_v1.1-add3.pdf
...it has done some good.  By simply requiring vendors to think about the
problem, it got them to uncover deficiencies and make improvements.  BTW
this is a useful document to read to understand what the government folks
are going after when it comes to entropy.

>to be evaluated via such schemes?

Not important.  The government certification people mandate that vendors
implement IETF standards, not the other way around.  Sometimes they pick
subsets - for example "Product SHALL implement TLS 1.2, but only with
specific ciphersuites (things based on various combinations of AES, RSA,
ECDSA, ECDHE, etc.)"  But no, I don't think we should let their requirements
drive standards activity.

-Jon

--
Jon Green
[email-masked]
http://www.hosed.org

-----Original Message-----
From: Stephen Farrell [mailto:[email-masked]] 
Sent: Thursday, January 23, 2014 1:57 AM
To: [email-masked]; 'Krisztián Pintér'
Cc: [email-masked]
Subject: evaluating stuff (was: Re: [dsfjdssdfsd] Any plans for drafts or
discussions on here?)

S.

---

From: "Jon Green" <[email-masked]>
Date: Thu, 23 Jan 2014 07:59:46 -0800

-Jon

--
Jon Green
[email-masked]
http://www.hosed.org

-----Original Message-----
From: Stephen Farrell [mailto:[email-masked]] 
Sent: Thursday, January 23, 2014 1:57 AM
To: [email-masked]; 'Krisztián Pintér'
Cc: [email-masked]
Subject: evaluating stuff (was: Re: [dsfjdssdfsd] Any plans for drafts or
discussions on here?)

S.

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 23 Jan 2014 10:00:45 -0800

On Jan 23, 2014, at 1:57 AM, Stephen Farrell <[email-masked]> wrote:

> really useful.

Not.

> I know they are formal form-filling requirements

Quite right. The base requirement boils down to "prove that input X gave the DBRG N bits of entropy that could not be known by any external system". That proof is always hand-waving for nearly any typical computer or network device. If the inputs are chosen conservatively enough, you can be confident that you got N unguessable bits, but you cannot prove it.

> as applied to crypto implementations, are actually counter-

Exactly. Vendors tend to copy the claims of other systems that have earlier passed the evaluations, even when the claims do not fully apply to the new system. After a few rounds of this, the claims are meaningless and the vendor is not trying hard enough to get truly random bits.

> nice-to-have level requirements.

Advice on where you might find the bits in typical computers and network boxes is probably useful. Advice about the value of N for input X is actively dangerous.

--Paul Hoffman

---

From: Krisztián Pintér <[email-masked]>
Date: Thu, 23 Jan 2014 21:40:20 +0100

[email-masked] (at Thursday, January 23, 2014, 4:54:59 AM):

> where is the entropy for /dev/random coming from?

you do know what the OS does, you have either a source code, or at
least a statement that everything is fine. you choose the OS with this
knowledge in mind.

it is another issue that in certain systems, the opsys might be not
good enough. but i doubt that you can write RFC that is of any use for
such special systems. plus, it is still advisable that you only
implement your additional entropy collection, and just feed it back to
the OS.

> to document entropy sources,

we need to make clear what the goal is. me being new here, maybe i
just missed it, but i'm not sure what do we want:

a, building a secure system
b, building a system that can be marketed as secure
c, building a FIPS compliant system

because i claim that these are very different. if you want to be FIPS
compliant, you probably need to do a whole sort of silly things that
helps nobody in no way. but again, this won't be too much aided with
an RFC, or any "best practices" document. nor i personally care much.

marketability of a system can be affected by any number of things,
probably boasting a lot of numbered acronyms, like RFCs as well. but
i'm not into this kind of psychology.

but if we aim at security, the OS can provide the best service here.
yeah, windows don't, but really, how secure your application will be
on windows anyway? you don't need to go over that level.

> developers to use one.  But the question is how do we seed the DRBG in the

i certainly agree that we are well equipped with good DRBGs (which
linux still refuses to include, forget about windows).

do you happen to know fortuna, and especially its entropy collection
policy? i think it is kind of a cool concept, except does not really
solve the problem you mentioned before, namely early entropy
gathering.

one more point: it is not only about creating good random. you also
need to protect it. what sources of entropy do you have that other
processes on the same system don't? what information do you leak
during the collection phase? a little bit harder problem than a DRBG.

---

From: Michael Hammer <[email-masked]>
Date: Thu, 23 Jan 2014 20:49:32 +0000

This may get off-topic, but are there good software tools for testing
entropy, 
that could help applications determine if the underlying system is giving
them good input?

Michael Hammer
Principal Engineer
[email-masked]
Mobile: +1 408-202-9291
500 Yosemite Drive Suite 120
Milpitas, CA 95035 USA

-----Original Message-----
From: dsfjdssdfsd [mailto:[email-masked]] On Behalf Of
Krisztián Pintér
Sent: Thursday, January 23, 2014 12:40 PM
To: [email-masked]
Cc: [email-masked]
Subject: Re: [dsfjdssdfsd] Any plans for drafts or discussions on here?

[email-masked] (at Thursday, January 23, 2014, 4:54:59 AM):

> The problem with relying on the OS is that you're not always sure what 
> the OS is doing.  Example:  If I run Linux under ESXi on a headless 
> server, with an SSD for a disk, and I use the Ethernet NIC built into 
> the motherboard, where is the entropy for /dev/random coming from?

you do know what the OS does, you have either a source code, or at least a
statement that everything is fine. you choose the OS with this knowledge in
mind.

it is another issue that in certain systems, the opsys might be not good
enough. but i doubt that you can write RFC that is of any use for such
special systems. plus, it is still advisable that you only implement your
additional entropy collection, and just feed it back to the OS.

> Those of us who deal with FIPS 140 and Common Criteria are now being 
> asked to document entropy sources,

we need to make clear what the goal is. me being new here, maybe i just
missed it, but i'm not sure what do we want:

a, building a secure system
b, building a system that can be marketed as secure c, building a FIPS
compliant system

because i claim that these are very different. if you want to be FIPS
compliant, you probably need to do a whole sort of silly things that helps
nobody in no way. but again, this won't be too much aided with an RFC, or
any "best practices" document. nor i personally care much.

marketability of a system can be affected by any number of things, probably
boasting a lot of numbered acronyms, like RFCs as well. but i'm not into
this kind of psychology.

but if we aim at security, the OS can provide the best service here.
yeah, windows don't, but really, how secure your application will be on
windows anyway? you don't need to go over that level.

> application, and lacking anything better today I would certainly 
> encourage developers to use one.  But the question is how do we seed 
> the DRBG in the first place?

i certainly agree that we are well equipped with good DRBGs (which linux
still refuses to include, forget about windows).

do you happen to know fortuna, and especially its entropy collection policy?
i think it is kind of a cool concept, except does not really solve the
problem you mentioned before, namely early entropy gathering.

one more point: it is not only about creating good random. you also need to
protect it. what sources of entropy do you have that other processes on the
same system don't? what information do you leak during the collection phase?
a little bit harder problem than a DRBG.

---

From: Krisztián Pintér <[email-masked]>
Date: Thu, 23 Jan 2014 23:38:07 +0100

> that could help applications determine if the underlying system is giving

disclaimer: i'm no expert, it is just what i gathered. (i'm pretty
much interested in randomness.)

short answer: no

long answer: in some situations yes. if you are handed a bunch of
data, all you can do is to try different techniques to put an upper
limit on the entropy. for example you can calculate the shannon
entropy assuming independent bits. then you can hypothesize some
interdependence, and see if you can compress the data. you can apply
different lossless compression methods. the better compression you
find puts an upper limit on the entropy. but never a lower limit.

you can only do better if you have an idea about the process that
created the data. for example you might assume that it is mostly
thermal noise. you can assume that thermal noise has some frequency
distribution, or energy or whatever, etc. within this assumption, you
can determine the entropy content by measurements. but at this point,
you are pretty much prone to two errors: 1, what if your assumption is
wrong and 2, what if your physical model overestimates the
unpredictability of the given system. example for the former: the
signal might be largely controllable by an external EM interference,
and then you measure not noise, but attacker controlled data. example
for the latter: a smartass scientist might come up with a better
physical model for thermal noise.

it is also important to note that entropy is observer dependent. we
actually talk about the entropy as seen by the attacker. but it is not
straightforward to assess what is actually visible to an attacker and
what is not. observation methods improve with time.

---

From: Michael Hammer <[email-masked]>
Date: Thu, 23 Jan 2014 23:19:03 +0000

Hmmm...  that makes it sound rather subjective.
If we don't have objective measures, 
then who is to say that one's randomness is better or worse than another?

Understand that user data input may or may not have good entropy.
But, hope that would be the only source of any output entropy, 
given good encryption algorithms and random keys.

Was thinking in terms of how an app with access to alternate random sources,

some which might be from OS or from some software, might choose one over
another.

Michael Hammer
Principal Engineer

-----Original Message-----
From: Krisztián Pintér [mailto:[email-masked]] 
Sent: Thursday, January 23, 2014 2:38 PM
To: Michael Hammer
Cc: [email-masked]; [email-masked]
Subject: Re: [dsfjdssdfsd] Any plans for drafts or discussions on here?

Michael Hammer (at Thursday, January 23, 2014, 9:49:32 PM):
> entropy, that could help applications determine if the underlying 
> system is giving them good input?

disclaimer: i'm no expert, it is just what i gathered. (i'm pretty much
interested in randomness.)

short answer: no

long answer: in some situations yes. if you are handed a bunch of data, all
you can do is to try different techniques to put an upper limit on the
entropy. for example you can calculate the shannon entropy assuming
independent bits. then you can hypothesize some interdependence, and see if
you can compress the data. you can apply different lossless compression
methods. the better compression you find puts an upper limit on the entropy.
but never a lower limit.

you can only do better if you have an idea about the process that created
the data. for example you might assume that it is mostly thermal noise. you
can assume that thermal noise has some frequency distribution, or energy or
whatever, etc. within this assumption, you can determine the entropy content
by measurements. but at this point, you are pretty much prone to two errors:
1, what if your assumption is wrong and 2, what if your physical model
overestimates the unpredictability of the given system. example for the
former: the signal might be largely controllable by an external EM
interference, and then you measure not noise, but attacker controlled data.
example for the latter: a smartass scientist might come up with a better
physical model for thermal noise.

it is also important to note that entropy is observer dependent. we actually
talk about the entropy as seen by the attacker. but it is not
straightforward to assess what is actually visible to an attacker and what
is not. observation methods improve with time.

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 23 Jan 2014 16:01:45 -0800

> Hmmm...  that makes it sound rather subjective.

The value of an individual bit from an entropy source is subjective.

> then who is to say that one's randomness is better or worse than another?

Once there is agreement on the amount of entropy in a given set of initial bits, there are objective measures about the effectiveness of ways to distribute them.

> given good encryption algorithms and random keys.

Adding additional sources of output entropy, if done correctly, is *always* at least as good as only using the initial bits.

--Paul Hoffman

---

From: "Henderickx, Wim (Wim)" <[email-masked]>
Date: Fri, 24 Jan 2014 04:52:48 +0000

ok

---

From: Krisztián Pintér <[email-masked]>
Date: Fri, 24 Jan 2014 18:02:25 +0100

> then who is to say that one's randomness is better or worse than another?

as i said, we need to examine the physical processes. the best source
of entropy as of now is thermal noise. we understand thermal noise to
a great degree, we don't expect sudden breakthrough in modeling it,
and it is relatively abundant and easy to access. user input also
contains noise, as the user can control keystroke or mouse movement
timing up to some 10's or 100's of milliseconds, below that, it is
just noise from the equipment and the "biological equipment".

> some which might be from OS or from some software, might choose one over

if you are adamant on doing homebrewed, why choose? you can combine
them. if your combinator is good, you can't lose.

---

From: "Dan Harkins" <[email-masked]>
Date: Fri, 24 Jan 2014 11:16:09 -0800 (PST)

> them. if your combinator is good, you can't lose.
              ^^^^^^^^^^^^^^^^^^^^

  Is that all there is to it? This sounds like only the generation function
of a random bit generator. Shouldn't there also be some process that
handles the internal state necessary to do the generation? Shouldn't
that process have certain security properties, for instance allowing the
continued generation of a random bit stream* when an attacker is able
to limit (some of) the input(s) to the "combinator"?

  This is really more like "home distilling"  than "home brewing" in
that if you don't do it right it will kill you instead of just taste bad.
So, on the contrary, I think you definitely can lose.

  Dan.

* random in the sense that to the attacker an n-bit sample appears
uniformly distributed over the entire set of n-bit vectors.

---

From: Michael Hammer <[email-masked]>
Date: Sat, 25 Jan 2014 16:16:32 +0000

So, if you mix a non-random input with a random input, 
using a deterministic algorithm, the output will be more random?
That doesn't seem right to me.

We can't control the randomness of the input data, 
but the other inputs to an encryption or hashing algorithm are my concern.

Michael Hammer
Principal Engineer

-----Original Message-----
From: Krisztián Pintér [mailto:[email-masked]] 
Sent: Friday, January 24, 2014 9:02 AM
To: Michael Hammer
Cc: [email-masked]; [email-masked]
Subject: Re: [dsfjdssdfsd] Any plans for drafts or discussions on here?

> then who is to say that one's randomness is better or worse than another?

as i said, we need to examine the physical processes. the best source of
entropy as of now is thermal noise. we understand thermal noise to a great
degree, we don't expect sudden breakthrough in modeling it, and it is
relatively abundant and easy to access. user input also contains noise, as
the user can control keystroke or mouse movement timing up to some 10's or
100's of milliseconds, below that, it is just noise from the equipment and
the "biological equipment".

> Was thinking in terms of how an app with access to alternate random 
> sources, some which might be from OS or from some software, might 
> choose one over another.

if you are adamant on doing homebrewed, why choose? you can combine them. if
your combinator is good, you can't lose.

---

From: Paul Hoffman <[email-masked]>
Date: Sat, 25 Jan 2014 11:35:11 -0800

> That doesn't seem right to me.

That's because it is not right for many reasons. To start, you haven't defined "non-random" and "more random".

A better description:

Value A has X bits that cannot be known to adversary M. Value B has Y bits that cannot be known to M.

Securely mixing A and B into a value C whose length is greater than or equal to (X + Y) will result in C having (X + Y) bits that cannot be known by M. If C's length is less than (A + B), every bit in C cannot be known by M.

In your question above, the fact that B might be 0 is irrelevant to the calculation. 

--Paul Hoffman

---

From: Michael Hammer <[email-masked]>
Date: Sat, 25 Jan 2014 21:17:00 +0000

I am not sure if we are talking past each other.

Using terms from your example, 
I have one input (trying to map to what you say) that is message A.
I have another input, call it key B.
The output of the "secure algorithm" is C.
C will be known by adversary M.

The question is whether B is sufficiently random that C cannot guess it.
Also, that M cannot easily discover A knowing C.
The strength of the algorithm is part of the assurance.
The strength of the key is the other part.
Weak key B does not adequately protect message A.

Now, being random does not guarantee that the key B is not weak, just not
easily deduced by M.
But, if B is generated from inputs B1 and B2 in such a way that it tends to
reduce the randomness 
(worse case results in very small subset of keys B), then M can brute force
B to reveal A.

One of the papers cited earlier pointed out how a complex algorithm 
actually ended up converging on a small number of values.
I would hope to avoid repeating that mistake.

Michael Hammer
Principal Engineer

-----Original Message-----
From: Paul Hoffman [mailto:[email-masked]] 
Sent: Saturday, January 25, 2014 11:35 AM
To: Michael Hammer
Cc: [email-masked]
Subject: Re: [dsfjdssdfsd] Any plans for drafts or discussions on here?

On Jan 25, 2014, at 8:16 AM, Michael Hammer <[email-masked]>
wrote:

> So, if you mix a non-random input with a random input, using a 
> deterministic algorithm, the output will be more random?

That's because it is not right for many reasons. To start, you haven't
defined "non-random" and "more random".

A better description:

Value A has X bits that cannot be known to adversary M. Value B has Y bits
that cannot be known to M.

Securely mixing A and B into a value C whose length is greater than or equal
to (X + Y) will result in C having (X + Y) bits that cannot be known by M.
If C's length is less than (A + B), every bit in C cannot be known by M.

In your question above, the fact that B might be 0 is irrelevant to the
calculation. 

--Paul Hoffman

---

From: Krisztián Pintér <[email-masked]>
Date: Sun, 26 Jan 2014 00:25:06 +0100

> I would hope to avoid repeating that mistake.

yep, you need to take good care. but we do have good mixing functions.