Reliably Testing Randomness
We interrupt the regular stream of posts1 for an open question to the few readers I have - a very serious one:
Testing for true randomness is easy enough - you can just look at the entropy of the system, in whatever measure it is in. In random sourced data, the entropy should be zero - each piece of information tells you nothing about the next. However, most 'random' processes are actually entropy generators. Take, for example, a Bernoulli Process, which is the sum of a series of independent Bernoulli Trials. The probability of a particular value being sampled in a Bernoulli Process will be known to most people as the Binomial distribution.
This distribution has well-known properties, but is still a randomised process. When testing such a distribution, you have to make sure it's accurate for all ranges of its properties - in this case the number of trials, and the probability of each trial 'succeeding' - which means that you need other degrees of freedom if you want a proper analysis of the results - we can't check the value of one sample, and see that the entire distribution is accurate; for most continuous distributions, the domain is uncountably infinite in size, so the majority of values would be 'valid'.
The...'traditional' way of doing this is to look at the summary statistics over a large number of values; the other possible option is to use a static source of randomness in generating the values, and seeing if the 'correct' ones are obtained.
If you want to see the problems that occur, take a look at https://github.com/javajawa/stochastics - run the tests a couple of time (warning, this can take a long time), and you'll notice that, although there are patterns, different tests fail on different runs.
- 1 ↑ This should have been a LaTeX one, but I haven't finished the code that goes with it, so I had to find something to fill the gap