Relative Randomness And Randomness Hierarchies: A New Framework For Analysis

Defining Relative Randomness

Relative randomness formalizes the intuitive notion that some probability distributions exhibit more randomness deficiencies than others. We introduce a rigorous approach for quantifying and comparing the randomness properties of probability distributions.

Let $X$ and $Y$ be discrete probability distributions. We say $X$ is relatively more random than $Y$, denoted $X \succ_{rr} Y$, if there exists a randomized algorithm $T$ such that:

$T(Y)$ is statistically close to $X$ according to some similarity or distance measure $d$
$T$ cannot increase the randomness deficiency of any input distribution

Intuitively, $T$ transforms the relatively less random $Y$ into the relatively more random $X$, while preserving randomness properties. This ordered relationship $X \succ_{rr} Y$ induces a hierarchy across probability distributions, allowing us to analyze the relative randomness of distributions in a rigorous framework.

Table of Contents

Formalizing Randomness Deficiencies

We quantify randomness deficiencies by measuring statistical distance from the uniform distribution over the sample space. Common measures include:

Total variation distance
Kullback-Leibler divergence
Chi-squared statistics

These detect biases and structure within a probability distribution that reduce randomness. A uniform distribution over the sample space exhibits maximum randomness with no deficiencies.

Constructing Randomness Orders

Using the above statistical distance measures, we can construct rigorous binary relations $\succ_{rr}$ over probability distributions that capture relative randomness. Some examples:

$X \succ_{tv} Y$ if $d_{TV}(T(Y), X) < \epsilon$ for a small $\epsilon > 0$
$X \succ_{kl} Y$ if $D_{KL}(T(Y) || X) < \delta$ for a small $\delta > 0$

Here $d_{TV}$ and $D_{KL}$ are the total variation and KL divergence measures. These relations are irreflexive and asymmetric, inducing a partial order over distributions. We can visualize these orders as randomness hierarchies, with increased uniformity and randomness properties higher in the hierarchy.

Properties of Relative Randomness

The defined relative randomness relations exhibit several useful mathematical properties allowing rigorous analysis. We highlight two key properties.

Transitivity

If $X \succ_{rr} Y$ and $Y \succ_{rr} Z$ then $X \succ_{rr} Z$. Intuitively, if $X$ is more random than $Y$, and $Y$ is more random than $Z$, then $X$ must also be more random than $Z$. This allows us to make such inductive arguments when reasoning about randomness hierarchies.

Connections to Entropy

If $X \succ_{rr} Y$ then entropy $H(X) \geq H(Y)$, with equality only if $X \equiv_{rr} Y$ (equally random). Higher entropy implies greater randomness, so more relatively random distributions also have higher entropy. However, entropy alone does not fully capture relative randomness deficiencies.

Applications of the Relative Randomness Framework

Analyzing probability distributions through the lens of relative randomness has a wide range of applications. We outline a few examples.

Evaluating Learning Algorithms

Statistical learning algorithms attempt to estimate structured probability distributions from samples. More effective learning corresponds to output distributions higher in the randomness order.

For instance, given samples from $X$, if learner $A$ outputs $Y_A$ and learner $B$ outputs $Y_B$, then:

If $Y_A \succ_{rr} Y_B \succ_{rr} X$, then $A$ is more effective
If $Y_A \equiv_{rr} X \succ_{rr} Y_B$, then $A$ learns the true distribution $X$

Analyzing Randomness Extraction

Entropy extraction, randomness pumping, and randomness amplification protocols aim to output distributions higher in the randomness hierarchy from weaker input sources. Analyzing via relative randomness allows quantifying effectiveness.

Comparing Cryptographic Primitives

Pseudorandom generators used in cryptography should output distributions that are computationally indistinguishable from truly random. The relative randomness framework allows directly comparing security and indistinguishability.

Implementing Randomness Tests and Comparisons

We provide example Python code for concretely instantiating the relative randomness definitions in practice to empirically test, compare, and order probability distributions by their randomness properties:


import scipy.stats as st

def test_randomness(P, Q):

  # Statistical distance
  dist = st.distributions.rv_histogram(P).histogram_distance(st.distributions.rv_histogram(Q))  

  # Entropy
  e_P = st.entropy(P) 
  e_Q = st.entropy(Q)

  # Extract ordered relationship
  if dist < epsilon and e_P >= e_Q:
    return P + "_rr" + Q 
  elif dist < epsilon and e_Q >= e_P: 
    return Q + "_rr_" + P
  else:
    return P + "_rr_eq_" + Q

# Usage
order = test_randomness(dist_1, dist_2) 
print(order)

This allows empirically quantifying, comparing, and relating the relative randomness across probability distributions in an automated fashion for further analysis.

Limitations and Open Problems

While relative randomness provides a useful framework, there remain open questions and limitations.

Robustness of Transformations

The choice of randomized transformation $T$ can heavily influence relative randomness conclusions. Developing canonical $T$’s that are universal and robust remains an open area.

Connections to Computational Hardness

There may exist deep connections between relative randomness orders and computational hardness assumptions used in cryptography. Further research is needed to formalize these potential links.

Incorporating Additional Properties

Relative randomness focuses solely on randomness deficiencies. Incorporating additional properties like compressibility may yield more nuanced hierarchies in certain domains.