Sufficient Statistics Factorization
Group binary samples by their success count and see which ordering information stops depending on the unknown probability.
Free theory articleGroup binary samples by their success count and see which ordering information stops depending on the unknown probability.
Concept & interpretation
In statistics, a statistic is sufficient for an underlying parameter if it contains all the information available in the sample about that parameter. The sufficiency principle states that if a sufficient statistic exists, any inference about the parameter should depend on the sample only through the value of the sufficient statistic. The Factorization Theorem provides a formal mathematical criterion for establishing sufficiency: write the joint mass as gp(K(x))h(x), where h does not depend on p. Here gp(k) = pk(1-p)4-k and h(x) = 1 on the sixteen binary tuples (and 0 outside that sample space).
Experiment with Conditional Likelihoods
Open the Sufficient Statistics Factorization experiment to see how the factorization theorem applies to binary sequences.
- Select a Bernoulli probability of
0.25and an Ordered four-bit sample of1010. You'll see the sequence belongs to the group K = 2. The conditional sequence probability is0.166666666666667with exact fraction1/6. - Now, change the Bernoulli probability to
0.5. Notice that while the individual ordered likelihood and the count likelihood change significantly, the conditional sequence probability remains exactly1/6(with its displayed decimal approximation). Given K = 2, the identity of the specific ordering within the group K = 2 carries no information about p. - Change the sequence to
1000. It falls into the K = 1 bucket, which has 4 orderings. The conditional sequence probability is0.25 (1/4), which is still independent of p. - Set the probability to
0and the sequence to1000. Since p = 0 implies K = 1 is impossible, the condition K = 1 has zero probability. In this case, the conditional law is unavailable, rather than dividing 0 by 0.