Exact Permutation Test
Enumerate all equally likely treatment pairs under the sharp no-effect randomization null.
Free theory articleEnumerate all equally likely treatment pairs under the sharp no-effect randomization null.
Concept & interpretation
The exact permutation test is a non-parametric method for testing hypotheses under the sharp null hypothesis of no treatment effect. Instead of relying on large-sample approximations (like the normal distribution), it uses the exact randomization distribution of test statistics formed by permuting the treatment assignments across the observed outcomes.
Experiment
You can interact with the Exact Permutation Test module to see how the null distribution and p-value change based on the selected dataset and alternative hypothesis.
- Set Dataset to Spread, Treated pair to 23 and Alternative to Two-sided. Read observed difference 2 and p = 1/3 from two extreme assignments.
- Set Treated pair to 02. The observed difference is -1 and four assignments are extreme, giving p = 2/3.
- Restore pair 23 and choose Alternative Greater. Only the +2 assignment is extreme, so p = 1/6.
- Select Dataset Ties and Two-sided. Pair 23 has difference 1 and p = 1/3; the four zero statistics still represent four distinct labelled assignments.
- Select Dataset Constant. All six statistics are zero, all are extreme and p = 1. There is no data-dependent deduplication.