The Permutation Test is an intuitive statistical test that relies on computation, rather than complex formulae and assumptions. Consequently, it is far easier to understand and implement than the “classical” statistical alternatives that are generally taught in quantitative methods classes. A detailed description of permutation testing (written in the context of teaching statistics) is given by Cobb (2007). A shorter descrition is given by Allen Downey in his well known blog articles there is only one test (2011) and there is still only one test (2016). A nice visual illustration of how it works is given is given by Jared Weiber (2019).

This tool is intended to provide a simple interface that allows students to use a permutation test to compare the mean of two groups of data (sample X and sample Y).

Hypotheses

Null Hypothesis (always the same) There is no difference between the values in Sample X and Sample Y (i.e., they are overall neither larger nor smaller).

Alternate hypothesis (set this): Sample X is different to Sample Y Sample X is Greater than Sample Y Sample X is Less than Sample Y

Data

Now paste your data for the two samples from Excel into the boxes below.

Sample X

Sample Y

Run the Test!   Clear

Load demo data

Results

Interpretation

For a relatively straightforward discussion on the interpretation of p-values (including a list of common misinterpretations), see Dahiru (2008). For a more detailed exploration of the challenges associated with the misinterpretation of statistical testing, see Ioannidis (2005). For a discussion on the importance of testing for effect size (not just significance), see see Sullivan and Feinn (2012).

Permutation test powered by Simple Statistics