The American Statistical Association (ASA) have released a statement warning about the misuse of the P value, following concern amongst its members that the P value was being misapplied. The statement published in The American Statistician contains six principles to guide in proper use of the P value:

1. P values can indicate how incompatible the data are with a specified statistical model.
2. P values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
3. Scientific conclusions and business or policy decisions should not be based only on whether a P value passes a specific threshold.
4. Proper inference requires full reporting and transparency.
5. A P value, or statistical significance, does not measure the size of an effect or the importance of a result.
6. By itself, a P value does not provide a good measure of evidence regarding a model or hypothesis.

The recommendations should encourage authors to disclose all statistical analyses and prevent them from drawing important conclusions based on P value alone.

The ASA’s executive director, Ron Wasserstein has warned that the P value was never intended to be a substitute for scientific reasoning, claiming “Well-reasoned statistical arguments contain much more than the value of a single number and whether that number exceeds an arbitrary threshold.  The ASA statement is intended to steer research into a ‘post p<0.05 era.’”

P values are commonly used to test a null hypothesis by giving the probability of obtaining results at least as extreme as the ones observed, assuming that the null hypothesis is true. In such a test, the smaller the P value, the less likely an observed set of values would occur by chance. This can lead to rejection of the null hypothesis if the P value is less than or equal to a chosen significance level (typically 0.05). However, rejection of a null hypothesis based on the P value does not prove that the tested hypothesis is true.

The limitations of the P value have long been debated with one journal even banning its use. However, this is the first time the ASA board of directors have issued a statement to address the such a foundational statistical matter, which is being published alongside several articles that are intended to give more perspective on the topic.

——————————————————–

Summary by Louise Niven, DPhil from Aspire Scientific.