The P value is viewed by many as an integral part of scientific decision making; determination of ‘statistical significance’ using the P value is said to impact substantially on the acceptance or rejection of hypotheses, on policy and on publishing of academic manuscripts. Yet possible misuse of the P value seems to have become the subject of scrutiny, and not for the first time. An editorial in The American Statistician and a comment in Nature explore moving beyond the concept of ‘statistical significance’.

A measure of data reliability, the P value determines how unlikely an observed result is compared with that expected as a result of random chance. A P value less than 0.05 has become the generally accepted threshold for statistical significance. However, overuse of the P value and this arbitrary threshold is believed to lead to misconceptions: 1) a ‘statistically non-significant’ result proves the null hypothesis; or 2) a ‘statistically significant’ result proves some other hypothesis. The three statisticians who authored The American Statistician editorial describe a world in which researchers are able to treat P values of 0.051 and 0.049 as not being categorically different, and where authors are not encouraged to selectively publish results based on a “single magic number”.

Meanwhile, the comment in Nature outlines a number of proposals as to how scientists may avoid “falling prey” to statistical misconceptions. These include advice “never” to conclude there is ‘no difference’ or ‘no association’ if a P value exceeds an arbitrary threshold (such as 0.05) or if a confidence interval includes zero. Researchers are also implored to “quit categorising”; the human practice of “bucketing” results as ‘statistically significant’ or ‘statistically non-significant’. Ultimately, the authors have advised not to completely ban the P value, but to abandon the concept of ‘statistical significance’. This suggestion has attracted over 800 signatories, including statisticians, clinical and medical researchers, biologists and psychologists.

Ultimately, the statisticians behind these articles envisage that venturing beyond ‘P<0.05’ would reduce overstated claims and overlooked discoveries, and allow development of more customised statistical approaches.

——————————————————–

Summary by Emma Prest PhD from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd