Key steps to avoiding artistry with significance tests
Author and article information
Abstract
Statistical significance provides evidence for or against an explanation of a population of interest, not a description of data sampled from the population. This simple distinction gets ignored in hundreds of thousands of research publications yearly, which confuse statistical with biological significance by referring to hypothesis-testing analyses as demonstrating significant results. Here we identify three key steps to objective reporting of evidence-based analyses. Firstly, by interpreting P -values correctly as explanation not description, authors set their inferences in the context of the design of the study and its purpose to test for effects of biologically relevant size; nowhere in this process is it informative to use the word ‘significant’. Secondly, empirical effect sizes demand interpretation with respect to a size of relevance to the test hypothesis. Thirdly, even without an a priori expectation of biological relevance, authors can and should interpret significance tests with respect to effects of reliably detectable size.
Cite this as
2017. Key steps to avoiding artistry with significance tests. PeerJ Preprints 5:e3394v1 https://doi.org/10.7287/peerj.preprints.3394v1Author comment
This is a preprint submission to PeerJ Preprints.
Sections
Additional Information
Competing Interests
The authors declare that they have no competing interests.
Author Contributions
C Patrick Doncaster conceived and designed the experiments, performed the experiments, analyzed the data, contributed reagents/materials/analysis tools, wrote the paper, prepared figures and/or tables, reviewed drafts of the paper.
Thomas H G Ezard contributed reagents/materials/analysis tools, wrote the paper, reviewed drafts of the paper, ideas.
Data Deposition
The following information was supplied regarding data availability:
The research in this commentary article did not generate any data or code.
Funding
THGE is funded by NERC Fellowship NE/J018163/1. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
