The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology

We present a simple mathematical technique that we call GRIM (Granularity-Related Inconsistency of Means) for verifying the summary statistics of published research reports in psychology. This technique evaluates whether the reported means of integer data such as Likert-type scales are consistent with the given sample size and number of items. We tested this technique with a sample of 260 recent articles in leading journals within empirical psychology. Of the subset of articles that were amenable to testing with the GRIM technique (N = 71), around half (N = 36; 50.7%) appeared to contain at least one reported mean inconsistent with the reported sample sizes and scale characteristics, and more than 20% (N = 16) contained multiple such inconsistencies. We requested the data sets corresponding to 21 of these articles, receiving positive responses in 9 cases. We were able to confirm the presence of at least one reporting error in all cases, with 2 articles requiring extensive corrections. The implications for the reliability and replicability of empirical psychology are discussed.