Quantifying the variability of optimal cutpoints and reference values for diagnostic measures

Aim: Empirical studies in medicine – and most other fields of study - yield results that are uncertain to a certain degree. Medical research on interventions has made tremendous leaps forward by quantifying and reporting this uncertainty using p-values and confidence intervals. In contrast to this, most diagnostic studies that determine optimal cutpoints or reference values for diagnostic measures ignore that their outcomes, i.e. the specific cutpoints or normal ranges they recommend, are subject to chance variability. Methods: In this paper we use a simple simulation approach to quantify the variability of optimal cutpoints for two published studies. The first determined an optimal cutpoint for Becks Depression Inventory (BDI) in adults. The second determined reference values for Quantitative Sensory Testing (QST) in children. Results: We find that frequently employed cutpoints to interpret BDI scores and QST results are highly variable. For the BDI we find that replication of this study may identify values between 14 and 21 as optimal cutpoints. The lower cutpoint results in a misclassification of 15% of the healthy adults as depressed, the upper cutpoint results in a misclassification rate of 2%. For the QST we find that the upper end of the normal range HPT varies between 46.9 and 50.2 degrees Celsius. Conclusions: Based on our results we argue that researchers should be required to estimate and report the variability of reference values and optimal cutpoints for diagnostic tools. This may improve the harmonization of findings across studies and provides a rationale for planning future studies.