Pattern analysis of total item score and item response of the Kessler Screening Scale for Psychological Distress (K6) in a nationally representative sample of US adults

Dataset and analysis procedure

Data were drawn from the MIDUS 1 (ICPSR 2760, 2016). The MIDUS 1 is a nationally representative sample of non-institutionalized, English-speaking adults aged 25–74 from the US. Conducted by the MacArthur Foundation Research Network on Successful Midlife Development from 1994 to 1995, the MIDUS 1 comprises a total of 7,108 participants (3,395 men) of an average age of 46.4 years old (SD = 13 years). As noted previously, the MIDUS 1 comprised individuals from four subsamples: (1) a national RDD sample (n = 3,487), (2) oversamples from five metropolitan areas in the US (n = 757), (3) siblings of individuals from the RDD sample (n = 950), and (4) a national RDD sample of twin pairs (n = 1,914).

The MIDUS 1 comprised a telephone interview of approximately 30 min and two self-administered questionnaires, each of which was approximately 45 pages in length. All participants provided informed consent, and the MIDUS was approved by the institutional review board at each data collection site.

The main RDD sample was selected and invited to participate using telephone banks. For each household contacted, a list was generated of all people in that household aged between 25 and 74 years old, from which a respondent was randomly selected. The MIDUS 1 researchers also carried out oversampling of five metropolitan areas. Of the RDD respondents who reported having one or more siblings, 529 were randomly selected and complete telephone interviews were conducted with 950 of their siblings, some of whom came from the same household. Finally, twin pairs were recruited from a representative national sample of approximately 50,000 households for the presence of a twin. The response rates and select sociodemographic characteristics of the MIDUS 1 samples are reported in detail elsewhere (ICPSR 2760, 2016; Brim, Ryff & Kessler, 2004).

Ethics statement

The present paper is a secondary analysis of freely available data so format clearance was not needed. As our local institutional review board does not consider de-identified secondary data analysis to be human subjects research, our study did not require ethical approval from that board.

Measures and analysis

In the self-administered questionnaire of the MIDUS 1, depressive symptoms were assessed using the K6. The K6 comprises six items that ask about the frequency that participants felt sad, nervous, restless, hopeless, that everything was an effort, and worthless during the past 30 days. Each item is rated on a 5-point scale with options of 0 = none of the time, 1 = a little of the time, 2 = some of the time, 3 = most of the time, and 4 = all of the time, yielding a total item score of 0–24. Higher scores indicate greater psychological distress. One of the K6 items used in the MIDUS and NHIS surveys is worded as follows: “How much of the time did you feel so sad nothing could cheer you up.” This contrasts with the wording generally used in the K6 today: “How often did you feel so depressed that nothing could cheer you up?” (Kessler et al., 2002).

Participants who did not respond to all K6 items were excluded from the total score analysis. The final sample for the total score analysis comprised 2,975, 647, 858, and 1,743 individuals from the national RDD sample, oversamples, siblings, and twin pairs, respectively. To evaluate whether K6 total scores follow an exponential pattern, the distributions of the total score were analyzed using a log-normal scale and an exponential regression model.

For the item analysis, participants who did not respond to each item were excluded from the item response analysis (Table 1 and Table S1). We analyzed the item responses for the six items using normal and log-normal scales. The regression curve for the exponential model was estimated using the least squares method. JMP Version 11 for Windows (SAS Institute, Inc., Cary, NC, USA) was used to calculate descriptive statistics and the frequency distribution curves.

K6 total score analysis

Figure 2 depicts the distribution of K6 total scores for (A) the national RDD sample, (B) oversamples from five metropolitan areas, (C) the siblings of the RDD sample, and (D) the national RDD sample of twin pairs. While the total score distributions for the four subsamples were commonly right-skewed, the frequencies of the zero score were different across groups due to the difference in sample sizes.

All the four sub-samples showed linear patterns with similar gradient on the log-normal scale, suggesting that the K6 total scores followed an exponential pattern with similar rate parameters (Fig. 3). While the national RDD subsample and the twin pairs exhibited a linear pattern for almost the entire range of total score (Figs. 3A and 3B), the frequencies of the sibling and the oversample subsamples were often zero over scores of 16–18 because of the small sample sizes (Figs. 3C and 3D). In fact, it is likely that the small sample sizes for the higher scores governed the increasing fluctuation in lines for each subsample as total score increased.

The regression curves for the exponential model were calculated for the national RDD sample (y = 860.9e^−0.259x, R² = 0.95), oversamples from five metropolitan areas (y = 189.46e^−0.264x, R² = 0.96), siblings of individuals from the RDD sample (y = 249.9e^−0.274x, R² = 0.95), and national RDD sample of twin pairs (y = 470.8e^−0.261x, R² = 0.94). The independent variable, x is the K6 total score and the dependent variable, y is the number of subjects. R² is the coefficient of determination. This analysis revealed high coefficients of determination with similar parameters in all four subsamples, thus indicating that the total K6 scores fit well to an exponential distribution with similar rate parameters.

Item response analysis

Table 1 shows the item response rates for each of the 6 items in the national RDD subsample. A similar pattern was observed in response rates for all 6 items, with the highest response frequency being for “none” and a decreasing frequency thereafter as item scores increased, and the lowest response frequency observed for “all of the time.” There were no exceptions to this pattern.

To evaluate the pattern of item responses, all 6 item response rates were plotted together on the same graph. As shown in Fig. 4A, the item responses of the six items showed a common pattern, which displays different types of distributions with a boundary at “a little of the time.” As indicated by the arrow in Fig. 4A, the lines for the six items crossed at a single point between “none of the time” and “a little of the time,” after which they showed a decreasing pattern. As verified in our previous study, if the response distributions of the 6 items follow an exponential pattern with the similar parameters between “a little of the time” and “all of the time,” then all of the lines will inevitably cross at a single point between “none of the time” and “a little of the time” (Tomitaka, Kawasaki & Furukawa, 2015b).

The item response rates for the remaining three subsamples exhibited a similar pattern as for the national RDD subsample—the highest response frequency was for “none of the time,” after which the response frequency decreased with increasing item score (such that the lowest response frequency was observed for “all of the time”; Table S1). No exceptions to this pattern were observed. Furthermore, as depicted in Figs. 4B– 4D, the lines for the 6 items crossed at a single point between “none of the time” and “a little of the time,” after which they showed a decreasing pattern.

On the log-normal scale, the item response rates approximated a linear pattern for the “a little of the time” to “all of the time” responses across all four subsamples (Fig. 5). In addition, the gradients of the linear patterns for the 6 items were rather similar. Specifically, the lines for “a little of the time” to “some of the time” were somewhat parallel, whereas the lines for “most of the time” to “all of the time” were less so.

K6 total scores approximated an exponential pattern

Our findings indicate that K6 total scores of a representative sample of US adults approximate an exponential pattern, which is consistent with the results of other national representative surveys in England and Japan using the CIS-R and CES-D, respectively (Melzer et al., 2002; Bebbington et al., 2013; Tomitaka, Kawasaki & Furukawa, 2015a). Although the K6, CIS-R, and CES-D differ in terms of the number of items, item content, and scoring methods (the CIS-R uses a binary response scale, while the CES-D uses a four-point scale), the exponential pattern of total scores in a representative population was confirmed within all three scales. Taken together, the current evidence suggests that the total score of depressive symptom measures approximates an exponential pattern irrespective of the choice of items or the scoring method. In support of this conclusion, the previous study using the CES-D has demonstrated that, for any number or combination of items, the sum of the item scores of depressive symptoms approximates an exponential pattern (Tomitaka et al., 2016d). Of course, the depressive symptom items mentioned here do not include the reverse-scored positive affect items because our previous studies using the CES-D showed that reverse-scored positive affect items were not considered manifest variables of the latent trait of depressive symptoms (Tomitaka, Kawasaki & Furukawa, 2015b; Tomitaka et al., 2016c).

The reason that the total scores for the depressive symptoms scale approximates an exponential pattern might be explained by a theory that our research group has proposed (Tomitaka, Kawasaki & Furukawa, 2015a; Tomitaka et al., 2016b; Tomitaka et al., 2016c). The theory comprises the following three conditions: (1) depressive symptom items are manifest variables influenced by a unidimensional latent trait of depressive symptoms; (2) the latent trait of depressive symptom items follow an exponential distribution; and (3) the threshold of ordinal scale scoring forms a distribution in accordance with the unidimensional latent trait. The three conditions were devised based on earlier findings concerning how the total score distribution, the item responses and boundary curves of the total scores of depressive symptoms measures exhibit an exponential pattern (Tomitaka, Kawasaki & Furukawa, 2015b; Tomitaka et al., 2016c). In support of this theory, our previous study simulating these three conditions confirmed that the total score of depressive symptom items approximates the exponential pattern of a latent trait distribution, except for the lower end of the distribution (Tomitaka et al., 2016b).

Considering the rate parameters of the exponential models of total score distribution, the estimated parameters were similar across the four subsamples (−0.26 to −0.27). Our previous analysis of a national representative survey using the CES-D demonstrated that the rate parameters of the exponential model of summed scores were similar across the groups with the same number of items and the rate parameters of the summed item scores increased as the number of summed items increased (Tomitaka et al., 2016d). Interestingly, the estimated rate parameters of the present four subsamples were similar to those of samples that used the sum of eight CES-D items (−0.26 to −0.29), not the sum of two (−0.62 to −0.93), four (−0.41 to −0.52) or 16 CES-D items (−0.14). The similarity of the estimated rate parameters between the K6 and the sum of the eight CES-D items might be attributed to the fact that both the K6 and the sum of the eight CES-D items had the same maximum total score of 24 points. According to the aforementioned theory, a maximum total score means the number of thresholds in accordance with the latent trait. Further mathematical explanation is necessary to elucidate the mechanism of the rate parameter variance.

Although several groups have pointed out that total depressive symptomatic scores do not follow an exponential curve at the lower end of the distribution (Melzer et al., 2002; Bebbington et al., 2013), the K6 total scores of the present studies exhibited an exponential pattern for almost the entire range of total score (Fig. 3). The previous studies demonstrated that the probability of “none of the time” is a key index to predict the non-exponential pattern at the lower end of the distributions (Tomitaka et al., 2016d). Specifically, the sum of the item scores with high probability of “none of the time” exhibited higher scores compared to those predicted from the exponential pattern, whereas the sum of the item scores with low probability of “none of the time” exhibited lower scores compared to those predicted at the lower end of the distributions. In fact, according to our estimation, the mean probability of “none of the time” in the present studies (61.8%) is close to that of the CES-D 16 items (67.7%), which exhibited an exponential pattern for almost the entire range of total score (Tomitaka et al., 2016d).

Item response of K6 items

Consistent with the results using the CES-D (Tomitaka, Kawasaki & Furukawa, 2015b), the item responses of the K6 for the four subsamples exhibited a common pattern with a boundary at “a little of the time.” Furthermore, the lines for the 6 items crossed at a single point between “none of the time” and “a little of the time,” while the item responses from “a little of the time” to “all of the time” approximated a linear pattern on log-normal scales.

The mechanism of this pattern of the K6 items might be speculated as follows (Tomitaka, Kawasaki & Furukawa, 2015b). First, each item of the K6 is rated in two stages. First, each subject determines whether a symptom is present; if the symptom’s severity does not exceed the threshold for everyday mild symptoms, then it is regarded as “absent” (i.e., it occurs “none of the time”). Second, if the depressive symptom does meet the threshold, then the duration of the symptom is quantified according to the item response format as “a little of the time,” “some of the time,” “most of the time” and “all of the time.” This two-step process increases the likelihood that “none of the time” will cover the range of symptoms that does not meet the threshold, while the remaining response options (“a little of the time,” “some of the time,” “most of the time” and “all of the time”) cover about one fourth of the above-threshold range. If the latent trait of depressive symptoms follows an exponential distribution, which uniquely has a memoryless property, the item responses for the K6 would all exhibit the mathematical distribution observed in the present study. Due to memoryless property, regardless of the difference of the threshold of each item, the decreasing ratio of the probabilities among “a little of the time,” “some of the time,” “most of the time” and “all of the time” is constant. However, as this remains a mere speculation, further research would be needed.

In the present study, the K6 items followed an exponential pattern with similar rate parameters between responses of “a little of the time” and “all of the time” (Fig. 5). However, the extent to which the parameters of the six items were similar—expressed as the degree of parallelism of the regression lines—was lower compared with the results of our previous general population sample (N = 32,022; Tomitaka, Kawasaki & Furukawa, 2015b). A possible reason for this difference in parallelism might be the sample sizes. In support of the sample size effect, the lines for the 6 items were highly parallel between “a little of the time” and “some of the time,” but became less so and began fluctuating between “some of the time” and “all of the time,” where the frequencies were much smaller compared to “a little of the time” (Fig. 5).

We assumed that the latent trait of depressive symptoms follows an exponential distribution because this assumption enables us to explain the mathematical patterns of the total scores and item responses. However, the K6 items were developed using item response theory, which generally assumes a normally distributed latent trait (Kessler et al., 2002; Van der Linden & Hambleton, 1997). As noted in the introduction section, there is still little evidence of depressive symptoms being described by a normally distributed latent trait. Claiming that the latent trait of depressive symptom items follows a normal distribution would require a theory explaining how a normally distributed latent trait leads to the observed patterns in this study.

Strengths and limitations

This study has some limitations. First, although we evaluated whether the total item scores and the item responses follow an exponential distribution, we did not consider other possible mathematical models in our analysis. In general, the most important aspect of model evaluation is testing whether a model fits the data better than other models. However, to our knowledge, no other mathematical models have been reported so far. Future studies might seek to evaluate the comparative fit of other models to the data from the National Archive of Data on Aging. Second, participants of the MIDUS did not receive a structured psychiatric interview to diagnose their symptoms. Third, although we found that the total score distribution and item responses of the K6 exhibited exponential patterns, consistent with the screening test for depression (CES-D), K6 is not solely a measure of depression, but a broader measure of psychological distress.

However, our study has a methodological advantage. First, the sample was representative of the general American population, thus ensuring limited selection bias. In addition, four subsamples from different populations were recruited, thus ensuring the reproducibility of the findings. Finally, the present study provides important information on the distribution of depressive symptoms in the general population.