Predicting population: development and validation of a new predictive nomogram for evaluating medication nonadherence risk in a type 2 diabetes

Patients

Participants were recruited from among 18–80-year-old individuals visiting the Inner Mongolia People’s Hospital between July and December 2020. The included patients met the WHO (1999) diagnostic criteria for T2DM and had been on antihyperglycaemic agents. This study was approved by the local ethics committee of the Inner Mongolia People’s Hospital (NO.202000406). All participants provided written informed consent, filled questionnaires evaluating treatment adherence, and were involved in the 10-min questionnaire. The questionnaire consisted of items covering demographic characteristics (e.g., sex, height, and body weight), disease characteristics (e.g., diabetes duration, family history of diabetes), and treatment features (e.g., the class of hypoglycaemic pills used daily, daily dosing frequency of hypoglycaemic agents). The doctors guided the participants in completing the questionnaire in the plain language. Illiterate patients were excluded, as were those with severe cognitive conditions or disabling physical constraints.

Adherence assessment

The Morisky Medication Adherence Scale (MMAS)-based prediction nomogram may identify patients with diabetes with low medication adherence. MMAS is an established tool for assessing patients’ medication adherence in an eight-question format (MMAS-8) by evaluating factors such as whether patients forget to take medication at home or when travelling, if they skip medication due to its unpleasant effects, if they stop medication because they feel healthy, or if they fail to take medication because it is inconvenient or difficult to remember (Morisky, Green & Levine, 1986). Responses to these items are dichotomous (yes/no), with the last item containing a five-point Likert response. We categorised adherence as follows: low adherence ≤6, 6 ≤ medium adherence <8, and high adherence ≥ 8. Patients with low or medium adherence were considered non-adherent.

Statistical analyses

Demographics, treatment, and disease characteristics are presented as counts (%). Participants were randomly divided into a modelling and validation group at a 7:3 ratio using a computer program, and the modelling group was used to establish the nomogram, and the validation group was used for validation. Statistical analyses were performed using R software (Version 3.6.3).

The logistic least absolute shrinkage and selection operator (LASSO) model, a shrinkage method, can actively select from a large and potentially multicollinear set of variables in the regression, thereby obtaining a more relevant and interpretable set of predictors. The LASSO method was utilised to select optimal predictive risk factors from modelling patients with diabetes by reducing high-dimensional data (Sauerbrei, Royston & Binder, 2007; Friedman, Hastie & Tibshirani, 2010). LASSO is performed by successive shrinkage operations to minimise the regression coefficients to reduce the possibility of overfitting; however, the technique is calculated by shrinking the sum of the absolute values of the regression coefficients, forcing and producing coefficients to be exactly zero, thereby selecting non-zero variables to be retained in the model (Kidd et al., 2018). Next, multivariable logistic regression analysis was applied to construct the predictive model by incorporating the characteristics selected in the LASSO regression model. The characteristics were considered as odds ratios (ORs) with 95% confidence intervals (CIs). A two-sided test was used as the statistical significance tool. Sociodemographic variables (p-value ≤ 0.05) associated with disease and treatment characteristics were included in the model (Xing et al., 2017). All potential predictors were applied to build predictive models for the risk of medication nonadherence using the cohort (Balachandran et al., 2015; Iasonos et al., 2008).

Calibration curves were plotted to assess the calibration of the non-adherence nomogram. A significant test statistic implies that the model is not perfectly calibrated (Kramer & Zimmerman, 2007). Harrell’s C-index was used to quantify the discrimination performance of the nonadherence nomogram. The non-adherence nomogram was tested on a validation sample to calculate a relatively corrected C-index (Pencina & D’Agostino, 2004). Decision curve analysis (DCA) was performed to determine the clinical utility of the nonadherence nomogram by quantifying the net benefits at different threshold probabilities in the diabetes cohort (Vickers et al., 2008). DCA is a simple method for evaluating clinical predictive models, diagnostic tests, and molecular markers. Its advantage is that it integrates the preferences of patients or decision makers into the analysis (Fitzgerald, Saville & Lewis, 2015). Doctors may be particularly concerned about missing disease after consultation and discussion with some patients. However, doctors may be more concerned about avoiding unnecessary intervention for other patients. Doctors may also differ in their propensity to intervene, more conservative, or more aggressive (Vickers, Calster & Steyerberg, 2019). DCA is a method for assessing and comparing prediction models based on clinical consequences. It is based on the principle that the probability of a doctor recommending treatment is informed by how doctors and patients weigh the harm of a false-positive outcome against the harm of a false-negative outcome. Investigators should first consider the relative harm of avoiding interventions for patients with disease and unnecessary interventions for patients without disease in order to develop a clinically reasonable range of threshold probabilities. Then, they should decide whether the net benefit of their model or test is more appropriate than the alternatives within this threshold probability range. In a similar way, decision curves are applied to assess whether a model or test would be beneficial in clinical practice. If the results are positive, the model or test can be applied to appropriate patients as part of a shared decision (Huang et al., 2016).

Characteristics of patients

A total of 753 patients (438 men and 315 women; mean age: 57.12 ± 12.25 years, range 20–79 years) who visited our clinic between July and December 2020 completed the questionnaire. Based on the MMAS-8 score, patients were divided into two groups: adherence and non-adherence groups. Patient data, including demographics, disease characteristics, and treatment features, in the two groups are shown in Table 1.

Feature selection

Data regarding demographics (age, sex, BMI, marital status, education level, employment, working strength, monthly per capita income, type of medical insurance, distance to hospital), disease characteristics (disease duration, family history of diabetes, complications, comorbidities), treatment features (types of all prescribed daily, types of hypoglycaemic pills prescribed daily, current use of insulin and/or GLP-1, and frequent dosing of hypoglycaemic agents) were considered potential prognostic factors affecting medication adherence and were included in the LASSO regression. Regarding demographics, disease, and treatment features, 19 features were reduced to eight potential predictors since the 753 patients (Figs. 1A and 1B) had nonzero coefficients based on LASSO regression analysis. These features included sex, marital status, education level, employment, distance, self-monitoring of blood glucose (SMBG), disease duration, and daily dose frequency of hypoglycaemic agents (pills, insulin, or glucagon-like peptide 1 [GLP-1]).

Development of an individualized prediction model

Logistic regression analysis results of sex, marital status, education level, employment, distance, SMBG, disease duration, and daily dosing frequency of hypoglycaemic agents (pills, insulin, or GLP-1) are shown in Table 2. The risk of medication nonadherence was negatively related with being a woman (β = −0.62, OR = 0.54, 95% CI [0.14–0.90]), secondary (β = −0.53, OR = 0.59, 95% CI [0.37–0.93]) and higher (β = −1.47, OR = 0.23, 95% CI [0.06–0.72]) education, unemployment (β = −0.37, OR = 0.69, 95% CI [0.43–1.12]), distance to hospital (<30 km) (β = −0.56, OR = 0.57, 95% CI [0.32–1.01]). The risk of medication nonadherence was positively correlated with single (β = 0.62, OR = 0.54, 95% CI [0.14–0.90]), self-monitoring of blood glucose times ≤5/week (β = 1.17, OR = 3.23, 95% CI [1.81–5.97]), disease duration of 5–10 years (β = 1.57, OR = 4.8, 95% CI [2.42–9.85]), and three times daily of hypoglycaemic agents (pills, insulin, GLP-1) (β = 2.15, OR = 8.59, 95% CI [4.68–16.21]). However, some risk factors’ confidence intervals are wide because of the small sample sizes; meanwhile, the magnitude of the confidence interval reflects the contribution to the outcome event. Next, a model based on these independent predictors was developed (Fig. 2). The nomogram was developed by graphically assigning an initial score to each of the eight independent prognostic factors, ranging from 0 to 100. All scores were drawn as a vertical line from the total points, indicating the estimated probability of medication nonadherence. A higher total score in the nomogram was predicted to be associated with a higher probability of medication nonadherence, whereas a lower total score was associated with a lower probability of medication nonadherence.

Apparent performance of the nonadherence risk nomogram in the cohort

The calibration curve of the risk nomogram for predicting T2DM medication nonadherence revealed a decent agreement in this cohort (Fig. 3). The C-index of the prediction nomogram was 0.79 (95% CI [0.75–0.83]) for the modelling cohort, and 0.75 in the validation samples, indicating the good discrimination of this model. Apparent performance exhibited good prediction capability in the non-adherence risk nomogram.

Clinical use

DCA of the medication nonadherence nomogram revealed that if the threshold probability was >12% and <85%, using this nonadherence nomogram to predict medication nonadherence risk and take intervention adds more benefit than intervening all or none of the patients (Fig. 4). This shows that the nomogram has a good clinical value. The medication nonadherence model can then be utilised in appropriate patients as part of shared decision-making. For example, at the 20% risk cut-off and defining it as medication nonadherence, the net benefit was about 35%, which was equivalent to performing 35 out of every 100 cases benefiting from it without harming the interests of others.

Limitations

Some factors may limit the generalisability of our findings. First, the sample size may not be sufficiently large, which may underlie the null effect findings. Second, the study was conducted at a single location; a multicentre study may improve the generalisability of our findings. Other elements, such as social support related to nonadherence, were not closely examined. Although the factors discussed above can improve medication adherence, longitudinally designed studies are advocated to reveal strategies with the best outcomes for this specific group of patients, and to identify the long-term outcomes of nonadherence to diabetes medication. Third, we did not externally validate the nomogram; hence, it cannot be applied to T2DM populations in other regions and countries. Fourth, there are potential drawbacks to using a nomogram. The exact answer to whether nomogram-assisting decisions improve patient prognosis lies in prospective assessment—randomly assigning patients to nomogram-based or non-nomogram-based decisions and comparing outcomes. However, the prospective validation of each nomogram form prior to use is impractical. Hence, despite its good performance, nomograms may lack clinical utility and assessments of whether the nomogram improves patient and physician satisfaction, quality of life, and outcomes are often ignored.

Despite these limitations, our model predicted patient medication nonadherence with a relatively high C-index, which when combined with personalised interventions can improve medication adherence and generate potentially significant cost savings by reducing the levels of nonadherence.