Data concordance between ESRD Medical Evidence Report and Medicare claims: is there any improvement?

Introduction

The United States Renal Data System (USRDS) is a national data system that includes extensive information about chronic kidney disease and end-stage renal disease (ESRD) in the US (USRDS, 2016). In addition to patient-level data, USRDS also includes specifics of the dialysis clinics in which patients initiate dialysis. In the US, dialysis clinic oversight is divided amongst 18 networks that represent geographic regions of the country. Although the main strengths of the USRDS are its size and representativeness, its limitations include lack of validated comorbidity information and lack of complete laboratory data at initiation of renal replacement therapy (Foley & Collins, 2013).

The USRDS provides two major data sources for ascertaining comorbid conditions in patients on dialysis that are being used in current health care policy research and practice. One source is medical claims data and the other is medical evidence data (Medical Evidence Record Form, designated as CMS-2728 by Medicare), a single form completed by health care providers or staff exclusively at the time of dialysis initiation, a critical transition period in a patient’s life, and providing baseline data upon patient entry into the ESRD program. Although the concise comorbidity data from CMS-2728 are easy to access, there has been ongoing concern about the reliability of CMS-2728 (Byrne & Vernon, 1991; Kim et al., 2012; Krishnan et al., 2015; Longenecker et al., 2000; Solid et al., 2014).

An important use of comorbidity data in health policy is in case-mix adjustment in the development of quality metrics of individual dialysis clinics, which are used to profile health care providers and facilities (Ash et al., 2012). Medicare administers the ESRD Quality Incentive Program (QIP) to promote high quality services in dialysis facilities treating outpatients with ESRD. ESRD-QIP in 2017 included standardized readmission ratio (SRR) as a clinical measure (CMS, 2016; CMS, 2017c). CMS also developed the Dialysis Facility Compare, a 5-Star Rating system of dialysis facilities for public reporting, which initially included a Standardized Mortality Ratio (SMR) and Standardized Hospitalization Ratio (SHR). SMR and SHR use CMS-2728, while the SRR is calculated using past year claims for each index hospital discharge to adjust comorbidities (CMS, 2014a; CMS, 2017a; Kshirsagar et al., 2017; UM-KECC, 2017).

Differences in health care delivery based on local and regional factors, items beyond patient or provider control, are also topics that have been highlighted over the last several years (Bernheim et al., 2016; Kshirsagar et al., 2017; Manickam et al., 2017; Martsolf et al., 2016a; Martsolf et al., 2016b). For example, regional differences have been associated with outcomes in the ESRD setting, particularly when the West coast is compared against other regions of the US (Almachraki et al., 2016; Kshirsagar et al., 2017; Mu et al., 2018), despite near universal care and coverage offered in the US for ESRD.

Building upon the prior studies, we estimated and compared the prevalence and concordance of 11 common comorbid conditions recorded on CMS-2728 versus those determined by medical claims (Aim 1). We implemented different methods of claims data processing in recognition that different algorithms are currently being used in different medical conditions—for example, ESRD vs. cancer—and that substantial discordance has been reported in claims even within a one-year period in a given dialysis patient cohort (Krishnan et al., 2015). We also examined if associations with regions and other external factors (we call ‘environment-related factors’) were found in comorbidity ascertainment between CMS-2728 and medical claims data (Aim 2); if no association is found, we can perhaps be more comfortable in the accuracy of these data (Solid et al., 2014). Through these analyses, we intended to study if there has been any improvement in comorbidity ascertainment data quality since 2010.

Materials and Methods

Study design and data sources

Our study required comorbidity data prior to and at the time of ESRD. Therefore, we chose to examine subjects who had Medicare parts A and B coverage and medical claims information prior to ESRD. Since Medicare eligibility begins at age 65, we looked at those who started dialysis at age ≥67 years of age, allowing for at least one year of claims data prior to ESRD, though this age restriction cannot guarantee the completeness of the past two years of Medicare claims data.

Specifically, we included Medicare-eligible patients in the USRDS ≥67 years of age at the time of dialysis initiation, who were also enrolled in Medicare prior to ESRD, and who started hemodialysis between January 1, 2011 and June 30, 2013. Additional inclusion criteria included subjects who had a completed CMS-2728 form and had Medicare parts A and B as primary payer 1 year prior to and at time of dialysis initiation. The final study cohort included 61,280 patients; see Fig. 1. To ensure Medicare enrollment, we linked the USRDS to pre-ESRD Medicare claims data. We next linked the USRDS to the Dartmouth Atlas of Health Care and the American Community Survey to provide environment-related variables.

Results

Discussion

Administrative databases are increasingly used in research, public policy and even in informing health care consumers, as in the case of the Dialysis Facility Compare program. In our study, we utilized two major sources of comorbidity data in nephrology research and health policy in the US—the Medical Evidence report (CMS-2728) that is completed once at the time of dialysis initiation and Medicare claims.

Prior studies on differences in comorbidity ascertainment methodology in ESRD patients have demonstrated a high degree of discordance and overall poor sensitivity, similar to what we found in our study. Longnecker et al. used clinical chart data to validate comorbid conditions reported on CMS-2728 in 1,005 patients who started dialysis from 1995 through 1998. The average sensitivity was 0.59 and specificity was above 0.9, with HIV showing the highest accuracy (Longenecker et al., 2000). Krishnan et al. examined comorbidities recorded on CMS-2728 and claims before and after dialysis initiation based on 45,357 Medicare-enrolled patients who initiated dialysis between 2007 and 2009. Excluding DM, the kappa statistic for measuring data concordance (between claims before dialysis initiation vs. CMS-2728 at dialysis initiation, and between claims before vs. after dialysis initiation) ranged from 0.05 to 0.58. They also found that patient demographics and the USRDS network in which the patient received dialysis were associated with data discordance, but did not study specific dialysis facility or regional characteristics, factors that we included in our analysis (Krishnan et al., 2015). On the other hand, Solid et al. used Medicare outpatient dialysis claims to validate vascular access—a variable newly added in the 2005 revision of the CMS-2728 form—among patients starting hemodialysis in 2010. The two sources agreed for 94% of 9,777 patients with a kappa statistic of 0.83 (Solid et al., 2014). Cause of renal failure and predialysis nephrology care were also evaluated using different data sources over different time periods; the former was reasonably accurate, not the latter (Byrne & Vernon, 1991; Kim et al., 2012).

The two main goals of our study were (1) to check if there is a meaningful improvement in quality of comorbidity data in the modern era of the many quality-based initiatives, compared to that noted in prior studies; and (2) to identify environment-related (e.g., living condition, socio-economic, dialysis facility or regional area) factors associated with data disagreement. An ancillary goal was to examine different methods of claims data processing, although comorbidity ascertainment algorithm using claims is relatively well established for research purposes, as adapted from the SEER method and Charlson comorbidity index.

Our findings relating to the first goal do not demonstrate a meaningful improvement in data quality. We found similar degrees of poor data discordance and sensitivity between CMS-2728 and medical claims as were noted in prior studies, even in a more contemporary era where the stakes are higher for health care providers of ESRD patients to record accurate comorbidity data. Previous studies reported low kappa and sensitivity, and substantial under-reporting in CMS-2728, and our data did not show that these values have improved over time (Krishnan et al., 2015; Longenecker et al., 2000). In particular, our study can be directly compared with that by Krishnan et al. with these two studies adopting very similar inclusion and exclusions criteria, using different years of the USRDS. In contrast, we utilized a more contemporary cohort, examined three different methods of claims data processing (including the method of Krishnan et al.), employed multiple statistical measures (beyond kappa statistic), and used a better suited statistical model (accounting for clustering). As anticipated, the two cohorts showed a similar distribution of patient characteristics and similar kappa values. Medicare has tried on their part to improve comorbidity entry by health care providers. As a part of the national effort, CMS provides education and outreach for “Provider Compliance Program” (CMS, 2014b).

Regarding the second goal, we newly identified that institutionalization (vs. community-dwelling) and some geographic regions showed a higher frequency of misreporting comorbidity data than others. Midwestern, Northeastern and Southern regions tended to show more data discordance, compared to the Western region in most comorbidities except for drug dependence. Notably, the phenomenon of “West vs. others” in the context of ESRD care has been observed in prior studies with different outcomes (Kshirsagar et al., 2017; Mu et al., 2018). This regional variation may have policy implications because near universal care and coverage are uniquely available for ESRD by Medicare in the US. Medicare regulators charged with developing future quality metrics that are, in part, based on comorbidities may need to consider these additional regional variations in monitoring comorbidity data reliability. As a model example provided by the investigators from the USRDS data center, ‘vascular access type’ used for dialysis that is being used for payment (since 2010) successfully passed the data reliability test based on high values in the proportion of agreement as well as kappa statistic, coupled with the absence of statistically significant predictors (among demographics) for disagreement (Solid et al., 2014).

We used three different but related methods of claims data processing. While we did not necessarily find one method to be overall superior to the others, we believe that Method A is easiest and most inclusive and Method C may be most restrictive (Baldwin et al., 2006; Krishnan et al., 2015). It is our opinion that investigator assumptions and preferences, and possibly the particular variable involved, may dictate the method that future investigators choose. We feel that including the three methods in our study might provide some guidance on these decisions. CMS currently established the Medicare and Medicaid Electronic Health Record (EHR) Incentive Programs to encourage health care providers and institutions to adopt, implement, upgrade, and demonstrate meaningful use of certified EHR technology (CMS, 2017b). Hopefully with this and advanced medical informatics and algorithms (Hehner et al., 2017), comorbidity data capture can be more complete and accurate. Indeed, since the time of this study, Medicare has begun using prevalent comorbidities to the SMR and SHR calculations (CMS, 2017d).

Our study has several limitations. First, the findings are limited to elderly in-center hemodialysis patients with Medicare as primary payer prior to dialysis initiation; thus, generalizability to other populations may be limited. Second, neither data source used can serve as a gold standard; thus, we could not assess true data accuracy. Claims data are generally regarded as more accurately recorded, so we used it as the reference standard. Indeed, Krishnan et al. proposed the use of claims during the 3 months after dialysis initiation as a useful source of comorbidity data (Krishnan et al., 2015). In our study, kappa (for concordance) and sensitivity (for diagnostics) yielded qualitatively similar results. Third, we did not investigate how certain factors, especially those that are not included in the current profiling model and quality metrics being used by CMS, such as prior nephrology care and geographic region, impact on the profiling status, which is warranted in future research (Liu et al., 2016).

Conclusion

Our study raises continued concern about the highly varied data quality in comorbidity information amongst patients with ESRD in the US. Although we used a more recent cohort, we conclude that no improvement was observed over the last two decades, and our results emphasize the concerns about the data accuracy of the comorbidities on the CMS-2728 form and its associated use in developing health policies. Hopefully, these studies provide further incentive to improve data accuracy, identify more reliable data sources and raise awareness of potential bias.

Supplemental Information