lab: an R package for generating analysis-ready data from laboratory records

Introduction

Electronic health records (EHRs), which have been widely adopted in medical institutions, serve as comprehensive repositories of clinical data and offer valuable resources for clinical research (Safran et al., 2007; Murdoch & Detsky, 2013; Rudin et al., 2020; Macias, Remy & Barda, 2023). Leveraging secondary EHR data, clinicians and researchers can gain deeper insights into disease progression, identify appropriate treatments strategies (Safran et al., 2007; Murdoch & Detsky, 2013), and make data-driven healthcare decisions in healthcare (Hersh, 2007; Rudin et al., 2020). Within EHR, laboratory test results play a crucial role (Murdoch & Detsky, 2013; Shickel et al., 2017), providing valuable information for developing risk models and predicting disease progression (Perotte et al., 2015; Tseng et al., 2019; Norgeot et al., 2019; Li et al., 2023; Chien et al., 2023). These predictive models can be further enhanced by incorporating machine learning and deep learning technologies, enabling more accurate assessments and predictions.

Unlike diagnosis or procedure records, laboratory records present unique challenges in EHR analyses. These records include information about the tests performed and contain test results with variations in units and formats. For example, hospitals may record C-reactive protein (CRP) levels in mg/dL or mg/L. Furthermore, inconsistencies in coding systems across institutes or EHR systems make data collection and analysis more difficult, as laboratory tests are often coded using local coding systems (Lin, Vreeman & Huff, 2011; Zunner et al., 2013; Miotto et al., 2016). Sometimes, the same laboratory test may be coded differently, further complicating data aggregation. To address these challenges, there is a need for a tool that enables users to standardize and manipulate laboratory data using the Logical Observation Identifier Names and Codes (LOINC), which serves as the international standard terminology for laboratory and clinical observations (Abhyankar, Demner-Fushman & McDonald, 2012). By mapping local coding systems to LOINC, analysts can access standard names, reference ranges, and standard units for each test. Additionally, LOINC allows for searching laboratory items based on related information such as disease names, specimens, and other relevant terminology.

Analysis of EHRs incorporating temporal information has been shown to improve outcomes, prognoses, and diagnosis predictions (Singh et al., 2015; Liu et al., 2019; Placido et al., 2023). Furthermore, prediction models developed using longitudinal laboratory records have improved predictive capabilities (Perotte et al., 2015; Norgeot et al., 2019). However, the inherent features of EHRs, including heterogeneity and sparseness, pose challenges in extracting and analyzing meaningful data (Anhøj, 2003). The irregular frequency and varying data collection intervals, especially in real-world healthcare systems, are widely recognized as problems that need to be addressed for practical analysis (Yadav et al., 2018). Sparseness and missingness in EHRs can arise for various reasons, including instances when certain tests or measurements were not conducted, data was not consistently recorded or documented, or information was omitted or unavailable for specific patients or encounters (Holmes et al., 2021; Knevel & Liao, 2023). One approach to address data irregularity involves developing algorithms capable of handling irregularly sampled time-series data with missing values (Horn et al., 2020; Wang et al., 2021). Another approach is the design of data preprocessing methods, such as resampling and imputation, to aggregate data into a regularly-sampled format before utilizing it in commonly used time-series machine learning algorithms (Tseng et al., 2015). These algorithms typically rely on regular time intervals, such as autoregressive integrated moving averages and basic recurrent neural network models.

Therefore, we developed an open-source R package lab that facilitates the retrieval of detailed information from laboratory records. This package includes functionalities such as mapping local laboratory codes to the international standard, annotating abnormal results, segmenting data into time-series windows, summarizing data using descriptive statistics, imputing missing values, and generating analysis-ready datasets. By utilizing the lab package, analysts can streamline and expedite the workflow of laboratory report extraction. Moreover, it can assist clinical data analysts in producing concise and reproducible research methodology, thereby enhancing collaboration and knowledge sharing within the field.

Materials and Methods

An overview of the lab package is presented in Fig. 1. The sample codes described below are available at https://github.com/DHLab-TSENG/lab-paper/blob/main/SampleCode.md. The development version of the package can be found on GitHub (https://github.com/DHLab-TSENG/lab) and is accessible through the remotes R package, which enables installation of packages from GitHub (Csárdi et al., 2021).

To install the lab R package, users can enter the following commands in an R session:

Time-series analysis

The lab package provides users with the capability to segment laboratory test results into multiple, consecutive non-overlapping time windows. The index date for the time windows can be defined as the first or last record date for each individual, or a specific date, such as the first diagnosis date for a particular disease. To assist users in determining an appropriate window size (e.g., 30, 90, or 180 days), the lab package includes a plot function called plotWindowProportion. This function allows the visualization of the distribution of missing values proportions after segmenting the data. Users can decide on the proper window size for data segmentation by observing and comparing missing rates of each time period. The plot also depicts the frequency of patients undergoing laboratory tests (Fig. 2). The function offers two approaches for missing rates calculation: “By Individuals,” which displays the proportion of patients who never received the laboratory test, and “By Window,” which displays the proportion of windows without laboratory test results.

windowProportion <-plotWindowProportion(labData =loincSample,
idColName = SUBJECT_ID,
labItemColName = LOINC,
dateColName = CHARTTIME,
indexDate = first,
gapDate =c(30, 90, 180, 360),
studyPeriodStartDays=0,
studyPeriodEndDays=360)
head(windowProportion$missingData)
#>	LAB	Gap	Method	Proportion
#> 1:	1742-6	30	By Individual	0
#> 2:	1742-6	30	By Individual	0
#> 3:	1742-6	30	By Individual	0
#> 4:	1742-6	30	By Individual	0
#> 5:	1742-6	30	By Individual	0
#> 6:	2160-0	30	By Individual	0
print(windowProportion$graph)

DOI: 10.7717/peerj-cs.1528/table-9

After the index date and window size are determined (e.g., the first test record date is set as the index date, and 30-days is set as the window size), the descriptive statistics, including the total number of tests within a window, maximum test result, minimum test result, test result mean, and result nearest to the index date, can be obtained using the getTimeSeriesLab function. The time window series is a complete sequence in order by default, even if no records are available for a certain window.

timeSeriesData <-getTimeSeriesLab(labData =loincSample,
idColName = SUBJECT_ID,
labItemColName = LOINC + LABEL,
dateColName = CHARTTIME,
valueColName = VALUENUM,
indexDate = first,
gapDate = 30,
completeWindows = TRUE)
head(timeSeriesData)
#>	ID	LOINC	LABEL	Window	Count	Max	Min	Mean	Nearest
#> 1:	36	1742-6	Alanine Aminotransferase (ALT)	1	2	12	8	10	8
#> 2:	36	1742-6	Alanine Aminotransferase (ALT)	2	NA	NA	NA	NA	NA
#> 3:	36	1742-6	Alanine Aminotransferase (ALT)	3	NA	NA	NA	NA	NA
#> 4:	36	1742-6	Alanine Aminotransferase (ALT)	4	NA	NA	NA	NA	NA
#> 5:	36	1742-6	Alanine Aminotransferase (ALT)	5	NA	NA	NA	NA	NA
#> 6:	36	1742-6	Alanine Aminotransferase (ALT)	6	NA	NA	NA	NA	NA
#>	firstRecord		lastRecode
#> 1:	2131-04-30		2131-05-17
#> 2:	<NA>		<NA>
#> 3:	<NA>		<NA>
#> 4:	<NA>		<NA>
#> 5:	<NA>		<NA>
#> 6:	<NA>		<NA>

DOI: 10.7717/peerj-cs.1528/table-10

A function for plotting line charts, plotTimeSeriesLab, is available to visualize long-term trends in the test results (Fig. 3). “L” (Low) and “H” (High) icons appear in the legends if abnormal values are identified.

timeSeriesPlot <-plotTimeSeriesLab(labData =timeSeriesData,

idColName = ID,

labItemColName = LOINC + LABEL,

timeMarkColName = Window,

valueColName = Nearest,

timeStart = 1,

timeEnd = 5,

abnormalMarkColName = NULL)

plot(timeSeriesPlot)

DOI: 10.7717/peerj-cs.1528/table-11

Results

To illustrate the lab package’s main functionalities and common workflow, we analyze data from MIMIC-III on newborns diagnosed with patent ductus arteriosus (PDA) (Johnson et al., 2016). MIMIC-III is a publicly available database that contains de-identified health-related data from approximately 60,000 patients admitted to the critical care unit of the Beth Israel Deaconess Medical Center between 2001 and 2012.

We first identified cases of PDA using the dxpr package (Tseng, Chiu & Chen, 2021), a package developed by our team for diagnosis and procedure data analysis. We defined patients with PDA as those with at least one PDA diagnosis (ICD-9-CM 7470*). The process used to identify patients with PDA is described in a previous study (Tseng, Chiu & Chen, 2021). We defined the primary outcome of interest as 30-day in-hospital mortality. The original laboratory records obtained from MIMIC-III were used to prepare time-series laboratory records and examine the differences in the laboratory test results of patients with and without 30-day in-hospital mortality using the lab package.

The codes used for the use case are provided at https://github.com/DHLab-TSENG/lab-paper/blob/main/UseCase.md. After we identified patients with PDA from a cohort of newborns whose data were obtained from MIMIC-III (n = 7,833) by using the dxpr package, we included data from 381 patients with PDA, among whom 12 (3.1%) experienced in-hospital mortality within 30 days. To compare the laboratory results of the patients with and without 30-day in-hospital mortality, we included the laboratory tests administered to more than 95% of the patients. The index date was the date of PDA diagnosis. The plotWindowProportion function was employed to visualize the proportion of missing values in different time windows, aiding in determining an appropriate window size for segmenting the laboratory records. Figure 4 illustrates that using a 1- or 3-day window could result in a large ratio of missing records, thereby affecting the analytical outcome. Using a 7-day window decreased the missing rate to approximately 50%, leading us to select 7 days as the segmenting window size. The analysis-ready data generated using the wideTimeSeriesLab function can be passed to the tableone (Yoshida & Bartel, 2022) package to create objects summarizing all laboratory test results stratified by the patients with and without 30-day in-hospital mortality. Statistical tests were conducted accordingly. The results revealed significant differences in five out of the 15 laboratory test results, including total bilirubin levels, monocyte levels, mean corpuscular hemoglobin levels, mean corpuscular volume, and red blood cell counts, between patients with and without 30-day in-hospital mortality at 7 days after their PDA diagnosis (The first time window; p < 0.05; Table 2). We applied the same process to each window to identify the key indicators of 30-day in-hospital mortality during different hospitalization periods. The time series analysis-ready data can be used as an input for training prediction models, employing time-series deep learning algorithms such as the long short-term memory algorithm with the keras (Falbel, Allaire & Chollet, 2022) package. The training performance reached an area under the receiver operating characteristic curve of 0.83 after 100 epochs with the long short-term memory algorithm.

Discussion

Utilizing the temporal information derived from time-series measurements to predict patient prognoses is a highly necessary yet challenging task. Tools that facilitate data processing for time-series analyses have emerged as the preferred approach to accomplish this task. The proposed lab package aims to streamline the processing of laboratory records, thereby enabling the generation of time series data suitable for subsequent analysis. To evaluate its efficacy, the package was tested using publicly available critical care data sourced from MIMIC-III (Johnson et al., 2016) and a multi-institutional medical care database known as the Chang Gung Research Database (Tsai et al., 2017; Shao et al., 2019; Chiang, 2022).

In the era of digital health, artificial intelligence has been successfully employed across various medical domains (Shortliffe & Sepúlveda, 2018; Tseng et al., 2019, 2020; Rajkomar, Dean & Kohane, 2019; Emanuel & Wachter, 2019). Preprocessing medical records is crucial in developing artificial intelligence–assistive services (Harris et al., 2018; Sarwar et al., 2023). Data correctness is a critical aspect of data quality that determines the suitability of EHR data for its research purposes (Weiskopf & Weng, 2013; Verheij et al., 2018; Sarwar et al., 2023). While data from EHR may be accurate at the source, choices made during the data extraction process can impact the results of research queries (Denney et al., 2016; Agniel, Kohane & Weber, 2018).

To improve the data processing step for EHR sharing or analysis, various guidelines and pipelines have been proposed for acquiring data from EHRs (Rea et al., 2012; Knake et al., 2016; Maletzky et al., 2022; Miao et al., 2023). Additionally, for processing and analyzing specific datasets like MIMIC-IV, multiple tools have been developed to cater to the unique characteristics of these datasets (Mandyam et al., 2021; Gupta et al., 2022). Furthermore, several R packages have been developed to assist in general EHR analyses, but they mainly focus on diagnosis or procedure data (Springate et al., 2017; Wasey & Lang, 2022), including the dxpr package (Tseng, Chiu & Chen, 2021). Another R package, cleanEHR, was specifically designed for creating a linkable database across multiple ICUs, although it does not include time-series data processing (Harris et al., 2018). Consequently, there is a need for a tool dedicated to time-series laboratory record analysis. In response to this gap, we developed the lab package to assist analysts in retrieving critical information from laboratory records and analyzing the differences in laboratory test results in different research groups.

The proposed package has limitations. First, users are required to provide a mapping table to establish a link between their local laboratory codes and LOINC. Acquiring such mapping tables can be challenging, potentially hindering the ability to connect laboratory records with LOINC codes effectively. If LOINC mapping cannot be completed, users should provide reference ranges or other information when performing some functions related to the data provided by LOINC, such as reference ranges. However, it is essential to note that distinguishing between normal and abnormal laboratory test results may not always be necessary for laboratory record analyses. Second, we introduced an easy-to-use package that assists analysts in processing raw laboratory records. However, it is worth mentioning that missing values in medical measurements often exhibit informative patterns that can be leveraged for prediction purposes. Exploring the underlying dynamics of these patterns can be valuable in preparing datasets for subsequent machine learning models (Li et al., 2021). Furthermore, the optimal imputation approach may vary depending on the specific dataset being analyzed (Hu et al., 2017). Although we have implemented common imputation approaches used in clinical research (Jakobsen et al., 2017; Hu et al., 2017), users should consider alternative imputation algorithms if there are any more suitable ones for their particular datasets. Nevertheless, even with the availability of this package, analysts must possess a comprehensive understanding of the data to determine the most appropriate method for data processing accurately.

Conclusions

The proposed package assists clinical data analysts in overcoming the challenges of heterogeneous and sparse laboratory records. It simplifies laboratory record preprocessing and facilitates the generation of analysis-ready time series data. By utilizing this package and conducting additional analysis, researchers can understand disease progression in-depth, make informed treatment decisions, and enhance prognostic and diagnostic predictions.