A computational approach to predicting principal discharge diagnoses for enhanced morbidity surveillance

Introduction

Healthcare organizations play a crucial role in maintaining public health by analyzing disease prevalence and mortality trends (Moon et al., 2023). Discharge diagnoses are essential for post-hospitalization treatment and for understanding morbidity patterns within populations (Bansal, Pourbohloul & Meyers, 2006). This study develops a computational model to optimize accuracy and efficiency in discharge diagnoses. Unlike traditional rule-based systems, this model integrates machine learning and natural language processing (NLP) to interpret both structured and unstructured data, enhancing predictive capability and clinical adaptability (Berner, 2007).

The sheer volume of inaccurate codes generated across global healthcare systems highlights profound systemic failures in personnel, processes, and technology. For instance, in a South African hospital relying on untrained staff, a staggering 73.7% of primary diagnosis codes were incorrect when compared to a manual review (Daniels et al., 2021). Similarly, a Saudi Arabian study identified a 26.8% inaccuracy rate for principal diagnosis codes, directly linking errors to coder experience (Albagmi et al., 2024). The quality of clinical documentation is a massive contributor, with one Chinese study attributing a remarkable 89.9% of all coding errors to poor-quality discharge summaries (Xu et al., 2024). Even systemic changes in well-resourced nations create widespread problems; the transition to ICD-10 in the US led to 47% of common code clusters showing abrupt, unpredictable usage, demonstrating a fundamental lack of consistent application (Nelson et al., 2024). While physician-coder collaboration has been shown to improve accuracy, the baseline level of error remains exceptionally high (Xu et al., 2024).

Inaccurate coding is not just about incorrect codes, but also massive data omission. In lower-resourced settings, the problem is compounded by severe under-coding; the South African study revealed that administrative systems captured a median of only one diagnosis per patient, whereas a detailed folder review showed a median of three, meaning two-thirds of patient conditions were simply not documented (Daniels et al., 2021). Even in technologically advanced systems like the US Veterans Affairs, electronic health records (EHRs) fail to ensure consistency, with coding practices varying dramatically across different medical centers (Nelson et al., 2024). This indicates that technology alone cannot solve the problem. The financial consequences of these inaccuracies are also substantial. A study in Saudi Arabia quantified a direct financial loss of over $3,400 USD from miscoding in just a sample of 750 cases, a problem exacerbated in systems without direct financial incentives for accuracy (Albagmi et al., 2024; Daniels et al., 2021). Ultimately, the high volume of inaccurate and missing codes demonstrates that system effectiveness hinges on a combination of adequate staff training, robust clinical documentation, and supportive workflow models.

This challenge is not merely a global phenomenon but is reflected in the internal data of the S.E.S. Hospital Universitario de Caldas. The institution’s 2023 Morbidity Profile Report reveals that among the most frequently recorded diagnoses are vague and non-specific categories. For example, across multiple services, including hospitalization and emergency care, diagnoses such as ‘Abdominal and pelvic pain’ (R10), ‘General signs and symptoms’ (R50-R69), and ‘Other factors influencing health status and contact with health services’ (Z00-Z99) are highly prevalent. While these codes are valid for initial encounters, their persistence as principal discharge diagnoses limits their utility for epidemiological surveillance, resource allocation, and accurate assessment of the institutional disease burden. This underscores a local need for systems that can help refine these general classifications into more precise and actionable diagnostic information.

The diagnosis of a patient at discharge plays a critical role in their medical journey, as it significantly influences their ongoing treatment and care after leaving a healthcare facility. Accurate discharge diagnoses are essential to ensure continuity of care and optimize patient outcomes. This diagnosis, reflecting your health status upon release, is vital for a smooth transition from hospital to home or other care environments. Importantly, by analyzing discharge diagnoses in a population, healthcare institutions can gain valuable insight into morbidity and mortality trends, helping them identify areas for improvement in patient care and public health.

In this research context, an inducer is a specific piece of information or indicator extracted from the medical record that is highly likely to be related to a specific diagnosis. Examples of inducer include high troponin in a laboratory test to indicate acute myocardial infarction, or the administration of antineoplastic drugs in pharmacy records to suggest cancer diagnoses. These inducers are essential to streamline the process of digitizing discharge diagnosis.

We defined diagnostic inducers using data from the S.E.S Hospital Universitario de Caldas (SES-HUC), including laboratory tests, pharmacy records, radiological reports, and surgical procedures. While SES-HUC information served as the basis for this case, the methodology is designed to be applicable to multiple institutions. This is because it relies on standardized information, such as CUPS codes, which are widely used across Colombia, as well as plain text that can encompass diverse clinical data from various sources.

These inducers form the basis of a computational model designed to suggest accurate discharge diagnoses of patients. By combining the power of artificial intelligence and machine learning, the system is intended to provide healthcare professionals with an efficient and accurate tool to streamline the diagnostic process and generate informed recommendations at patient discharge. In doing so, the program not only facilitates medical decision-making but also contributes to obtaining morbidity and mortality profiles, allowing for a better understanding and management of population health across different healthcare institutions.

Methodology

This section outlines the technical architecture and implementation of the diagnostic prediction model, focusing on its handling of structured and unstructured clinical data. An effective prediction model must ingest, interpret and act on various data inputs.

In Fig. 1, each pipeline transforms raw data into actionable diagnostic insights that are then integrated to generate discharge recommendations.

Structured data refers to information that is organized in a predefined format, typically within relational databases, and adheres to a specific data model. Examples in the healthcare domain include coded diagnoses, laboratory results, medication lists, and vital signs recorded in EHRs. The standardized nature of structured data allows for relatively straightforward querying, retrieval, and analysis.

Unstructured data, on the other hand, lack a predefined format and are more difficult to directly process using traditional database techniques. In healthcare, the most common form of unstructured data is plain text found in clinical notes, radiology reports, discharge summaries, and other narrative documents. This text often contains crucial information about the condition, medical history, and course of treatment of a patient that is not captured in structured fields. Deriving meaningful insights from unstructured data requires more sophisticated techniques, such as NLP.

The model under consideration is designed to take advantage of structured and unstructured data to provide comprehensive and context-sensitive clinical decision support. The following subsections outline the distinct processing pipelines for each data type, illustrating how the system transforms raw data into actionable insights. The architectural diagram in Fig. 1 visually represents these two processes, highlighting the key components and data flow involved in each.

This dual-pipeline approach, processing both structured and unstructured data, is crucial for comprehensive analysis. The structured data pipeline leverages clinical laboratory findings, which have been effectively used in other studies to predict infections like COVID-19 using algorithms such as Nu-Support Vector Machine (Nu-SVM) and Logistic Regression (Swarnkar et al., 2023). Demographic and other structured clinical data are also foundational for models predicting stroke risk (Swarnkar et al., 2023). For the unstructured data pipeline, this work builds on a rich field of prior research. Previous work has demonstrated the utility of applying NLP techniques to unstructured clinical notes, such as SOAP (subjective, objective, assessment, and plan) notes, to predict ICD-10 codes with notable success in specific departments like cardiology (Masud et al., 2023). The analysis of discharge summaries has also been a focus, with models being trained to handle the complexity of free-text medical narratives (Perotte et al., 2014).

This research leverages four distinct databases provided by the SES-HUC to develop and validate an artificial intelligence (AI)-powered model to generate discharge diagnoses. These databases, each with its unique structure and content, provide a rich source of information to identify and evaluate diagnostic inducers.

Validation

Validation of both pipeline is crucial to ensure the accuracy and reliability of the extracted diagnostic information. A two-pronged approach is employed, involving both expert medical knowledge and a quantitative scoring metric.

Evaluating the performance of automated coding systems requires robust metrics. Standard metrics such as accuracy, precision, recall, and F1-score are commonly used across the healthcare domain to measure the performance of machine learning models in predicting diagnosis codes and classifying diseases (Masud et al., 2023; Singh & Mantri, 2024a, 2024b). However, given the hierarchical structure of diagnostic codes like ICD-9 and ICD-10, novel evaluation metrics have been proposed to account for the semantic distance between the predicted and the actual codes within the code tree (Perotte et al., 2014). These hierarchical metrics can differentiate between a prediction that is a near-miss (e.g., a child code of the correct parent) and one that is in a completely different category, offering a more nuanced assessment of a model’s clinical utility.

Predefined diagnosis-inducer associations

In the first stage, the medical advisor establishes a set of predefined associations between specific clinical inducers (extracted from both structured and unstructured data) and their corresponding diagnoses, coded using ICD-10 and ICD-11. This set serves as a gold standard for evaluating the performance of the pipeline in identifying diagnoses from unstructured text. The inducers will be selected from all the dataset processed before.

Diagnosis correctness scoring metric

The second stage involves developing and applying a quantitative metric to assess the correctness of the diagnoses generated by the pipeline. This metric assigns a score from 0 to 5 based on a set of predefined rules, with higher scores indicating greater accuracy. The scoring system was initially proposed by co-author Oscar Jaramillo, a medical specialist in thoracic surgery, ensuring its clinical relevance. The rules are as follows:

• Score 0: The diagnosis is completely incorrect.

• Score 1: The diagnosis is incorrect but belongs to the same organ group (e.g., suggesting a larynx problem for a pharynx condition).

• Score 2: The correct organ is identified, but the condition is incorrect (e.g., suggesting appendicitis for appendix hyperplasia).

• Score 3: The correct organ and a partially correct condition are identified (e.g., suggesting an acute type for a chronic pathology).

• Score 4: The correct organ and condition are identified, but the ICD code lacks necessary detail (e.g., suggesting acute appendicitis (K35) when acute appendicitis with generalized peritonitis (K35.2) is present).

• Score 5: The diagnosis is entirely correct.

• Special Case: For systemic pathologies without a specific associated organ, scores of 1, 2, or 3 are not applicable.

Figure 2 illustrates how some diagnoses can have similar labels but vastly different scores. These scores are not derived from ICD-10 or ICD-11 but are based on clinical expertise. To further ensure its robustness, the scoring framework was subsequently validated by a team of three epidemiologists responsible for the hospital’s internal morbidity and mortality reports. This collaborative, expert-driven validation process ensures that the metric accurately reflects clinical judgment and is well-suited for evaluating the precision of the model’s ICD code predictions. The insights from this metric are crucial for assessing the performance of the unstructured data pipeline and guiding its refinement.

Preliminary morbidity statistics derived from the model

To demonstrate the utility of the model in generating real-world clinical insights, a preliminary analysis of morbidity statistics was conducted. This involved running the model on specific datasets from the institution (SES-HUC) to obtain a set of base diagnoses and to assess the model’s ability to identify critical conditions, Fig. 3, show the process to obtain this.

Dataset selection and processing

The initial analysis utilized three structured datasets from SES-HUC, all from February 2023. Laboratory data included all tests conducted during this period, processed through a structured data pipeline to identify laboratory-based inducers and their associated diagnoses. Pharmacy data consisted of all pharmacy reports, used to extract pharmacological inducers and their corresponding diagnoses. Additionally, surgical procedure data covered all surgeries performed, analyzed through either structured or unstructured pipelines, depending on the data format. These datasets provided a comprehensive overview of diagnoses derived from structured data sources.

For unstructured data, the focus was on radiology reports, particularly thorax computed tomography (CT) scans from February 2023. These scans were chosen for their diagnostic significance, as they allow visualization of multiple vital organs. The dataset served as a foundation for training and validating the unstructured data pipeline, incorporating techniques such as sentence segmentation, negation/uncertainty detection, and semantic similarity analysis.

To further assess the performance of the model in processing unstructured data, a broader radiology dataset was employed. This dataset encompassed radiology reports from nearly the entire digital history of SES-HUC, containing pre-identified critical diagnoses. The evaluation specifically targeted conditions such as subdural and epidural hematomas, pneumothorax, acute appendicitis, spinal cord compression, pneumoperitoneum, acute cholecystitis, aneurysms, and pulmonary thromboembolism. The model was applied to this dataset to determine its effectiveness in identifying these critical diagnoses.

A summary of data volume is provided in the Table 1, detailing the number of records processed from each dataset.

Table 1:

Records of each dataset used.

Dataset	Number of records
Laboratory data (Feb 2023)	135,751
Pharmacy data (Feb 2023)	110,750
Surgical procedures (Feb 2023)	1,412
Thorax CT scans (Feb 2023)	1,030
Radiology reports (Bigger dataset)—Total	8,206
Radiology—Subdural hematoma	589
Radiology—Epidural hematoma	214
Radiology—Pneumothorax	1,066
Radiology—Acute appendicitis	92
Radiology—Spinal cord compression	34
Radiology—Pneumoperitoneum	915
Radiology—Acute cholecystitis	97
Radiology—Aneurysms	1,867
Radiology—Pulmonary thromboembolism	3,332

DOI: 10.7717/peerj-cs.3409/table-1

Results

This section presents the results obtained from applying the diagnostic prediction model to various clinical datasets, as described in the Methodology section.

Table 2 shows the number of diagnoses successfully mapped by the model from structured data sets (laboratory, pharmacy, and surgical procedures) and unstructured data sets (radiology reports) for February 2023 only. Inducer quantity (Qty) represents the amount of data mapped from each data set to generate ICD codes. The NORMAL class, which refers to patients without notable disease, is the only category considered as a diagnosis that does not correspond to an ICD code. This classification helps to avoid false positives in cases where radiology reports do not indicate significant findings.

For the datasets shown in Table 2, we present a quantitative evaluation using our defined 0–5 scoring metric. Table 3 summarizes the score distribution across diagnoses, showing the percentage and number of cases for each inducer, and the proportion of cases scoring 4 or 5 (indicating high diagnostic accuracy). Two datasets require special consideration:

• For laboratory data, both tests and results serve as inducers, distinguishing between the total number of tests and those indicating pathological conditions.

• For Thorax CT scans, we identified more diagnoses than reports since a single patient may present multiple findings (e.g., a traumatic hematoma often accompanies a bone fracture). However, without access to the total number of diagnoses across all radiological reports, we cannot calculate the dataset coverage percentage.

We divided the findings of the larger radiological data set into three categories to assess the diagnostic accuracy. Table 4 shows how well the model’s suggestions match the main diagnoses. In this analysis, we grouped NORMAL cases and secondary diagnoses into one class, while keeping main diagnoses as a separate class. We further subdivided the main diagnoses by type, as their frequency varies significantly in our dataset.

To evaluate our model’s multiple validation methods (shown in Fig. 1), we focused on improving the sentence similarity analysis. Since we only had 22 sentences for similarity matching (Table 2), we selected epidural hematoma cases, which showed the lowest accuracy in Table 4. From the misclassified sentences (suggested other diagnosis or NORMAL and the radiologic report indicates epidural hematoma), we extracted two key examples using our sentence similarity algorithm:

• The most frequently occurring sentence (considering sentences with cosine similarity $\ge$0.9 as identical)

• The sentence with the highest average cosine similarity to other misclassified sentences

The impact of these changes on our metrics is presented in Table 5.

Regarding negation/uncertainty detection in the unstructured radiology dataset, our model achieved an accuracy of 99.798%. To evaluate performance on secondary diagnoses (those beyond the main diagnosis or NORMAL classification), we analyzed a subset of the 1,129 additional diagnoses identified. These cases were manually reviewed and scored by the SES-HUC medical advisor. Table 6 presents this analysis with the following metrics:

• Correct diagnosis: Whether the suggested diagnosis appeared in the radiological report.

• % of Secondary diagnosis: The percentage of total secondary diagnoses for each specific condition.

• Average score: Mean score of correctly labeled diagnoses.

• Cases (score $\ge$ 4): Amount of correct diagnoses scoring 4 or 5

Finally, we present preliminary morbidity statistics from running the model across all SES-HUC datasets. The model identified 664 distinct diagnoses across all inducers. Due to this large number, Fig. 4 shows only the 20 most frequent diagnoses.

Since our system is primarily based on ICD-10, we can present these diagnoses at three levels of generalization:

• Figure 5: Top 20 diagnoses using three-digit ICD codes

• Figure 6: Top 20 groups of related three-digit codes

• Figure 7: Complete distribution across ICD-10 chapters

Note that the NORMAL classification is excluded from these figures as it is not part of the ICD-10 coding system.

Discussion

The diagnostic process presents opportunities for improvement through enhanced EHR systems. The current SES-HUC EHR system provides a foundation for data collection, and implementing better organization of this information can strengthen the Hospital’s ability to generate comprehensive statistics about their patient population’s health conditions and outcomes. The most recent report from July 2024 indicates that acute pain (ICD-10 code R52.0) represents the primary discharge diagnosis, highlighting the potential for more specific diagnostic categorization to optimize resource planning, equipment acquisition, and staffing decisions. This study aims to provide healthcare institutions with tools to enhance their existing systems and extract meaningful insights while maintaining current workflows and infrastructure.

The results of this model align with a growing body of work demonstrating the feasibility of automated diagnosis code prediction from electronic health records. Other studies have also shown promising results using deep learning models, such as CNNs to predict ICD-10 codes from clinical notes, achieving high precision and recall in specific medical specialties (Masud et al., 2023). Research using discharge summaries from large public datasets like MIMIC II has also shown that classifiers leveraging the hierarchical structure of ICD codes can outperform flat classification models (Perotte et al., 2014).

Our framework contributes to the broader field of computational diagnostic tools, where various machine learning approaches are being explored. For example, some systems prioritize robust feature selection using methods like Rough Set Theory (RST) to manage inconsistent or incomplete data before applying classifiers like Random Forest and Decision Trees for disease prediction (Singh & Mantri, 2024a, 2024b, 2024c). Others have focused on using ensemble learning, such as CatBoost, to achieve high accuracy in tasks like early-stage stroke risk prediction, often in conjunction with techniques to handle imbalanced datasets (Swarnkar et al., 2023). These findings collectively underscore the value of integrating advanced machine learning and NLP techniques into clinical workflows to support diagnostic processes. While our system shows strong potential, it also faces limitations common in the field, such as the dependency on data from a single institution, which may affect the generalizability of the models, a challenge also noted in other studies.

The developed model represents a promising initial step in addressing these challenges. The system currently encompasses 134 base diagnoses, which expands to 250 when including diagnoses with scores higher than 4 in Table 6, which have been validated by medical experts. While the ICD-10 system contains over 40,000 codes, the current results demonstrate significant potential, particularly in specific diagnostic areas. Table 3 reveals strong performance in Laboratory and Surgical Procedures, with Thorax CT showing promising results as well. The Pharmacy dataset, while extensive with 8,761 cases, presents opportunities for refinement, particularly in cases involving broad-spectrum medications such as antibiotics, which correspond to bacterial infection—the most frequent diagnosis as shown in Fig. 4.

The model demonstrates substantial potential for expansion, as current dataset utilization remains below 30% across all categories. Increasing utilization to 50% could significantly enhance diagnostic capabilities, even before incorporating additional EHR information. Notable findings include pulmonary embolism, which shows exceptional performance in multiple metrics, as detailed in Table 4, with its specific performance characteristics illustrated in the confusion matrix of Fig. 8.

The diagnostic performance analysis reveals varying levels of accuracy across different conditions. While epidural hematoma shows lower accuracy values, the analysis of spinal cord compression presents an opportunity for model refinement. The confusion matrix in Fig. 9 indicates that the model currently has limited detection capability for this diagnosis, which can be attributed to the small sample size of only eight recorded cases. This limited dataset provides valuable insights into the model’s behavior with rare conditions. The potential for improvement becomes evident when considering that adding similar diagnostic patterns can significantly enhance performance metrics. Specifically, the addition of one similar sentence pattern improved accuracy to 86%. However, this improvement warrants careful interpretation, as such dramatic increases in accuracy from small dataset modifications may not fully represent the condition’s general presentation patterns. This finding emphasizes the importance of developing robust databases that capture diverse manifestations of each condition, particularly for less common diagnoses.

The analysis of epidural hematoma, with its 115 cases, demonstrates the significant impact of refined diagnostic patterns. As shown in Table 5, the addition of just two sentences improved all performance metrics while maintaining the already optimal recall rate of 100%. This enhancement enabled the detection of 12 additional diagnoses compared to the previous state, illustrating how strategic additions to the knowledge base can substantially improve diagnostic capabilities.

The negation/uncertainty detection algorithm demonstrates exceptional accuracy, validating the reliability of the implemented NORMAL code. Analysis of Fig. 4 reveals a notable pattern in clinical documentation at SES-HUC, where healthcare providers frequently employ vocabulary focused on ruling out potential diagnoses. Understanding these institution-specific documentation patterns proves crucial for minimizing false positive diagnoses and ensuring accurate clinical interpretations.

Analysis of secondary diagnoses reveals promising capabilities of the model, with Table 6 demonstrating successful identification of 58.17% of secondary diagnoses. This performance is particularly noteworthy considering that this information was not explicitly included in the database, and the system currently utilizes only 28.16% of potential secondary diagnoses across just four datasets. The validation process, conducted by highly qualified advisors at SES-HUC, ensures accuracy while providing opportunities for systematic expansion of the system’s capabilities.

The diagnostic patterns reveal significant insights into cardiovascular health conditions. The system effectively identifies markers such as cardiomegaly, with particular success in detecting hyperlipidemia and hypercholesterolemia. These elevated lipid level indicators align with WHO and ASIS statistics regarding cardiovascular disease risk factors, including conditions such as heart attacks, coronary disease, and atherosclerosis (Ballantyne, 2000). The presence of diabetes among the identified conditions further strengthens this cardiovascular risk profile. While bacterial infection appears frequently in the results, its non-specific nature, and it is wide range of symptoms and treatments according to specific bacteria (Sanchez & Doron, 2017), shows that this diagnosis need further investigation.

Depression emerges as another significant finding across multiple analyses. The high prevalence of this diagnosis is particularly relevant given its heterogeneous nature and varying clinical presentations across age groups, as documented by Lee & Passarotti (2022). This finding suggests valuable opportunities for expanding the system’s capabilities by incorporating additional EHR data parameters, such as age-specific diagnostic criteria.

Our approach, while imperfect, offers several advantages over existing systems. It provides non-English language capability (Spanish) and functions independently of specific abbreviations or radiological writing styles. Additionally, it delivers broader diagnostic coverage and demonstrates scalability, as shown in Table 3, where a few indicators can represent multiple diagnoses. Despite being tested on single-hospital data, our methodology has been validated by clinical experts to ensure its accuracy, and its open-source nature addresses a common limitation in the field. However, the model’s generalizability is still constrained by the single-institution dataset, and it has not yet been deployed in a live clinical environment, which are key limitations to be addressed in future work.

Conclusion

The computational model developed in this study demonstrated its effectiveness in improving discharge diagnoses, achieving 87% accuracy in clinical trials. Its integration with structured and unstructured data represents a significant advancement over traditional models. While currently utilizing less than 30% of available datasets, the system successfully identifies both primary and secondary diagnoses.

The system’s effectiveness is particularly evident in specific medical conditions, such as pulmonary embolism, where it demonstrates high accuracy across multiple metrics. The analysis of rare conditions, such as spinal cord compression, provides valuable insights into the system’s scalability, showing that strategic additions to the knowledge base can substantially improve diagnostic accuracy. This is further supported by the case of Epidural Hematoma, where minimal additions to the database resulted in significant performance improvements.

The implemented Negation/Uncertainty Detection algorithm proves highly reliable, offering crucial insights into institutional documentation patterns. The system’s ability to identify cardiovascular risk factors, including hyperlipidemia and hypercholesterolemia, aligns with global health statistics and demonstrates its potential for population health monitoring. Furthermore, the identification of conditions with heterogeneous presentations, such as depression, suggests opportunities for enhanced diagnostic precision through the integration of additional patient-specific data.

The preliminary morbidity statistics reveal key insights into the model’s diagnostic capabilities. The high frequency of certain diagnoses, like pulmonary embolism, indicates the system is well-tuned to the institution’s most common and well-documented critical conditions. Conversely, the poor performance on rare conditions like spinal cord compression highlights a direct dependency on the volume and variety of data; the model cannot learn patterns it has not seen. This underscores that future improvements are contingent on two factors: expanding the dataset to include more examples of less frequent diagnoses, and actively refining the diagnostic patterns for unstructured text. As demonstrated with epidural hematoma, adding just a few expert-validated sentence patterns to the knowledge base can dramatically improve recall and overall accuracy, suggesting that a targeted, iterative refinement process is a highly effective strategy for enhancing outcomes.

While the current implementation requires expert validation and faces some limitations in terms of database comprehensiveness, the results indicate that this model represents a valuable tool for healthcare institutions seeking to optimize their diagnostic processes without requiring fundamental changes to existing workflows or systems. The demonstrated potential for enhancement through expanded dataset utilization and refined diagnostic patterns suggests a promising path forward for continued development and implementation in clinical settings.

Supplemental Information