Development and validation of the AUDEXCEL algorithm as a diagnostic tool for occupational noise-related hearing disorder

Study population

Audiograms of the workers, who attended an occupational health clinic in Kuala Lumpur, Malaysia were obtained. To be eligible for inclusion in the present study, the workers must adhere to the following criteria: (i) Aged between 18 and 60 years old; (ii) workers of any gender or ethnic background; and (iii) workers must have baseline and annual measurements. However, workers with the following characteristics will be excluded for participation: (i) Workers experiencing cognitive impairment; (ii) workers who underwent ear surgeries at the time of recording; and (iii) workers with a history of notable ear disorders (such as Meniere’s disease or acoustic neuroma) that could impact hearing; and (iv) workers with presbycusis. The inclusion and exclusion criteria of these workers were ascertained via the review of their medical history (such as history of presenting illness, underlying medical history, social history, and occupational history) by the researchers.

Sample size estimation

The present study developed an excel-sheet based ONRHD diagnostic tool, named as the AUDEXCEL algorithm, which then undergo validity and reliability tests. The validity of the AUDEXCEL algorithm in diagnosing ONRHD was assessed through a concurrent validity test, via the sensitivity and specificity tests. As per Bujang & Adnan (2016), a diagnostic study typically aims for both sensitivity and specificity to be predetermined at a minimum of 70.0%. With an ONRHD prevalence of 51.4% (Yap et al., 2023), a precision of 0.05, a confidence interval of 95%, and expected sensitivity and specificity of 70%, the required sample size was calculated to be 320.

Meanwhile, the reliability of the AUDEXCEL algorithm in diagnosing ONRHD was assessed through inter-rater reliability, utilising Cohen’s Kappa coefficient. With a minimum acceptable Cohen’s kappa of 0.4 and an expected Cohen’s kappa of 0.6 (McHugh, 2012), along with an ONRHD prevalence of 51.4%, a power of 80%, and a significance level of 0.05, the required sample size was determined to be 156 using an online sample size calculator (Arifin, 2025). Since the sample sizes for validity and reliability are 320 and 156, respectively, the larger sample size (i.e., 320) was selected for this study.

Sampling technique

A random sampling method was employed, where 320 audiograms (considered as the sampling unit) were randomly selected from the occupational health clinic’s database, using an online random number generator.

Development of the AUDEXCEL algorithm

AUDEXCEL was created in the form of an Excel spreadsheet containing six distinct algorithms. Hearing impairment is characterised by a permanent shift of 25 dB or more in the pure tone average (PTA) of hearing thresholds at 500, 1,000, 2,000, and 3,000 Hz, relative to the standard reference level of 0 dB (Department of Occupational Safety and Health (DOSH) of Malaysia, 2019). Accordingly, the first algorithm, which aims to detect hearing impairment, was developed using the following formula:

$$\mathrm{P}\mathrm{T}\mathrm{A}=\frac{{\displaystyle \mathrm{T}\mathrm{h}\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{h}\mathrm{o}\mathrm{l}\mathrm{d}\mathrm{a}\mathrm{t}500\mathrm{H}\mathrm{z}+1\text{,}000\mathrm{H}\mathrm{z}+2\text{,}000\mathrm{H}\mathrm{z}+3\text{,}000\mathrm{H}\mathrm{z}}}{4}.$$

According to World Health Organization (WHO), hearing loss is defined as a partial or complete inability to perceive sound, indicated by a pure tone threshold (PTT) greater than 20 dB at any tested audiometric frequency (World Health Organization (WHO), 2021). Specifically, mild hearing loss falls within the range of 21 to 34 dB, moderate hearing loss falls between 35 and 49 dB, moderately severe hearing loss ranging from 50 to 64 dB, severe hearing loss characterised by thresholds between 65 and 79 dB, profound hearing loss falls between 80 to 94 dB HL, and complete or total hearing loss falls above 95 dB. As a simplification (i.e., to classify an audiogram as “yes” or “no” for hearing loss), the present study defines hearing loss as a pure tone threshold (PTT) greater than 20 dB at any tested audiometric frequency. Hence, the second algorithm, designed to detect hearing loss, was therefore developed using the following formula:

$$\begin{array}{rl}\mathrm{P}\mathrm{T}\mathrm{T}& (500\mathrm{H}\mathrm{z}\mathrm{o}\mathrm{r}1\text{,}000\mathrm{H}\mathrm{z}\mathrm{o}\mathrm{r}2\text{,}000\mathrm{H}\mathrm{z}\mathrm{o}\mathrm{r}3\text{,}000\mathrm{H}\mathrm{z}\mathrm{o}\mathrm{r}4\text{,}000\mathrm{H}\mathrm{z}\mathrm{o}\mathrm{r}6\text{,}000\mathrm{H}\mathrm{z}\mathrm{o}\mathrm{r}8\text{,}000\mathrm{H}\mathrm{z})\\ & >20\mathrm{d}\mathrm{B}.\end{array}$$

A standard threshold shift (STS) refers to a shift of 10 dB or more in the pure tone average (PTA) at 2,000, 3,000, and 4,000 Hz when compared to the baseline audiogram (Department of Occupational Safety and Health (DOSH) of Malaysia, 2019). This shift can be either temporary or permanent. An STS is considered permanent if the threshold shift remains present in a follow-up audiogram conducted within 3 months of the initial test. Based on this definition, the third algorithm (for PSTS) and the fourth algorithm (for TSTS) were developed using the following formula:

$$\mathrm{P}\mathrm{T}\mathrm{A}=\frac{{\displaystyle \mathrm{T}\mathrm{h}\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{h}\mathrm{o}\mathrm{l}\mathrm{d}\mathrm{a}\mathrm{t}2\text{,}000\mathrm{H}\mathrm{z}+3\text{,}000\mathrm{H}\mathrm{z}+9\text{,}000\mathrm{H}\mathrm{z}}}{3}.$$

Although various diagnostic criteria exist for NIHL, the present study adopted the criteria proposed by Hoffman et al. (2006) and McBride & Williams (2001). According to these definitions, NIHL is identified when the following conditions are met: (i) Threshold worse by ≥15 dB at 3,000, 4,000 or 6,000 Hz than the average thresholds at 500 and 1,000 Hz; (ii) V-shaped notch of ≥15 dB occurring at one audiometric frequency and (iii) ≥10 dB recovery at the high frequency. These criteria were translated into the following formula for the fifth algorithm:

$$\begin{array}{rl}\mathrm{P}\mathrm{T}\mathrm{T}& =(3\text{,}000\mathrm{H}\mathrm{z}-1\text{,}000\mathrm{H}\mathrm{z}\ge 15\mathrm{d}\mathrm{B})\mathrm{a}\mathrm{n}\mathrm{d}(4\text{,}000\mathrm{H}\mathrm{z}-500\mathrm{H}\mathrm{z}\ge 15\mathrm{d}\mathrm{B})\mathrm{a}\mathrm{n}\mathrm{d}\\ & (4\text{,}000\mathrm{H}\mathrm{z}-1\text{,}000Hz\ge 15\mathrm{d}\mathrm{B})\mathrm{a}\mathrm{n}\mathrm{d}(6\text{,}000Hz-500\mathrm{H}\mathrm{z}\ge 15\mathrm{d}\mathrm{B})\mathrm{a}\mathrm{n}\mathrm{d}\\ & (6\text{,}000\mathrm{H}\mathrm{z}-1\text{,}000\mathrm{H}\mathrm{z}\ge 15\mathrm{d}\mathrm{B})\mathrm{a}\mathrm{n}\mathrm{d}(8\text{,}000\mathrm{H}\mathrm{z}-3\text{,}000\mathrm{H}\mathrm{z}\ge 10\mathrm{d}\mathrm{B})\mathrm{a}\mathrm{n}\mathrm{d}\\ & (8\text{,}000\mathrm{H}\mathrm{z}-4\text{,}000\mathrm{H}\mathrm{z}\ge 10\mathrm{d}\mathrm{B})\mathrm{a}\mathrm{n}\mathrm{d}(8\text{,}000\mathrm{H}\mathrm{z}-6\text{,}000\mathrm{H}\mathrm{z}\ge 10\mathrm{d}\mathrm{B}).\end{array}$$

Finally, the sixth algorithm for normal hearing is defined using the following formula:

$$\begin{array}{rl}\mathrm{P}\mathrm{T}\mathrm{T}=& (\mathrm{N}\mathrm{o}\mathrm{t}\mathrm{H}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s})\mathrm{a}\mathrm{n}\mathrm{d}(\mathrm{N}\mathrm{o}\mathrm{t}\mathrm{H}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{I}\mathrm{m}\mathrm{p}\mathrm{a}\mathrm{i}\mathrm{r}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{t})\mathrm{a}\mathrm{n}\mathrm{d}(\mathrm{N}\mathrm{o}\mathrm{t}\mathrm{P}\mathrm{S}\mathrm{T}\mathrm{S})\mathrm{a}\mathrm{n}\mathrm{d}\\ & (\mathrm{N}\mathrm{o}\mathrm{t}\mathrm{T}\mathrm{S}\mathrm{T}\mathrm{S})\mathrm{a}\mathrm{n}\mathrm{d}(\mathrm{N}\mathrm{o}\mathrm{t}\mathrm{N}\mathrm{I}\mathrm{H}\mathrm{L}).\end{array}$$

The AUDEXCEL interface, shown in Fig. 1, enables OHDs to input audiogram data for hundreds or even thousands of workers simultaneously, delivering immediate diagnoses for ONRHD. To determine PSTS and TSTS, both baseline and annual audiograms must be available for each worker. AUDEXCEL generates separate diagnostic results for the left and right ears. It also alerts OHDs if a specific audiogram needs further actions, such as to be repeated or reported to the relevant authority. Furthermore, the tool features a real-time graphical display of the audiogram, enhancing visual interpretation and ease of analysis.

Validity and reliability tests of AUDEXCEL algorithm

In assessing the validity and reliability of the AUDEXCEL algorithm in diagnosing ONRHD, the diagnoses made by OHDs served as the benchmark or gold standard. All 320 audiograms were first analysed by 10 experienced OHDs to determine the diagnoses, whereby each OHD will receive 32 anonymised audiograms. These OHDs were recruited from a local medical university and the Ministry of Health, Malaysia. OHDs who fulfil the following criteria were recruited: (i) Actively practice occupational health service; (ii) qualified as occupational health doctor for at least 5 years; (iii) willing to provide consent for the currently study. Each of these 320 audiograms corresponds to a single worker and had at least one diagnosis. Some audiograms consisted of more than one diagnosis. Examples of audiograms and their diagnoses were presented in Table 1 below.

To evaluate the diagnostic performance of the AUDEXCEL algorithm, a 5-fold cross validation procedure was employed (Fig. 2). The dataset, comprising 320 audiograms, was randomly partitioned into five equal subsets (folds) of 64 audiograms each. In each iteration, four folds (256 audiograms) were used as training set for the algorithm, while the remaining fold (64 audiograms) served as the testing set. This process was repeated five times, with each fold used exactly once as the testing set.

Statistical analyses

The algorithm’s performance was assessed during each iteration. The final performance metrics were obtained by averaging the results across all five folds, providing a more robust and unbiased estimate of AUDEXCEL’s generalisability to new data.

The validity of the AUDEXCEL was assessed using sensitivity, specificity, positive predictive values (PPV), negative predictive value (NPV) for each diagnosis category (normal hearing, hearing impairment, hearing loss, PSTS, TSTS, and NIHL) by comparing the diagnoses made by the AUDEXCEL algorithm with those made by the OHDs.

The reliability of the AUDEXCEL algorithm will be evaluated using an inter-rater reliability test, specifically Cohen’s kappa coefficient, κ. According to Landis & Koch (1977), Cohen’s Kappa values can be interpreted as follows: values of 0 or less suggest no agreement, 0.01 to 0.20 indicate slight agreement, 0.21 to 0.40 reflect fair agreement, 0.41 to 0.60 represent moderate agreement, 0.61 to 0.80 signify substantial agreement, and 0.81 to 1.00 denote almost perfect agreement (McHugh, 2012). In the present study, a kappa value above 0.4 (i.e., moderate agreement) was considered acceptable, although the researchers expected a kappa value above 0.6 (i.e., substantial agreement). These thresholds were selected to reflect a balance between methodological rigor and practical expectations in health-related studies involving subjective judgement. All statistical analyses were performed using SPSS version 28.

Ethical consideration

The National University of Malaysia Ethics Committee granted ethical approval to carry out the study within its facilities (Ethical Application Ref: JEP-2024-871). The researchers adhered to the principles outlined in the Declaration of Helsinki as well as the Malaysian Good Clinical Practice Guidelines.

The researchers notified the workers before using their audiograms, which were recorded in the occupational health clinic’s database. Online informed consent forms were signed by the workers before analysing their audiograms. In cases where the workers decline to consent, their audiograms were not utilised for analysis, and instead, they were replaced by audiograms from other consenting workers to reach the target sample size of 320 audiograms. Online information booklets outlining the study’s objectives and procedures as well as the informed consent forms were provided to the OHDs via working email.

Strengths and limitations

This study has several notable strengths. Firstly, it was based on a dataset of 320 real workers audiograms, which enhances the clinical relevance and internal validity of the AUDEXCEL algorithm. By working with real-world data, the findings are more likely to reflect actual clinical scenarios. Secondly, as abovementioned, AUDEXCEL employs a rule-based, formula-driven approach grounded in established clinical definitions. This method ensures transparency and reproducibility, distinguishing it from black-box models such as those used in machine learning. Furthermore, the algorithm achieved perfect diagnostic performance in terms of sensitivity, specificity, PPV, and NPV for identifying normal hearing, hearing loss, PSTS, and TSTS. This demonstrates its robustness and accuracy for these conditions. Another key strength is AUDEXCEL’s simplicity and accessibility. Unlike machine learning models that require extensive training and tuning, AUDEXCEL does not rely on large datasets or computational power. This makes it especially valuable in resource-limited settings or occupational health environments where ease of use and interpretability are critical. Additionally, the use of standardised diagnostic criteria, such as Hoffman et al. (2006) as well as McBride & Williams (2001) criteria for NIHL, provides a consistent basis for classification and facilitates comparison with published benchmarks.

Despite its strengths, the study also has important limitations. First of all, the diagnostic performance of AUDEXCEL was not as strong for NIHL, which may be due to variability in the criteria used by different OHDs. While AUDEXCEL follows the Hoffman et al. (2006) as well as McBride & Williams (2001) criteria, other OHDs may use alternative frameworks, which differ slightly in their diagnostic thresholds. As a result, discrepancies between AUDEXCEL and the clinical gold standard may reflect differing diagnostic philosophies rather than true misclassification. Secondly, since the same dataset was used both to develop and validate the algorithm, there is a risk of overfitting, which could lead to overly optimistic performance metrics. To mitigate overfitting within the dataset, the researchers employed a five-fold cross-validation approach, where the dataset was partitioned into five subsets. Each subset was used once as a validation set, while the remaining subsets were used for training the AUDEXCEL algorithm. Thirdly, the algorithm provides binary classifications (e.g., hearing loss vs. no hearing loss), which may not fully capture the spectrum or severity of hearing impairment as defined by the WHO. Another limitation is that while the AUDEXCEL algorithm was developed and validated based on audiogram patterns for diagnostic purposes, it does not directly incorporate the specific environmental noise exposure parameters such as duration, intensity (dB level), or types of noise at the workplace. The AUDEXCEL algorithm also does not account for supra-threshold auditory deficits, such as cochlear synaptopathy (Hockley et al., 2023).

Recommendations for future study

To enhance the utility and applicability of AUDEXCEL, several areas of improvement and future research are recommended for future studies. Firstly, future versions of AUDEXCEL could incorporate multiple diagnostic criteria for NIHL, allowing users to select or compare between criteria such Coles, Lutman & Buffin (2000). This flexibility would accommodate variations in clinical practice and improve agreement with diverse gold standards. Secondly, to address the risk of overfitting, it is recommended that AUDEXCEL be validated using an independent external dataset, ideally sourced from different clinical settings or populations. This would provide a more robust evaluation of its generalisability and performance in real-world us. Thirdly, future studies may investigate and expand the algorithm’s capability to classify degrees and types of hearing loss (e.g., mild, moderate, mixed, conductive vs. sensorineural), which would enhance its clinical value and align it more closely with audiological standards. Finally, the future development of AUDEXCEL could consider integrating workplace exposure parameters such as noise exposure duration, noise intensity, type of noise, and usage of personal hearing protection devices to improve its diagnostic utility and to support causal attribution more robustly.