Unbiased machine learning-assisted approach for conditional discretization of human performances

Related work

In the broader context of various domains, existing discretization methods (Baron & Stańczyk, 2020; García et al., 2013; Dash, Paramguru & Dash, 2011) were designed to transform continuous data into nominal data either during the preprocessing or postprocessing stages of data mining. These methods are categorized as supervised or unsupervised schemes. Supervised algorithms incorporate class information, often in the form of entropy, to determine cut-points to partition data space. In contrast, unsupervised algorithms do not consider class information. Instead, they may evenly divide the data range into a specified number of bins (namely equal-width method) or allocate an equal number of continuous values into each bin (namely equal-frequency-binning method). Yang & Webb (2008) proposed unsupervised discretization methods that aimed to reduce classification bias or error. However, all of existing discretization methods predominantly prioritized the performance enhancement of general data classification rather than ensuring unbiasedness in the context of human performance.

Extensive research in human resource management evaluated personnel’s overall performance by classifying various performance factors into a set of predefined performance ranking labels (e.g., excellent, good, regular, and insufficient). The classification tasks leveraged machine-learning classifiers (e.g., decision tree, k-nearest neighbors, support vector machine, naive bayes, and artificial neural network) and/or fuzzy logic using class labels created via either norm- or criterion-referenced methods. However, these efforts primarily focused on the discretization of multiple raw performance values rather than single inclusive performance values. Furthermore, some of these studies were conducted without addressing bias issues such as Sharma & Goyal (2015), Pan (2021), and Ahned & Sultana (2013) whereas others were aware of biasedness at the level of either human performance indicators or algorithmic operations such as Nayem & Uddin (2024) and De Oliveira Góes & De Oliveira (2020). Apart from the classification-based discretization, there is limited research (Dang, Truong & Huynh, 2021; Huang & Wang, 2022; Prasad, Choudhary & Ankayarkanni, 2020) employing clustering techniques to identify overall similarities among multiple raw performance factors and align them with performance ranking labels to gain insights into personnel performances and enhance human resource management. Unfortunately, these clustering-based discretization approaches overlooked the aspect of unbiasedness. Furthermore, these research endeavors both classification-based and clustering-based entirely engaged UPD instead of CPD.

Prior norm-referenced performance discretization methods are primarily utilized in the field of education, particularly in scenarios where exams cannot comprehensively assess all learning topics due to time constraints and limited grading resources. In such cases, norm-referenced approaches are preferred over criterion-referenced schemes (Banditwattanawong & Masdisornchote, 2021). These methods include Arora & Badal (2013), Borgavakar & Shrivastava (2017), Parveen, Alphones & Naz (2017), Shankar et al. (2016), Xi (2015), Iqbal et al. (2019), Ramen & Joachims (2014), and Bai & Chen (2006). They mostly utilized K-means clustering as reviewed in previous work (Banditwattanawong, Jankasem & Masdisornchote, 2023). Recently, Omar, Alzahrani & Zohdy (2020) leveraged K-means and the elbow to partition students’ performance data, consisting of course name, course grade, and cumulative GPA, into groups for developing improvement plans. Pamungkas, Dewi & Putri (2024) also utilized K-means and the elbow method to cluster students’ GPAs, credits taken, and the number of poor grade symbols into three groups: high achievers, average performance, and needs improvement, for developing challenging or remedial programs. In previous studies (Banditwattanawong & Masdisornchote, 2020a, 2020b, 2021), we proposed a heuristic approach for norm-referenced grading and were the first to demonstrate the application of K-means and partitioning around medoids (PAM) for this purpose. We evaluated the grading quality of the heuristic, Z-score, K-means, and PAM based on Davies-Bouldin index without the notion of fairness. Our seminal article (Banditwattanawong, Jankasem & Masdisornchote, 2023) proposed a norm-referenced achievement grading method that focused on grading fairness. This approach is, however, limited to UPD. In summary, none of these norm-referenced educational assessments aimed for CPD unbiasedness, and major changes must be applied to these studies to achieve CPD.

Table 1 summarizes these related works and their shortage in addressing our research problem of unbiasedness-centric CPD.

Definition of conditional performance discretization

It is crucial to establish a rigorous foundation for proposing CPD algorithms and evaluating their performance. This section formally defines CPD that is applicable to both norm-referenced and criterion-referenced performance discretization schemes. Let P be a set of one-dimensional performance values, L be a set of assignable performance-ranking labels, L_a be a set of performance-ranking labels assigned to P, and $f_{CPD}$() denote a CPD function that belongs to either norm-referenced or criterion-referenced scheme, CPD implements Eq. (1). Note that L_a ⊂ L indicates that L_a is a proper subset of L.

$$l\in L_a\leftarrow f_{CPD}(p\in P)$$

(1) $$\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}{L}_a\subset L.$$

This definition will be empirically exemplified through the following section.

Conditional unbiasedness requirements and metric for norm-referenced CPD

Intuitively, the norm-referenced CPD of descendingly-sorted performance values will achieve exact unbiasedness if three following requirements are completely fulfilled. Let PRL represent a linguistic performance-ranking label and PVI denote a performance-value interval, defined as a difference between the maximum and minimum performance values associated with the same PRL.

• Requirement 1: Every gap between adjacent performance values associated with two unique PRLs, including a gap between the top performance value and its upper bound, as well as a gap between the worst performance value and its lower bound that exceeds the largest PVI preserves a certain unique PRL from being assigned. This realizes the CPD property specified by Eq. (1).

• Requirement 2: Every gap between contiguous performance-value boundaries associated with two unique PRLs is maximized. This prevents situations where any two closely situated performance values receive disparate PRLs, which may signify bias in a CPD process.

• Requirement 3: All PVIs are uniform. This ensures that all assigned PRLs have the comparable chances of being assigned.

Conditional unbiasedness is achieved when the CPD process fairly assigns PRLs without favoring specific intervals of performance values. This is mathematically formalized through Eq. (2). These three requirements ensure conditional unbiasedness in Norm-Referenced CPD by preventing artificial PRL assignment in sparse performance regions (Requirement 1), ensuring that performance value gaps between PRLs do not create unfair distinctions (Requirement 2), and distributing PRLs evenly across performance intervals (Requirement 3). Requirements 2 and 3 are also common for UPD and were demonstrated in Banditwattanawong, Jankasem & Masdisornchote (2023).

We further clarify Requirement 1 via a simplified example presented in Table 2. In this example, a vector of 14 performance values is mapped to a vector of PRLs <A, B, C, D, E, F, G, H, I> by using a possible norm-referenced CPD method (A₀). At an initial execution stage of A₀, the results contain the same number of PVIs as the unique PRLs and include the gaps (four instances of 14 in bold) larger than the largest PVI of 5. This indicates that Requirement 1 has not yet been satisfied, thus requiring further adjustment of PRLs. To fulfill Requirement 1, specific unique PRLs must be sacrificed for these gaps by being excluded from assignment. In the final execution stage, A₀ not only excludes A from being assigned due to the first gap of 14 but also reduces the number of PVIs by four causing the performance values 96, 92, 86, and 82 to be embraced into the same PVI associated with PRL B. Subsequently, the second gap of 14 between 82 and 77 excludes the associated PRL C from being assigned, while 77, 63, and 58 are grouped into the same PVI associated with a following PRL D. Likewise, the third gap of 14, following E, excludes F from assignment, and the last gap of 14, following H, excludes I from assignment. Consequently, no gap in the final stage exceeds the updated largest PVI of 19, thereby satisfying Requirement 1.

When applying norm-referenced CPD to real-world human performances, each of these requirements is often partially met. Hence, quantifying the attained degree of conditional unbiasedness in the deployment of norm-referenced CPD becomes beneficial for identifying the optimal CPD approach. Conditional unbiasedness metric Ω′ that measures the extent of conditional unbiasedness exhibiting in the outputs of a norm-referenced CPD approach can be figured out via Eq. (2). Ω′ is derived as the product of metrics, Ω₁, Ω₂, and Ω₃, measuring the degrees of accomplishment for the Requirements 1 to 3, respectively. A higher Ω′ value indicates that the CPD process has minimized potential biases, thereby ensuring a robust and equitable evaluation system.

(2) $$\begin{array}{rl}& {\mathrm{\Omega }}^{\mathrm{\prime }}={\mathrm{\Omega }}_1\times {\mathrm{\Omega }}_2\times {\mathrm{\Omega }}_3\\ & {\mathrm{\Omega }}_1=\{\begin{array}{c}1-{\displaystyle \frac{\left|\mathrm{\Theta }-\theta \right|}{\mathrm{\Theta }}if\mathrm{\Theta }\ge 1}\\ 1,otherwise\end{array}\\ & {\mathrm{\Omega }}_2=\{\begin{array}{c}{\displaystyle \frac{{\sum }_{i=1}^{N-1}{\delta }_i-{\sum }_{i=1}^{N-1}{\mathrm{\Delta }}_i^{\left(min\right)}}{{\sum }_{i=1}^{N-1}{\mathrm{\Delta }}_i^{\left(max\right)}-{\sum }_{i=1}^{N-1}{\mathrm{\Delta }}_i^{\left(min\right)}}ifN\ge 3}\\ 1,otherwise\end{array}\\ & {\mathrm{\Omega }}_3=\{\begin{array}{c}{\displaystyle \frac{1}{1+\sigma }ifN\ge 2}\\ 1,otherwise\end{array}\end{array}$$where N denotes the number of unique PRLs assigned to performance values, δ_i denotes a gap between the lower-bound performance-value of the i^th PRL and the upper-bound performance-value of the consecutive (i+1)^th PRL (assuming performance values are sorted in descending order corresponding to that of the PRLs), ${\mathrm{\Delta }}_i^{\left(min\right)}$ denotes the i^th narrowest gap between any adjacent performance values, ${\mathrm{\Delta }}_i^{\left(max\right)}$ denotes the i^th widest gap between any adjacent performance values, σ denotes the standard deviation of the PVIs of all assigned unique PRLs, θ denotes the number of unassigned unique PRLs, and Θ denotes the number of both δ_i and a gap (γ_u) between the top performance value and its upper bound as well as a gap (γ_l) between the worst performance value and its lower bound that are larger than a PVI (denoted by ${\nu }_i^{\left(I\right)}$), which is widest and associated with a unique PRL. The value ranges of Ω₁ is [0.0, 1.0], while Ω₂, Ω₃ have the range of (0.0, 1.0].

The rationale behind the calculation of Ω′, which is [0.0, 1.0], is as follows. Since Θ notates the count of PRLs that ought to remain unassigned, θ /Θ in Ω₁ indicates the amount of CPD efficiency in satisfying Requirement 1 when assigning PRLs. If Ω₁ equals 1.0, ${\sum }_{i=1}^{N-1}{\delta }_i$ in Ω₂ reaches its upper bound, ${\sum }_{i=1}^{N-1}{\mathrm{\Delta }}_i^{\left(max\right)},$ (i.e., Requirement 2 is met), and σ in Ω₃ is 0 (i.e., Requirement 3 is satisfied), Ω′ will be 1.0 indicating none of conditional biasedness. In contrast, as the term θ /Θ approaches 0, and/or ${\sum }_{i=1}^{N-1}{\delta }_i$ approaches its lower bound, ${\sum }_{i=1}^{N-1}{\mathrm{\Delta }}_i^{\left(min\right)}$, and/or σ approaches ∞, Ω′ will approach 0 indicating the low degree of conditional unbiasedness. Thus, a high value of Ω′ signifies a greater extent of conditional unbiasedness.

For clarification, Table 3 demonstrates the application of Eq. (2) through a vector of 30 descendingly-sorted performance values, a vector of assignable PRLs in a descendingly qualitative order <A, B, C, D, F>, and two possible norm-referenced CPD methods A1 and A2. Since A1 and A2 assign 4 and 3 PRLs, respectively, to different PVIs, the Ω′ values of CPD outputs delivered by A1 and A2 where N = 4 and 3, respectively, are calculated as follows:

$$\begin{array}{rl}A1:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{\left|2-1\right|}{2}})\times {\displaystyle \frac{\left(10-0\right)}{\left(10-0\right)}\times {\displaystyle \frac{1}{\left(1+5.07\right)}=0.08}}\\ A2:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{\left|2-2\right|}{2}})\times {\displaystyle \frac{\left(6-0\right)}{\left(7-0\right)}\times {\displaystyle \frac{1}{\left(1+2.89\right)}=\mathrm{0.22.}}}\end{array}$$

The reason for A2 exhibiting a higher degree of conditional unbiasedness compared to A1 is that A2 more effectively satisfies Requirements 1 and 3.

Norm-referenced CPD methodology

This section proposes four unbiasedness-centric methods for norm-referenced CPD: widest-gap-first CPD method (WGF-CPD), K-means CPD method (K-CPD), Partitioning-Around-Medoids CPD method (PAM-CPD), and multi-modal CPD method (M-CPD).

Widest-gap-first CPD method

Starting with all performance values, this method employs a heuristic approach, namely widest-gap-first, to identify the largest gaps (i.e., the widest δ_i) for categorizing performance values when assigning PRLs. The method iteratively reduces the number of PRLs assigned to ensure compliance with CPD requirements. Performance values falling into distinct categories are assigned different PRLs. WGF-CPD is simple and interpretable compared to the other CPD methods.

The method is outlined in Algorithm 1. Line 1 calculates gaps between every pair of adjacent performance values and select only (the total number of unique PRLs – 1) widest gaps. If there are multiple identical widest gaps, the one leading to more uniform PVIs (minimizing σ in Eq. (2)) will be selected. In line 2, heuristicPD() discretizes performance values based on the widest-gap-first heuristic approach by initially assigning all unique PRLs to all performance values grouped according to the |l|−1 widest gaps. Line 3 determines PVIs associated with each unique PRL. Line 4 calculates gaps γ_u, γ_l, and δ_{i=1 ..|l|−1}. Line 5 counts the number of gaps from Line 4 that their gap sizes equal or exceed the largest PVI from Line 3. Line 6 sets the number of unassigned PRLs initially to zero. Line 7 calculates Eq. (2). Line 8 sets the positive-integer index j (representing the number of sacrificed or unassigned PRLs) to 1. Line 9 iterates lines 10 to 17 only if the condition j ≤ Θ holds true. Each iteration sacrifices a unique PRL for each of the widest gaps equaling or exceeding the largest PVI by incrementing j by 1 from 1 to Θ. Lines 10 to 17 execute similar operations to lines 1 to 8, but constrained to the |l|-1−j widest gaps. Each iteration implies that there is a gap in g wider than maxElement(ν^(I)) causing heuristicPD() in line 11, which is the overloading variant of heuristicPD() in line 2, to sacrifice a respective PRL in l for some gap not only in g^(max) but also γ_u and γ_l exceeding the widest PVI and totally assigns |l|−j PRLs in each iteration. (The process of excluding PRLs from assignment has been demonstrated through Table 2.) Notice that the iteration might terminate when Θ is still greater than 0 meaning that Requirement 1 fulfillment is only partial. Line 15 updates θ value to be used for Ω′ calculation in the next iteration. Line 17 increases j by 1. Line 18 identifies the index value of k for which ${\mathrm{\Omega }}_{k=\mathrm{0..}j-1}^{\mathrm{\prime }}$ is maximized. Line 19 returns the output vector of PRL instances, delivering the maximum ${\mathrm{\Omega }}_k^{\mathrm{\prime }}$, that are aligned with each performance value in the input ν one by one. In fact, if Θ in line 5 equals 0, the while loop will be bypassed causing Algorithm 1 to function as a UPD method.

The practical cost-effectiveness of Algorithm 1 is analyzed in terms of worst-case time complexity as follows. Let n and N be the numbers of performance values (i.e., |v|) and unique PVLs (i.e., |l|), respectively. To sort input v and l takes n $lo{g}_2$n and N $lo{g}_2$N, respectively. Line 1 takes (N-1) $lo{g}_2$n. Line 2 takes n. Lines 3–5 identically take n. Line 7 is 3N+n $lo{g}_2$n to figure out all parameters and calculate Eq. (1). Line 9 iterates at most N+1 rounds (as Θ is bounded by |g|, which is at most N+1) of lines 10–17, which are equivalent to lines 2–8. Line18 takes N+1. Line 19 takes n. Therefore, WGF-CPD algorithm is O(n $lo{g}_2$n + N $lo{g}_2$N + (N−1) $lo{g}_2$n + n + 3n + 3N+n $lo{g}_2$n + (N+1)((N−1) $lo{g}_2$n + n + 3n + 3N + n $lo{g}_2$n) + N+1+n) that is O(n $lo{g}_2$n) as N << n.

K-means CPD method

This method engages K-means (Witten & Frank, 2016) for partitioning performance values into a predefined number, K, of clusters, each represents a unique PRL. Unlike supervised methods, this approach does not use labeled data but instead minimizes the variance within clusters to optimize grouping. In the context of CPD, the number of clusters is fewer than the number of assignable unique PRLs. The method iteratively adjusts clusters to align with performance gaps and ensures compliance with the CPD requirements, particularly focusing on equalizing PVIs while maintaining the widest-gap constraints. This ensures unbiased distribution, as measured by Ω′.

In specific, K-means method minimizes an objective function ${\sum }_{j=1}^K{\sum }_{i=1}^{n_j}\left|x_i-c_j\right|$ where n_j is the number of performance values in cluster j, x_i is a performance value in j with a centroid c_j, and |x_{i −} c_j| is Euclidean distance. K-CPD method invokes K-means more than once to produce CPD results and the best one with the highest Ω′ is finally returned as showed in Algorithm 2. Several lines of code are the identical to those in Algorithm 1 and have been explained with the exception of the followings. In line 1, KmeansPD() performs performance discretization by assigning all unique PRLs to all performance values grouped by using K-means method into |l| clusters. During each iteration, the overloading function KmeansPD() will not assign a unique PRL corresponding to a performance value gap between U and L equal to or wider than maxElement(ν^(I)). As long as the condition j ≤ Θ in line 8 is not evaluated to false, the algorithm continues to invoke K-means algorithm to re-cluster the performance values into |l|−j clusters. In each iteration of the while-loop, the resulting PRL-instance vector l contains a unique set of PRLs or clusters that differ from those in other iterations and is assumed to be stored. Finally, in lines 16–17, the K-CPD algorithm returns the vector of PRL instances that achieves the highest conditional unbiasedness.

The time complexity of Algorithm 2 is analyzed as follows. To sort input v and l takes n $lo{g}_2$n and N $lo{g}_2$N, respectively. Line 1 takes cNn where N is equivalent to the number of clusters and hyper parameter c is the bound number of internal iterations in each complete K-means execution. Lines 2–4 identically take n. Line 6 is 3N+n $lo{g}_2$n. Line 8 iterates at most N+1 rounds of lines 9–15, which are equivalent to lines 1–7. Line 16 takes N+1. Line 17 takes n. Therefore, K-CPD algorithm is O(n $lo{g}_2$n) as N << n. However, for large data sets, K-CPD becomes sensitive to the initial placement of centroids, potentially leading to inconsistent results across different runs. This issue can be mitigated by replacing random initialization with more reliable techniques, such as K-Means++.

Partitioning-around-medoids CPD method

This method adapts K-CPD by replacing K-means with PAM (Kaufmann & Rousseeuw, 1987), which enhances robustness to outliers. PAM is different from K-means in how it selects cluster centers, as PAM choose actual data points (medoids) as cluster centers instead of means. This makes PAM more robust to outliers, whereas K-means is sensitive to extreme values that can distort cluster assignments. However, this robustness comes at a computational cost, as PAM has a higher complexity of O(n²N), making it slower than K-means, which operates at O(cNn). Unlike K-means, which can produce different results due to random initialization, PAM provides more stable clustering outcomes. Thus, PAM is preferred for small datasets, while K-means is better suited for large-scale clustering tasks requiring efficiency.

Algorithm 3 details PAM-CPD method. The algorithm differs from Algorithm 2 in lines 1 and 9 substituting PAM_PD() for KmeanPD(). PAM_PD() discretizes the performance vector v by employing PAM method into |l| and |l|−j PRLs or clusters. Each while loop generates at most Θ PVL vectors. The algorithm returns a PVL vector with the maximum Ω′.

The time complexity of Algorithm 3 can be derived by replacing cNn with n²N. This makes PAM-CPD, which is O(n²), still tractable.

Multi-modal CPD method

M-CPD method leverages both WGF-CPD, K-CPD, and PAM-CPD methods to process the same set of inputs and return the results of either constituent method with the highest Ω′. M-CPD method takes the number of assignable unique PRLs to specify not only the initial number of clusters determined by K-CPD and PAM-CPD methods but also the initial number of PVIs into which WGF-CPD method divides performance values. M-CPD method employs conditional-unbiasedness maximization as its decision function, which is grounded in Ω′ to quantify the degree to which a norm-referenced CPD process satisfies the three core requirements justified below. Particularly, the decision function evaluates the results of WGF-CPD, K-CPD, and PAM-CPD, and selects the one with the highest Ω′, ensuring optimal conditional unbiasedness.

• Requirement 1: Unassigned PRLs are maximized for gaps larger than or equal to the widest PVI, ensuring no bias arises from arbitrary assignments in sparsely populated regions.

• Requirement 2: Gaps between assigned PRLs are maximized, preventing closely spaced performance values from receiving different PRLs, which could introduce unfair distinctions.

• Requirement 3: Uniformity of PVIs is maintained to equalize the likelihood of PRLs being assigned across the performance spectrum.

Algorithm 4 provides the details of M-CPD method. Lines 1 to 3 call WGF-CPD, K-CPD, and PAM-CPD algorithms, respectively, to process input performance values and PRLs. In line 4, the algorithm implements the maximization of its decision function to identify the best CPD result among a pair generated by the constituent algorithms. Note that getConditionalUnbiasedness() in line 4 overloads the function getConditionalUnbiasedness() in Algorithms 1, 2, and 3 by calculating θ and Θ merely from the input parameters ν, $l_j^{\left(o\right)}$, U, and L.

The worst-case time complexity of Algorithm 4 is as follows. To sort input v and l takes n $lo{g}_2$n and N $lo{g}_2$N, respectively. Lines 1 and 2 equally takes n $lo{g}_2$n. Line 3 takes n². Line 4 is 2(3N+n $lo{g}_2$n). Therefore, M-CPD algorithm totally takes 3n $lo{g}_2$n + N $lo{g}_2$N + n² + 2(3N+n $lo{g}_2$n) equaling O(n²).

Evaluation

The proposed methods are assessed by using open data sets and analyzed in terms of conditional unbiasedness as outlined below.

Data set and preprocessing

Multiple data sets from human-performance evaluation domains, each of which consists of one or more fields, are preprocessed to derive stakeholder-interpretable and one-dimentional performance values as follows. The statistical characteristics of each data set are described via a vector of <maximum performance value, minimum performance value, performance value mean, standard deviation, distribution skewness>, providing a comprehensive summary of the data’s central tendency, variability, and shape.

The first data set, entitled EMP1, concerns employee evaluation for promotion consideration. To derive EMP1, the following fields were selected from the raw data set (Zaman, 2022) and max–min normalized into a range of 0.00 and 1.00: previous year rating (between 1 and 5), length of service in years, won awards (1 for won, 0 otherwise), number of trainings (ranging from 1 to 10), and average training score (between 0 and 100). Subsequently, the normalized values were summed to obtain individual performance values ranging from 0.00 to 5.00. The sampled number of employees evaluated is 100. These 100 preprocessed performance values will be then discretized conditionally into three promotional levels as suggested by the raw data set’s meta data: executive (denoted by A), higher-qualified professional (B), and non-qualified promotion (C). The statistical characteristics of this data set are <3.00, 0.79, 1.97, 0.46, −0.84>.

The second data set, namely EMP2, pertains to employee performance rating correlated with salary increases. To obtain EMP2, the raw data set (Kumar, 2022) including the field of percentage salary hikes was filtered in randomly only 60 records of employees. The preprocessed data will be discretized conditionally into performance ratings ranging from 1 to 4. A lower rating corresponds to a greater salary increase. The statistical characteristics of this data set are <24, 11, 15.66, 3.64, 0.25>.

The third data set, named STD1, involves the evaluation of performance among higher education students. In the raw data set (Yilmaz & Sekeroglu, 2022), a field representing the 1 to 5 ratings of cumulative grade point average (CGPA): 1 for < 2.00 CGPA, 2 for 2.00–2.49 CGPA, 3 for 2.50–2.99 CGPA, 4 for 3.00–3.49 CGPA, and 5 for above 3.49 CGPA. This field was sampled into 50 records and transformed from interval ratings to random CGPAs by applying randomization. These preprocessed CGPA values are to be discretized conditionally into grade symbols: AA, AB, BB, BC, CC, CD, DD, and Fail. The statistical characteristics of STD1 are <3.46, 0.18, 2.45, 0.86, −1.00>.

The final dataset, STD2, represents student performance in exams. It was derived by weighting math, reading, and writing scores from the raw dataset (Chauhan, 2023), where each subject has a full score of 100. A subset of 40 samples was selected for conditional discretization into five grade symbols (A, B, C, D, and F). The statistical characteristics of STD2 are <90, 23, 53.28, 22.30, 0.17>.

Experimental setup

The experiments were conducted in the following steps to evaluate the proposed norm-referenced CPD methods, WGF-CPD, K-CPD, PAM-CPD, and M-CPD. First, each preprocessed dataset was conditionally discretized separately using all proposed methods. The algorithmic processing is also concisely clarified. Subsequently, Ω′ value was computed for each combination of method and data set to quantify the conditional-unbiasedness performance of each method. Finally, the results are evaluated by comparison across the methods.

CPD result

The CPD results for each proposed method are presented via each complete data set in Table 4 onward. The conditional unbiasedness degrees are measured to observe the contributions of the constituent methods to the final results delivered by M-CPD method.

In Table 4, EMP1 data set is processed independently by WGF-CPD, K-CPD, and PAM-CPD methods as well as baseline Z-score method as follows.

Initially, WGF-CPD method discretizes the performance values into three PRLs A, B, and C, each of which is associated with the performance values [3.00], [2.80–0.91], and [0.79], respectively, resulting in PVIs of <0.00, 1.89, 0.00> (not showed in Table 4). However, there remains the gap γ_u of 2.00 wider than the maximum PVI (1.89) and preceding the maximum performance value, thus PRL A should have been sacrificed. As a result, WGF-CPD re-discretizes the performance values into only 2 PRLs, B and C, as showed in Table 4. Notably, no gap wider than the maximum PVI (2.01) remains, leading to the completion of the method.

K-CPD method begins by grouping the performance values into 3 clusters: [3.00, 2.22], [2.20, 1.70], and [1.65, 0.79], associated with PRLs A, B, and C, respectively (not shown in Table 4). Since all PVIs <0.78, 0.50, 0.86> are smaller than γ_u, K-CPD re-groups the performance values into 2 clusters, [3.00, 1.77] and [1.70, 0.79], associated with B and C, respectively, as demonstrated in Table 4. However, the widest PVI of 1.23 still remains smaller than γ_u although K-CPD terminates since the condition in line 8 of Algorithm 2 evaluates to false (i.e., the number of excluded PRLs reached Θ).

PAM-CPD method initially groups the performance values into 3 clusters: [3.00, 1.04], [0.94, 0.91], and [0.79, 0.79], associated with PRLs A, B, and C, respectively, (not showed in Table 4). Since all PVIs <1.96, 0.03, 0.00> are smaller than γ_u, PAM-CPD re-groups the performance values into two clusters, [3.00, 0.91] and [0.79, 0.79], associated with B and C, respectively, as demonstrated in Table 4. The widest PVI of 2.09 is larger than γ_u, causing Θ to become 0 and terminating the while loop in Algorithm 3.

The CPD results in Table 4 produced by WGF-CPD, K-CPD, PAM-CPD, and Z-score methods exhibit different degrees of Ω′ as calculated below based on Eq. (2) with N = 2. Greater Ω′ value indicates that K-CPD offers higher degree of conditional unbiasedness. Consequently, M-CPD method returns the CPD result produced by constituent K-CPD. The baseline method, Z-score, fully assigns PVLs although γ_u is greater than 0.73, resulting in zero Ω₁, which leads to a failure in performing CPD for this data set.

$$\begin{array}{rl}\text{WGF-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=1\times 1\times {\displaystyle \frac{1}{1+1.42}=0.41}\\ \text{K-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{0}{1}})\times 1\times {\displaystyle \frac{1}{1+0.23}=0.82}\\ \text{PAM-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=1\times 1\times {\displaystyle \frac{1}{1+1.48}=0.40}\\ \text{Z-score}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=0\times {\displaystyle \frac{0.04-0.00}{0.32-0.00}\times {\displaystyle \frac{1}{1+0.01}=\mathrm{0.00.}}}\end{array}$$

By applying WGF-CPD, EMP2 data set is conditionally discretized into PRLs 1 to 4, corresponding to performance value ranges [24, 24], [22, 17], [15, 13], and [11, 11] with PVIs 0, 5, 2, and 0, respectively. However, the gap γ_l equals 6, larger than the maximum PVI of 5 resulting in WGF-CPD method reallocating PRLs 1, 2, and 3 to performance value ranges [24, 17], [15, 13], and [11, 11] with adjusted PVIs 7, 2, and 0, respectively, as illustrated in Table 5. This ensures that γ_l does not exceed the maximum PVI of 7, thus completing WGF-CPD operation.

K-CPD method initially assigns PVLs 1 to 4 to the performance value ranges [24, 20], [19, 17], [15, 13], and [11, 11] with PVIs of 4, 2, 2, 0, respectively. Consequently, the maximum PVI of 4 smaller than γ_l causes K-CPD to consolidate the performance values into three clusters [24, 20], [19, 17], and [15, 11] associated with PVLs 1, 2, and 3, respectively. Despite this adjustment, γ_l still exceeds the maximum PVI of 4 whereas K-CPD terminates.

PAM-CPD method begins by assigning PVLs 1 to 4 to the performance value ranges [24, 18], [17, 17], [15, 13], and [11, 11] with PVIs of 6, 0, 2, 0, respectively. Consequently, the maximum PVI of 6 equaling γ_l causes PAM-CPD to consolidate the performance values into three clusters [24, 17], [15, 13], and [11, 11] associated with PVLs 1, 2, and 3, respectively. Consequently, γ_l is smaller than the maximum PVI of 7 and PAM-CPD terminates.

The CPD outputs in Table 5 where WGF-CPD and PAM-CPD methods deliver the identical results possess the following Ω′, indicating that K-CPD yields higher degree of conditional unbiasedness. Consequently, M-CPD method selects the CPD result of K-CPD. Z-score method assigns all PVLs although γ_l is greater than 3, resulting in zero Ω₁.

$$\begin{array}{rl}\text{WGF-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=1\times {\displaystyle \frac{4-0}{4-0}\times {\displaystyle \frac{1}{1+3.606}=0.217}}\\ \text{K-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{0}{1}})\times {\displaystyle \frac{3-0}{4-0}\times {\displaystyle \frac{1}{1+1.155}=0.348}}\\ \text{PAM-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=1\times {\displaystyle \frac{4-0}{4-0}\times {\displaystyle \frac{1}{1+3.606}=0.217}}\\ \text{Z-score}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=0\times {\displaystyle \frac{3-0}{6-0}\times {\displaystyle \frac{1}{1+0.577}=0.000}}\end{array}$$

Initially, STD1 data set is discretized by WGF-CPD method into eight PVLs representing performance value ranges [3.46, 3.04], [2.88, 2.50], [2.36, 2.00], [1.78, 1.71], [1.55, 1.55], [1.33, 1.27], [0.93, 0.93], and [0.37, 0.18] with the largest PVI of 0.42, which is smaller than the widest gap of 0.56. Consequently, WGF-CPD excludes AA from PVLs resulting in an adjusted CPD result as showed in Table 6 along with the largest PVI of 0.88 larger than the widest gap of 0.56. In contrast, K-CPD method begins by discretizing STD1 into eight PVLs with the largest PVI of 0.40, less than γ_u. Even after K-CPD unassigns AA, the newly largest PVI (0.45) still remains smaller than the widest gap of 0.56 as depicted in Table 6. PAM-CPD discretizes STD1 into eight PVLs with the largest PVI of 1.35, greater than γ_u. Therefore, PAM-CPD does not exclude any PVL from assignment at all.

Based on Eq. (2), the outputs obtained via WGF-CPD method manifests higher Ω′, causing M-CPD method to return WGF-CPD’s result. Z-score method assigns all PVLs whereas γ_u is greater than 0.37, resulting in zero Ω′.

$$\begin{array}{rl}\text{WGF-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=1\times \frac{1.66-0.00}{1.66-0.00}\times {\displaystyle \frac{1}{1+0.32}=0.76}\\ \text{K-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{0}{1}})\times {\displaystyle \frac{1.26-0.00}{1.66-0.00}\times {\displaystyle \frac{1}{1+0.12}=0.68}}\\ \text{PAM-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=1\times {\displaystyle \frac{1.58-0.00}{1.80-0.00}\times {\displaystyle \frac{1}{1+0.46}=0.60}}\\ \text{Z-score}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=0\times {\displaystyle \frac{1.64-0.00}{1.80-0.00}\times {\displaystyle \frac{1}{1+0.13}=\mathrm{0.00.}}}\end{array}$$

The last data set STD2 undergoes two rounds of conditional discretization using WGF-CPD method, which eventually excludes PRL F as depicted in Table 7. Similarly, K-CPD and PAM-CPD methods re-assigns only four PRLs as illustrated in Table 7. Their Ω′ values are calculated below resulting in M-CPD method returning the CPD result from PAM-CPD. Z-score method assigns all PVLs regardless of γ_l greater than 23, resulting in zero Ω₁.

$$\begin{array}{rl}\text{WGF-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{0}{1}})\times {\displaystyle \frac{33-0}{33-0}\times {\displaystyle \frac{1}{1+8.70}=0.10}}\\ \text{K-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{0}{1}})\times {\displaystyle \frac{24-0}{33-0}\times {\displaystyle \frac{1}{1+7.80}=0.08}}\\ \text{PAM-CPD}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=(1-{\displaystyle \frac{0}{1}})\times {\displaystyle \frac{30-0}{33-0}\times {\displaystyle \frac{1}{1+7.27}=0.11}}\\ \text{Z-score}:& {\mathrm{\Omega }}^{\mathrm{\prime }}=0\times {\displaystyle \frac{33-0}{40-0}\times {\displaystyle \frac{1}{1+3.11}=0.00}}\end{array}$$

Outlier sensitivity analysis

Since empirical assessment generally provides performance evaluation based on limited datasets, the following analyses of algorithmic sensitivity to outliers (and noises) ensure broader generalizability across dataset variations.

Outliers are extreme performance values that significantly deviate from main data distribution. A well-designed CPD method should minimize their impact on PRL assignments while preserving fairness in ranking.

• WGF-CPD: Outliers create exaggerated gaps between performance values. Since WGF-CPD selects the widest gaps for segmentation, a single outlier can disrupt PRL distribution by unintentionally excluding either the first or the last assignable PRL. As outliers significantly increase the largest gaps, the method will over-segment performance values, leading to each boundary PRL covering a small or even zero PVI in a region where the outliers are present. This uneven distribution of PVIs undermines the fairness and consistency of the discretization process. Additionally, the unintended exclusion of certain PRLs reduces the granularity of the CPD, limiting its ability to accurately reflect the relative performance of individuals within the dataset.

• K-CPD: This K-means-based method inherently mitigates outlier influence by forming groups around majority performance data points. Outliers can pull centroids away from the main clusters, leading to distorted PRL assignments.

• PAM-CPD: Unlike K-means, which calculates centroids as means, PAM selects actual data points (medoids) as cluster centers, making it more robust to outliers and preserving the integrity of PRL assignments. This is evident in STD2 where the four performance values of 90, considered outliers, are not assigned a distinct PVL as seen in K-CPD’s results.

• M-CPD: This method adaptively selects WGF-CPD, K-CPD, or PAM-CPD result with the highest Ω′ metric, inherently mitigating the impact of outliers. If outliers severely distort WGF-CPD results, M-CPD will favor either K-CPD or PAM-CPD output. However, if outliers also affect K-CPD, M-CPD will select PAM-CPD, ensuring robustness against extreme deviations.

Noise sensitivity analysis

Noisy data introduces small and random variations that affect performance values, potentially disrupting PRL assignments.

• WGF-CPD: Noise artificially inflates or deflates small gaps, leading to erratic PRL segmentation. WGF-CPD has low noise robustness as it directly relies on precise gap values.

• K-CPD and PAM-CPD: Noise minimally affects clustering, as PRLs are assigned based on groups rather than absolute values. In other words, noise only impacts clustering when variance is high enough to shift assignments significantly. K-CPD and PAM-CPD tolerate high noises as clustering absorbs small variations. However, PAM-CPD offers an advantage over K-CPD because K-CPD assumes data points are clustered around centroids, making it less effective for irregular or non-spherical clusters, which are often treated as noise, reducing performance. In contrast, PAM-CPD handles such clusters better by minimizing the sum of dissimilarities rather than variance.

• M-CPD: If noise affects WGF-CPD significantly, M-CPD selects either K-CPD or PAM-CPD as better alternatives.

Finding and discussion

We summarize the findings along with discussions based on the CPD results as follows:

• Illustrated in Fig. 1, K-CPD tends to contribute more significantly to the output of M-CPD compared to WGF-CPD and PAM-CPD due to higher degree of Ω′ for EMP1 and EMP2. Nevertheless, STD1 and STD2 yield the higher degrees of Ω′ when using WGF-CPD and PAM-CPD, respectively, establishing all constituent methods as indispensable components of M-CPD to function optimally. Notice that Z-score method delivers zero Ω′ for all data sets due to its failure to meet Requirement 1.

• Figure 1 projects Ω′ values ranging widely among the proposed algorithms, depending on the characteristics of the sizes and distribution of performance-value divides, within each data set. The reasons for the highest and lowest degrees of conditional unbiasedness in every data set are further analyzed in Fig. 2.

• Across all evaluation data sets, the Ω₁ values of WGF-CPD, K-CPD, PAM-CPD, and M-CPD consistently reach their upper bound of 1.00 in Fig. 2 since the number of unassigned PRLs is sufficient to cover Θ with respect to each of the data sets. This completely achieves Requirement 1 and surpasses the performance of the conventional norm-referenced CPD method, Z score. Z-score method fails to unassign PVLs across all of the data sets resulting in zero Ω₁. However, if large gaps exist between performance values within a dataset, Z-score method will demonstrate its capability for conditional discretization. For example, in STD1, if the last three performance values adjacent to the last one were changed to 0.18, Z-score method would exclude DD from assignment.

• The plots of all data sets indicate that WGF-CPD method tends to outperform K-CPD and PAM-CPD methods in terms of Ω₂ because WGF-CPD method determines PVIs by prioritizing the widest gaps of performance values, thus aligning more closely with Requirement 2.

• According to all Ω₃ values presented in Fig. 2, K-CPD method tends to excel over WGF-CPD method in fulfilling Requirement 3 as K-CPD indirectly factors in PVIs via Euclidian-distance model parameter, whereas WGF-CPD is totally blind to PVIs. For the same reason, PAM-CPD method also outperformed WGF-CPD method for Ω₃ in EMP2 and STD2 data sets. Without outliers, K-CPD tends to maintain more comparable PVIs than PAM-CPD because K-CPD relies on centroids rather than medoids. Z-score method has the highest Ω₃ values across all data sets as it divides the range between maximum and minimum t scores into equal intervals, leading to comparable PVIs.