The Silhouette coefficient and the Davies-Bouldin index are more informative than Dunn index, Calinski-Harabasz index, Shannon entropy, and Gap statistic for unsupervised clustering internal evaluation of two convex clusters

Silhouette coefficient

The Silhouette metric is the more famous and widespread method for evaluating and validating clustering consistency. Introduced by Peter J. Rousseeuw in 1986 (Rousseeuw, 1987), its rationale consists in employing a natural and straightforward procedure to assess the relation between inter-cluster and intra-cluster mutual distances between points. As such, the Silhouette formula is a functional, defined for each point of the dataset, having also the chosen distance $d$ as an hyperparameter, with the Euclidean $L_2$ often chosen as the default in several implementations. However, different distances $d$ such as Manhattan $L_1$ will lead to different numerical outcomes for the final score. In detail, for a chosen distance function $d$ and for a dataset partitioned in $k$ clusters $\{C_t:t\in [1,k]\}$, given a data point $i$ belonging to a non-singleton cluster $C_I$, define

$$a(i)=\frac{1}{|C_I|-1}\sum _{j\in I\backslash \{i\}}d(i,j)$$as the intra-cluster dissimilarity measure, and

$$b(i)=\underset{J\ne I}{min}\frac{1}{|C_J|}\sum _{j\in C_J}d(i,j)$$as the intercluster dissimilarity measure and

(1) $$s(i)=\frac{b(i)-a(i)}{max\{a(i),b(i)\}}$$as the Silhouette score; for a single point cluster $C_I=\{i\}$, set $s(i)=0$.

The Silhouette score ranges from –1 (worst and minimal value) to +1 (best and maximum value) (Table 1).

Shannon entropy

Shannon entropy (Shannon, 1948; SciPy, 2025; Meilǎ, 2007), also known as information entropy and derived from information theory, is a measure that quantifies the uncertainty associated with a probability distribution. In clustering quality assessment, Shannon entropy is used to measure how well clusters separate data. A low value of entropy indicates that the data are well separated into clusters, suggesting that the clustering is of good quality. A high value of entropy, on the other hand, indicates that the data are more evenly distributed among the clusters, suggesting that the clustering may not be effective in separating the data into distinct groups. To calculate the Shannon entropy for a clustering result, the following steps are taken:

• 1.

  Assignment of the data to the clusters: Let us suppose we have $k$ clusters and $n$ data point. Every data point $i$ is assigned to the $C_j$ cluster by some method.

• 2.

  Probability distribution: We calculate the proportion of data in each cluster. The probability $p_j$ that a data point belongs to the $C_j$ cluster is given by: $$p_j=\frac{n_j}{n}$$where $n_j$ is the number of data points within the $C_j$ cluster and $n$ is the total number of data points.

• 3.

  Calculation of the Shannon entropy: We finally calculate the Shannon entropy H for this clustering by using this formula: (2) $$H=-\sum _{j=1}^k{p}_j\log (p_j).$$

Shannon entropy possible values range from 0 (minimal and perfect value) to ${\log }_{2}k$ (maximum and worst possible value) (Table 1). If we have $k=2$ clusters, Shannon entropy ranges from 0 to 1.

To have a scenario where values of the metric increases when results get better, we decided to use the complementary Shannon entropy, defined as $\mid 1-Shannonentropy|$. The complementary Shannon entropy with $k=2$ ranges from 0 to 1, where 0 is the worst possible outcome and 1 is the best possible outcome.

Davies-Bouldin index

The Davies-Bouldin index or ratio (Davies & Bouldin, 1979) represents the ratio of the sum of the within-cluster dispersion to the between-cluster separation. Compared with the Silhouette coefficient, the Davies-Bouldin index does not have a limited range of values; rather it can take any value greater than or equal to 0. The closer the value is to 0 the higher the clustering quality, consequently the larger the value the lower the clustering quality.

The formula is the following:

(3) $$DBI=\frac{1}{k}\sum _{i=1}^k{R}_i$$where:

$$R_i=\underset{j\ne i}{max}{R}_{ij}$$and

$$R_{ij}=\frac{S_i+S_j}{M_{ij}}$$with:

$$S_i=\frac{1}{|C_i|}{\sum }_{x\in C_i}\Vert x-{\mu }_i\Vert M_{ij}=|{\mu }_j-{\mu }_i|$$where $x$ is a data point of the $C_i$ cluster, ${\mu }_i$ is the centroid of the $C_i$ cluster, and $|C_i|$ is the number of data points in the $C_i$ cluster. Thus $S_i$ represents the dispersion of the $C_i$ cluster. $M_{ij}$ is the distance between the two centroids.

The DBI ranges from 0 (minimal and perfect outcome) to infinity (maximum and worst outcome possible) (Table 1). Since all the other metrics have low values for bad results and high values for good results, we decided to use reciprocal DBI = 1/DBI. The interval of reciprocal DBI is (0; $\mathrm{\infty }$), the higher the value the better the result.

Dunn index

The Dunn index (DI) (Dunn, 1974) is an index that measures the degree of compactness of clusters and the degree of separation between clusters. Dunn index does not have a limited range of values, but rather can take any value greater than or equal to 0. The more the value tends to infinity the higher the cluster quality will be; and consequently the smaller the value the lower the clustering quality will be. The formula for the Dunn index is as follows:

$$DI=\frac{{min}_{1\le i<j\le k}\delta (C_i,C_j)}{{max}_{1\le i\le k}\mathrm{\Delta }(C_i)}$$where:

• $\delta (C_i,C_j)$ is the distance between the $C_i$ cluster and the $C_j$ cluster. This distance can be defined in several ways, for example as the minimal distance between two data points belonging to two different clusters (like we do in this study).

• $\mathrm{\Delta }(C_i)$ is the intra-cluster distance for the $C_i$ cluster, that is the maximum distance between two points within the $C_i$ cluster.

• $k$ is the number of clusters.

The Dunn index ranges from 0 (minimal and worst possible outcome) to infinity (maximum and best possible outcome) (Table 1).

Calinski-Harabasz index

The Calinski-Harabasz index (CH) (Calinski & Harabasz, 1974), also known as Variance Ratio Criterion, is defined as the ratio of between-cluster separation (BCSS) to within-cluster dispersion (WCSS), normalized by the number of degrees of freedom. The Calinski-Harabasz index does not have a limited range of values, but rather can take on any value greater than or equal to 0. The more the value tends to infinity the higher the cluster quality, consequently the smaller the value the lower the clustering quality. The formula for the Calinski-Harabasz index is as follows:

$$CH=\frac{BCSS/(k-1)}{WCSS/(n-k)}$$where:

• BCSS (between-cluster dispersion matrix) is the distance between the centroid of each cluster $c_i$ and the general centroid $c$. $$BCSS=\sum _{i=1}^k{n}_i\cdot \Vert c_i-c{\Vert }^2$$

• WCSS (within-cluster dispersion matrix) is the Euclidean distance between the $x$ data points and the centroid of their cluster $c_i$. $$WCSS=\sum _{i=1}^k\sum _{x\in C_i}\Vert x-c_i{\Vert }^2$$

• $k$ is the number of clusters.

• $n$ is the total number of data points.

The Calinski-Harabasz index ranges from 0 (minimal and worst possible outcome) to infinity (maximum and best possible outcome) (Table 1).

Gap statistic

The Gap statistic (Tibshirani, Walther & Hastie, 2001) is a metric commonly used to determine the optimal number of clusters in a dataset, but it can also be interpreted to assess the quality of clustering too, as the other scores listed above. The Gap statistic helps to understand how much better the intra-cluster dispersion of the analyzed data is than that expected from random clustering. A high value of the Gap statistic suggests that the clustering is of good quality, while a low value suggests clustering of very poor quality. To calculate the Gap statistic, the following steps are taken:

• 1.

  Calculate the intra-cluster dispersion $W_k$ for the clustering result obtained. This intra-cluster dispersion for $k$ clusters is defined as: $$W_k=\sum _{r=1}^k\frac{1}{2n_r}\sum _{i,j\in C_r}{d}_{i,j}$$where:

  • $C_r$ represents the $r$ cluster

  • $n_r$ is the number of points within the $r$ cluster

  • $d_{i,j}$ is the distance between the $i$ and $j$ data points.

• 2.

  Generate the B number of reference casual datasets with the same uniform distribution and the same dimension of the original. For each of these datasets, apply the same clustering algorithm and calculate the intra-cluster dispersion $W_k^{(b)}$.

• 3.

  Calculate the Gap statistic as the difference between the logarithm of the average intra-cluster dispersion of the reference datasets and the logarithm of the intra-cluster dispersion of the real analyzed dataset: $${\mathrm{G}\mathrm{a}\mathrm{p}}_n(k)=\frac{1}{B}\sum _{b=1}^B\log (W_k^{(b)})-\log (W_k)$$where $\frac{1}{B}{\sum }_{b=1}^B\log (W_k^{(b)})$ is the mean of the logarithms of the intra-cluster dispersions of the reference datasets.

The Gap statistic ranges from $-\mathrm{\infty }$ (minimal and worst possible outcome) to $+\mathrm{\infty }$ (maximum and best possible outcome) (Table 1), where a negative value means bad result and a positive value means good result.

Use cases on artificial data

In this section, we applied $k$-means with $k=2$ clusters and with Euclidean distance to different configurations of artificial data where data partitioning was clearly worsening or improving. We then calculated the six metrics for each of these configurations, and verified if the values of these coefficients were actually worsening or improving. If the partitioning was worsening and a metric’s value was worsening too, or if the partitioning was improving and a metric’s value was improving too, we considered the metric consistent. Otherwise, we considered it inconsistent.

Matrices of zeros and ones

We designed a controlled experiment where we observed the behavior of the studied metrics on an artificial dataset containing only zeros and ones at the beginning and containing only real values in the end.

In detail, we created a matrix of five columns and ten rows where all the first five rows contained zeros and all the last five rows contained ones (Fig. 1, first matrix on the left). We applied $k$-means with $k=2$ clusters to this matrix and calculated the results of the analyzed metrics.

We then randomly selected a row and replaced it with real values in the $[0;1]$ interval, reapplied k-means, and calculated the analyzed metrics again. Afterwards, we randomly selected two, three, four rows and repeated the same procedures until we reached the total number of ten rows (Fig. 1). We saved the results of each metric at each step: since the matrix uniformity was clearly worsening at each iteration, we expected to see a worsening in the values of the metrics, too.

We analyzed the trends of the six analyzed metrics applied to the $k$-means results obtained on the ten described matrices, and we noticed their trends (Fig. 2). The Silhouette coefficient, Davies-Bouldin index, Dunn index, and Calinski-Harabasz index showed a consistent (worsening) trend. Shannon entropy and Gap statistic, instead, showed an inconsistent trend.

Artificial groups of points

We then utilized a series of artificial datasets inspired by the documentation of the scikit-learn Python package (scikit-learn, 2025a). We selected the cases where two distinct clusters of points were initially recognizable, and then we increasingly manipulated the initial data until no clear cluster was visible. We employed 5.000 points for each artificial dataset, except for the $\overline{\text{W}}$ one where that number was insufficient to show relevant trends, and where we employed 10.000 points instead. At each step, we applied the $k$-means clustering method by using $k=2$ clusters, and calculated all the six analyzed metrics on its results. This change of positions of the points clearly indicates a worsening of the separation of the two clusters, and therefore we studied the six metrics to understand if they would confirm or not this worsening. We applied the changes of points by using different standard deviations for the centers of the clusters of blobs of points (cluster_std parameter in the make_blob() function of the scikit-learn Python library (scikit-learn, 2025b)).

Since we focus our study on convex-shaped clusters, we avoided the nested, concave sets of points. We eventually observed the plotted trends of the values of the six metrics studied, and considered regular the trends that result constantly decreasing, constantly stable, or constantly increasing. We considered irregular all the trends that had locally divergent points (that are, local minima or local maxima).

Clouds dataset: The first case included two clouds of points, initially clearly separated, that end up into a common cloud (Fig. 3).

We applied $k$-means and calculated the six metrics on ten cases (more than what shown in Fig. 3), and depicted their trends in Fig. 4.

Silhouette, Davies-Bouldin index, Dunn index, Calinski-Harabasz index, and Gap statistic showed consistent trends, indicating the worsening of the clustering. In contrast, the Shannon entropy did not.

Semicircles dataset: We repeated the same controlled experiment described earlier on an artificial dataset made of two semicircles. These two semicircles are first merged together and then start to diverge, by creating two distinct clusters eventually (Fig. 5). We applied $k$-means with $k=2$ clusters to these datasets, by changing standard deviations of the distributions of the point generation, and computed the six analyzed metrics. In the end, we checked which of these metrics would confirm the clustering trend, that is the improvement on distinguishing the two clusters.

We depicted the trends of the six analyzed coefficients in six cases in Fig. 6.

On these semicircles points, the metrics that consistently confirmed the improvement in the cluster separations were Silhouette, Davies-Bouldin index, Dunn index, and Gap statistic. On the other hand, Shannon entropy and Calinski-Harabasz index trends did not show any improvement.

Ball dataset: We repeated the same experiment on a dataset made of two balls, initially clearly separated, and eventually all mixed up. The ball on the right, in fact, increases its size until it joins the other ball, indicating a clear worsening regarding cluster separation (Fig. 7).

We calculated the analyzed six metrics on seven distinct cases, and represented their trends in Fig. 8.

To summarize, the Silhouette coefficient, Calinski-Harabasz index, Davies-Bouldin index, and Dunn index confirmed the worsening trend of the ball clusters. Shannon entropy and Gap statistic, instead, did not.

Brush strokes dataset: We then selected two sets of points that look like brush strokes: initially, they are clearly separated, but in the next plots they both move close to each other, until they end up into a single shape (Fig. 9), by changing the standard deviation. Again, we applied $k$-means with $k=2$ and calculated the six analyzed internal clustering metrics at each step.

We depicted the trends of the six analyzed coefficient at seven different steps in Fig. 10.

The trends of five considered metrics (Silhouette coefficient, Calinski-Harabasz index, Davies-Bouldin index, Gap statistic, and Dunn index) resulted being consistent with the worsening, while Shannon entropy’s trend resulted being inconsistent with the cluster separation worsening.

$\overline{\mathrm{W}}$ Dataset: The two clusters here initially are a straight horizontal line on the top and W-shaped set of points on the bottom. We randomly changed the distributions of these groups of points in the following steps, so that no shape would be recognizable in the end (Fig. 11). Again, we executed $k$-means clustering by using $k=2$, and calculated the six analyzed internal metrics.

Figure 12 represents the trends of the six analyzed measures at each step.

Regarding the worsening trend of this $\overline{\mathrm{W}}$ case, the Silhouette coefficient, Davies-Bouldin index, Shannon entropy, and Dunn index confirmed this outcome. The trends of Calinski-Harabasz index and Gap statistic, instead, did not show any worsening.

Artificial dataset test recap: After performing the just-described tests on the five artificial datasets, we summarized the results in Table 2.

Table 2:

Ranking of the analyzed metrics on the artificial datasets based on the consistency of the results.

Ranking	Metric	Consistent	Trends
1	Silhouette coefficient	5 out of 5	100%
1	Davies-Bouldin index	5 out of 5	100%
1	Dunn index	5 out of 5	100%
4	Calinski-Harabasz index	3 out of 5	60%
4	Gap statistic	3 out of 5	60%
6	Shannon entropy	1 out of 5	20%

DOI: 10.7717/peerj-cs.3309/table-2

As one can notice, the Silhouette coefficient, Davies-Bouldin index, and Dunn index produced consistent outcomes with the trends we expected to see in all the five scenarios (that means, worsening values while the cluster separation was worsening, and improving values while the cluster separation was improving). Calinski-Harabasz and Gap statistic resulted being consistent in three cases out of five, and Shannon entropy only once (Table 2). However, regarding results on bad clusters, when the clouds of points are indistinguishable (right end of the previous plots (Figs. 4, 6, 8, 10, and 12), we can see that Davies-Bouldin index always stays around 0 (as expected), while the Silhouette coefficient lays around +0.5. In these cases, we can state that Silhouette shows an inconsistent outcome with respect to the clustering results: in the right end of these plots, we would expect Silhouette to get a value around 0 or $-1$, which never happens. This behavior is a flaw of the Silhouette coefficient that needs to be considered.

To recap, these results indicate a higher reliability and effectiveness of the Silhouette coefficient, Davies-Bouldin index, and Dunn index compared with the other analyzed metrics for internal clustering evaluation, with the Silhouette score having the just-mentioned flaw for bad clustering results.

Real-world medical scenarios

To test the effectiveness and the reliability of the six metrics considered in this study, we developed an approach based on clustering applied to data of real-world electronic health records (EHRs) of patients with different diseases (Boonstra, Versluis & Vos, 2014).

Electronic health records (EHRs) are digital information reports designed to collect, store, and provide healthcare data in computer-readable format (Kim et al., 2019). The information contained in EHRs generally includes essential demographic details like date of birth and sex at birth, along with data on diagnoses, treatments, clinical investigation results, and outcomes such as hospital discharge or death. EHRs collect data at the individual level, and by using unique patient identifiers, it becomes possible to track various aspects of care and outcomes over time (Cowie et al., 2017). The real-world data derived from EHRs can be valuable for researchers focused on understanding health and disease, identifying disease trajectories, and assessing disease risks and treatments through modern statistical and machine learning methods (Aminoleslami, Anderson & Chicco, 2024).

These data were collected for clinical reasons, and not for scientific reasons, and therefore have a high level of complexity, especially for clustering purposes it is even possible that no meaningful partition of patients among these data exists. For these reasons, we decided to employ these datasets for our study: if any cluster is found, we can be confident that no “shortcut” or pre-arranged setting generated them. If a cluster is found, it is because of the clustering algorithm used.

Euclidean similarity clustering. Among the measures for external clustering validity, there is consensus about the usage of the ARI (Hubert & Arabie, 1985), that necessitates of an external ground truth. Since we are studying convex-shaped clusters and are focusing this study on $k$-means with Euclidean distance, we developed a new clustering approach based on the Euclidean similarity between couples of rows. We set k = 2 clusters and apply this approach to each single dataset: we assign the first row to the cluster A, and then we assign all the other rows of the dataset having Euclidean similarity to the initial row to the same cluster A. We measure the similarity by computing the Euclidean distance between two rows, and using a particular threshold. If we find a row that is not enough similar to the first row (and therefore its similarity is above the threshold), we assign it to the cluster B. Then we cycle on the remaining rows to see if any of them is similar to the row assigned to cluster B, and, if found, we put it in cluster B. We called this approach Euclidean similarity clustering (ESC) and we describe it more in detail in Algorithm 1 and in Fig. 13.

The rationale behind this algorithm is as follows. If the Euclidean distance between two data rows is lower than a specific relative threshold, we consider both data rows to be part of the same cluster. Conversely, if the Euclidean distance between two data rows is high, we do not consider them to belong to the same cluster. Since we based our entire study on two clusters, we also utilize only two clusters for the ESC. Given that we employ $k$-means clustering based on Euclidean distance, we regard the clusters identified by this ESC method as a form of “ground truth” for assessing the results of our clustering coefficients.

After applying the ESC algorithm, we execute $k$-means with $k=2$ on all the rows found by ESC in its two clusters, and we measure its results through the six metrics for the internal clustering validation. We use the results of ESC as gold standard for the results of $k$-means, and we compute the adjusted Rand index between them (Algorithm 2 and Fig. 13).

Coefficients’ trends at varying thresholds: We performed two tests for each dataset: one by using the 20% similarity threshold, and then another one by employing the 33% threshold instead: the former is stricter and selects only data rows similar between each other, and the latter is a bit more permissive. Afterwards, we checked the values of the adjusted Rand index and the values of the six metrics for the scenarios of both similarity thresholds. We observed six possible scenarios for each metric (Silhouette coefficient, Davies-Bouldin index, Calinski-Harabasz index, Dunn index, Shannon entropy and Gap statistic) when moving from the 20% case to the 33% case:

• If the values of the adjusted Rand indices were similar (=), and the values of the analyzed metric were similar (=), we considered this trend consistent;

• If the values of the adjusted Rand indices were similar (=), but the values of the analyzed metric were increasing ( $\nearrow$) or decreasing ( $\searrow$), we considered this trend inconsistent;

• If the values of the adjusted Rand indices were decreasing ( $\searrow$), and the values of the analyzed metric were decreasing ( $\searrow$), we considered this trend consistent;

• If the values of the adjusted Rand indices were increasing ( $\nearrow$), and the values of the analyzed metric were increasing ( $\nearrow$), we considered this trend consistent;

• If the values of the adjusted Rand indices were increasing ( $\nearrow$), but the values of the analyzed metric were similar (=) or decreasing ( $\searrow$), we considered this trend inconsistent;

• If the values of the adjusted Rand indices were decreasing ( $\searrow$), but the values of the analyzed metric were similar (=) or increasing ( $\nearrow$), we considered this trend inconsistent.

We consider two values of the same clustering metric similar if their absolute difference is lower than 20%. In the next tables, we call this difference quantitative consistency. That means, if the quantitative consistency of a specific metric value is lower than $+20\mathrm{\%}$ or lower than $-20\mathrm{\%}$, we assign the stable sign tho that trend (=).

We described our approach more in detail in Fig. 13.

We explain how our algorithm works on a toy synthetic EHRs dataset that we represented fully in Table 3. In this toy dataset, each row represents the profile of single patient, and each column contains a clinical or demographic feature. The features indicate the age of the patients in years, the sex, the level of education and the stage of cancer, plus a column indicating the ID of the patient. As one can notice immediately, these invented data points can be easily clustered in two partitions by our ESC method: the first five rows belong to the cluster A, and the last five rows belong to cluster B.

After using the Euclidean similarity clustering method, we applied $k$-means to its clusters, and compared the results of both algorithms by using the ARI. By using both the 20% and the 33% similarity thresholds, we notice that adjusted Rand indices are always +1 in the $[-1;+1]$ interval, which indicates perfect match (Table 4).

The clustering internal metrics computed on the results of $k$-means applied to the elements grouped in the two clusters by the Euclidean similarity clustering (ESC) (Table 5) are identical for the 20% and 33% threshold cases (=) and confirm the trend of the ARI computed earlier (Table 4): in this case, we can consider the trends of all the six metrics consistent with the trends of ARI. All the six metrics passed this sanity check.

Sepsis and SIRS EHRs dataset: We executed our just-described algorithm on a dataset of real-world electronic medical records of Turkish patients who had sepsis or systemic inflammatory response syndrome (SIRS) (Gucyetmez & Atalan, 2016). S This dataset table contains 1,257 patient profiles (rows) and 16 clinical variables (columns), without missing data. We applied our proposed our ESC approach and $k$-means first with a 20% similarity threshold and then with a 33% threshold, and we noticed that the ARI value moved from 1 to 0.318 (Table 6).

We then calculated the six analyzed metrics on the results of $k$-means and checked their trends when changing similarity thresholds (Table 7). The Silhouette coefficient, Davies-Bouldin index, Dunn index, and Gap statistic confirmed the decreasing trend, while Shannon entropy and Calinski-Harabasz index resulted being inconsistent with the ARI trend.

Depression and heart failure EHRs dataset

We executed again algorithm on a dataset of real-world electronic medical records of patients diagnosed with depression and heart failure (Jani et al., 2016), collected in Minnesota, USA.

This dataset table contains 425 patient profiles (rows) and 15 clinical factors (columns), without missing data. We applied our proposed our ESC approach and $k$-means first with a 20% similarity threshold and then with a 33% threshold, and we noticed that the ARI value decreased from 1 to 0.179 (Table 8).

We then calculated the six analyzed metrics on the results of $k$-means and checked their trends when changing similarity thresholds (Table 9). The Silhouette coefficient, Davies-Bouldin index, Shannon entropy, and Gap statistic confirmed the decreasing trend, while Calinski-Harabasz index and Dunn index resulted being inconsistent with the ARI trend.

Cardiac arrest EHRs dataset: We executed again algorithm on a dataset of real-world electronic medical records of patients who had an out-of-hospital cardiac arrest (Requena-Morales et al., 2017), collected in Spain.

This dataset table contains 422 patient profiles (rows) and 10 clinical factors (columns), without missing data. We applied our proposed our ESC approach and $k$-means first with a 20% similarity threshold and then with a 33% threshold, and we noticed that the ARI value decreased from 1 to 0.47 (Table 10).

We then calculated the six analyzed metrics on the results of $k$-means and checked their trends when changing similarity thresholds (Table 11). All the six metrics confirmed the decreasing trend of ARI.

Neuroblastoma EHRs dataset: We executed again algorithm on a dataset of real-world electronic medical records of children suffering from neuroblastoma (Ma et al., 2018), collected in eastern China.

This dataset table contains 169 patient profiles (rows) and 13 clinical factors (columns), without missing data. We applied our proposed our ESC approach and $k$-means first with a 20% similarity threshold and then with a 33% threshold, and we noticed that the ARI value decreased from 1 to 0.182 (Table 12).

We then calculated the six analyzed metrics on the results of $k$-means and checked their trends when changing similarity thresholds (Table 13). Five scores confirmed the decreasing trend of ARI, all except Gap statistic.

Diabetes type one EHRs dataset: We executed again algorithm on a dataset of real-world electronic medical records of children and adolescents suffering from diabetes type one (Takashi et al., 2019), collected in Japan.

This dataset table contains 68 patient profiles (rows) and 20 clinical features (columns), without missing data. We applied our proposed our ESC approach and $k$-means first with a 20% similarity threshold and then with a 33% threshold, and we noticed that the ARI value remained consistent, resulting +1 in both cases (Table 14).

We then calculated the six analyzed metrics on the results of $k$-means and checked their trends when changing similarity thresholds (Table 15). Among the analyzed metrics, only the Silhouette coefficient, and Davies-Bouldin index stayed stable, confirming the trend of the ARI. Calinski-Harabasz index, Shannon entropy, Dunn index, and Gap statistic, on the contrary, showed increasing or decreasing trends, resulting inconsistent with the trend of ARI.

Recap of the results on the real-world EHRs datasets: We summarized the results obtained on the five real-world open datasets of electronic medical records in Table 16.

As one can notice, the Silhouette coefficient and Davies-Bouldin index are the only two metrics which resulted being consistent with the ARI trends in all the five experiments (Table 16). All the other four coefficients had an inconsistent behavior and failed to confirm the ARI trend in three datasets (Calinski-Harabasz index) or two datasets (Dunn index, Gap statistic and Shannon entropy). These results prove a higher reliability of the Silhouette coefficient and of the Davies-Bouldin index.

These results differ from those obtained on the artificial datasets (Table 2) due to the presence of outliers in the data derived from electronic medical records. This type of data is known to contain several exceptions, because of the varying nature of the subpopulations of patients considered (Chicco & Coelho, 2025). The Silhouette coefficient and the Davies-Bouldin index appear to handle this aspect well, while the other four metrics seem to be adversely affected by the presence of outliers.

The Dunn index, in particular, moved from correctly identifying the right trends on all artificial datasets (Table 2) to correctly detecting only three right trends out of five on the EHRs datasets (Table 16). We believe this result is due to the lack of the Dunn index to handle the outliers.

The studied metrics in the scientific literature

To further explore the usage of the six analyzed metrics, we studied the scientific literature and quantified the number of scientific articles mentioning them. We looked for the articles mentioning the six metrics by performing a search on Google Scholar through an Apple Safari web browser in Milan (Italy, EU) on 14th July 2024 and by employing the following search terms:

• “Silhouette coefficient” OR “Silhouette index” OR “Silhouette score”

• “Davies-Bouldin coefficient” OR “Davies-Bouldin index” OR “Davies-Bouldin score”

• “Dunn coefficient” OR “Dunn index” OR “Dunn score”

• “Calinski-Harabasz coefficient” OR “Calinski-Harabasz index” OR “Calinski-Harabasz score”

• “Entropy clustering”

• “Gap Statistics score” OR “Gap Statistics index” OR “Gap Statistics coefficient” OR “Gap Statistic score” OR “Gap Statistic index” OR “Gap Statistic coefficient”

The results show that the Silhouette coefficient and Davies-Bouldin index are the most commonly employed metrics, since they have been mentioned by around 41 and 24 articles per month since their publication in 1987 and 1979, respectively (Table 17 and Fig. 14). The indexes of Dunn and Calinski-Harabasz, instead, collected approximately 11 and 10 citations per month since the time they were introduced. It was difficult to detect the usage of Shannon entropy for clustering, because the word “entropy” have several meanings in computer science literature. That being said, we noticed a small habit to employ it: only approximately three articles per month utilized it actually. The Gap statistic is positioned at the last step of this ranking, having around 0.25 articles per month citing it since 2001.

Tests on artificial datasets with a “correct” number of clusters

To further assess the efficiency of the six clustering scores analyzed in this study, we decided to apply them to the results obtained by $k$-means on five public artificial datasets where the number of “correct” convex clusters is known (Thrun & Ultsch, 2020, 2023). We report the results measured by the Silhouette coefficient, Calinski-Harabasz index, reverse Davies-Bouldin index, reverse Shannon entropy, Dunn index, and Gap statistic when the number of clusters $k$ varies from 2 to 10. We used the reverse formula of the Davies-Bouldin index and of the Shannon entropy to make all the six coefficients increase when the quality of the clustering result increases: the higher, the better for all the six metrics.

We report the results in Figs. 15, 16, 17, 18, and 19.

We then checked which metrics had their highest value when corresponding to the “correct” number of clusters, and we recap the results obtained in Table 18. As one can notice, no metric detected the “right” number of clusters in all the five datasets. The Silhouette coefficient, Davies-Bouldin index, Dunn index, and Calinski-Harabasz found the “correct” number of clusters for $k$-means on the majority of cases: three out of five. These tests confirm the higher effectiveness of these three metrics compared to the other ones considered here.