All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
Dear authors,
Thank you for addressing my concerns.
[# PeerJ Staff Note - this decision was reviewed and approved by Massimiliano Fasi, a PeerJ Section Editor covering this Section #]
Dear Authors,
First I would like to thank all the reviewers involved in this contribution.
As the last step and to ensure robustness of your approach, I kindly ask you to clarify my concerns.
If I understood you well, you defined MPV using density peaks and minimum distances and tested on datasets where density-based methods are naturally strong (rings, semi-rings). This may result in bias in favor of MPV without showing robustness in settings unfavorable to density measures. Please discuss it.
Also, I am confused about two claims.
“Many algorithms use the average value of all data objects in a cluster as the center point to measure the degree of separation between clusters. While the average value can effectively represent a cluster in spherical datasets, it performs poorly for ring, semi-ring, and flow-shaped datasets. Because the object obtained by averaging is not within the cluster and is not a real data point, using it to measure inter-cluster separation can lead to inaccurate or erroneous results.”
The previous one seems in contrast with:
Finding the minimum value among the data points here can better represent the compactness between points. Since the number of data points in each cluster is different, we cannot determine the compactness of a cluster solely based on the sum of distances. Therefore, calculating the average value can better demonstrate the compactness of the cluster”.
Please clarify.
**PeerJ Staff Note:** Please ensure that all review, editorial, and staff comments are addressed in a response letter and that any edits or clarifications mentioned in the letter are also inserted into the revised manuscript where appropriate.
-
-
-
I have to mention that the reviewers are more satisfied with the current state of the manuscript. I have found that the progress achieved by the authors produces serious hopes that the manuscript may be brought to the status of "ready for publication". Think carefully whether your decision will be "to explicitly acknowledge the limitations in the manuscript" or "comparative evaluations with experiments using k-means".
no comment
no comment
no comment
1. I strongly recommend that the authors revise the manuscript title to make it more concise, as the current title is overly long. For example, a suitable alternative might be: "A density-peak-based validity index for determining the number of clusters"
2. From the standpoint of a validity index’s generality, I note that all clustering results in this work are generated using density-peak clustering, without evaluating more widely used algorithms such as k-means. For example, the study “An Internal Validity Index Based on Density-Involved Distance” (2019) employs k-means–style experiments when comparing its proposed index; since your manuscript also benchmarks against that paper’s metric yet reports substantially different results, it seems likely that the choice of clustering algorithm underlies this discrepancy. Additionally, because your index is inherently based on density peaks, demonstrating its performance only on isomorphic clustering methods may not be fully convincing. You should therefore include experiments with other clustering algorithms to strengthen the persuasiveness of your approach. I therefore recommend that the authors either explicitly acknowledge this limitation in the manuscript or supplement their evaluation with experiments using k-means (or similarly ubiquitous clustering algorithms) to demonstrate broader generality.
Both reviewers are not satisfied with the introduced improvements while Reviewer 2 demands heavy editing and Reviewer 1 expects more serious changes. The authors must apply great efforts to improve the manuscript to satisfy both requirements. I expect to see more serious exploration of higher-dimensional data and better English in the revised version.
**Language Note:** The Academic Editor has identified that the English language must be improved. PeerJ can provide language editing services - please contact us at [email protected] for pricing (be sure to provide your manuscript number and title). Alternatively, you should make your own arrangements to improve the language quality and provide details in your response letter. – PeerJ Staff
Although the authors have made major revisions, none of the key concerns raised previously have been adequately addressed. The manuscript still lacks the fundamental improvements necessary to meet the journal’s standards.
The authors predominantly rely on two-dimensional datasets, with limited exploration of higher-dimensional data. This introduces a significant selection bias, and raises doubts regarding the generalizability and applicability of the proposed method to real-world high-dimensional scenarios. Furthermore, the manuscript lacks ablation studies to convincingly demonstrate that the combination of the two components yields a tangible improvement over using a single component. The comparative experiments with state-of-the-art methods are insufficient, and the absence of evaluations on higher-dimensional datasets further undermines the credibility of the experimental validation.
The proposed method exhibits limited originality and does not offer substantial contributions beyond existing approaches. Overall, the novelty and effectiveness of the method remain unclear.
no comment
The writing quality still suffers from grammar and other issues. It requires professional proof-reading. Below are few examples.
- The paper proposes INDEX, but at several places text refers to it as ALGORITHM ("practicality of this algorithm")
- K-means is claimed to have "fast and simple computation". The algorithm is simple, but "simple computation" is not sensible.
- Several algorithms are denoted by acronym (AHD, HCLORE, DLORE-DP, and many others) without writing their full name, or giving a citation.
- External indices are claimed not to be accessible in line 50, but later in Section 2 it is stated that "In Rezaei proposed using external indices to measure the stability...". Maybe the authors missed that this stability-based approach is exactly designed for solving ("counting") the number of clusters! So external indeces ARE applicable (not accessible). Their use is just more complex and Rezaei's paper this is analyzed why so, and how to use.
- "Currently" when referring existing measures.
- "and so on". What information it carries? Why not just list all that are compared later.
- "However, these indices perform well"... Why "However" here?
- Spaces are lacking in many places: "... proven.However...". These should be trivial to detect by proper proofing tool. The manuscript should not have such issues.
- Section 2: "Count cluster number". It is not "counted". It is detected or determined, but not "counted".
- What means "particularly ideal"?
- All related work are listed in awfully long paragraph starting in line 97!
- "research algorithms have broadened my thinking for my research": Ok, good to know, but does this personal experience have something relevant to the paper? I think just remove this sentence.
- There are numbers and "(" marks in italic font in line 243, 247 and elsewhere. Please fix.
Seems ok.
I suggest to count how many times each index found the correct number of clusters and report. Would give at least some easy way to compare the results.
I propose to consider and address all suggestions of the reviewers including requirements to extend tested databases.
[# PeerJ Staff Note: It is PeerJ policy that additional references suggested during the peer-review process should *only* be included if the authors are in agreement that they are relevant and useful #]
General Comments: The manuscript presents an interesting approach to clustering, but there are several areas that could benefit from further clarification and enhancement. Below, I provide specific comments and suggestions that may help improve the overall quality of the paper.
Image Quality: In several instances, the figures are unclear and of low resolution. It is recommended that you use higher-quality images or consider replacing them with vector graphics. This will help readers to better understand the visual representations and ensure that all details are clearly visible.
Experimental Evaluation: The experiments in the manuscript are valuable, but I would recommend expanding the experimental setup to include more complex or high-dimensional datasets. Testing on a wider range of datasets will help strengthen the generalizability and robustness of your approach, and provide a more comprehensive evaluation of its performance.
Comparison with State-of-the-Art Methods: The paper could benefit from a more detailed comparison with current state-of-the-art (SOTA) methods in clustering validation. I suggest adding a comparison with a well-regarded cluster validity index such as the one proposed by Hu and Zhong (2019) in "An Internal Validity Index Based on Density-Involved Distance" (IEEE Access). This comparison should include both the datasets and methods used in their work to provide a more direct experimental comparison.
Novelty of the Method: While your approach is valuable, the innovative aspects of your methodology are not entirely clear. Although it is not mandatory for every method to introduce groundbreaking innovation, I noticed that you employ both distance and density information in your formulation. To highlight the contribution of these two factors more effectively, I suggest conducting an ablation study to demonstrate the individual importance of each term in your approach. This will provide clearer insight into how these components contribute to the overall performance of your method.
Discussion on Internal Validity Indices: It would be valuable to discuss the relevant literature on internal validity indices in clustering. I recommend referencing the recent review by Hassan et al. (2024) titled "From A-to-Z review of clustering validation indices" (Neurocomputing), which provides a comprehensive overview of the topic. A discussion of the similarities and differences between your method and those covered in the review could further contextualize your contribution and provide a deeper understanding of where your approach stands in relation to existing methods.
Clear but require professional proof-reading. Flaws in text and equations. Figures can be improved.
Lacks core benchmarking with typical clustering datasets.
Over-optimistic results via easy and selective use of datasets.
The paper presents cluster validity index tailored for density peaks. It is constructed as a ratio of inter cluster separation and intra cluster compactness, which is one of the most common approaches to construct such index. The overall design is valid, but the exact details, and also the presentation of the paper, require major revisions.
1.
Constructing index as inter cluster separation and intra cluster compactness is well documented in [1], including adopting the measure also to text data. The proposed index is an extension of this general approach to density peaks and should be put into this context accordingly.
2.
The exact formulation of the measure, however, is seemingly flawed. The separation is product of distance to nearest centroid and distance between nearest points of the same clusters. Highly heuristic but valid construction. Compactness, however, has some design flaws but they are easy to fix.
a) First, it is highly unclear what is calculated as both summation and min operation are used. The current formulation is mathematically meaningless. It seems to sum all pairwise distances within the cluster divided by the cluster size but the purpose of min operation is confusing. If it really sums up all distances, this actually equals to sum of distances to centroid, i.e. the normal way to calculate cluster compactness; see [2].
However, if the minimum of something is taken, revise the formula and explain it in text.
b) Second, only the maximum value of the least compact cluster is taken. Again, this is highly heuristic as it would use merely the biggest cluster at the end. Fine, not very convincing, but still possible construction. Just mention this clearly.
b) Third, the sd value is not directly used but mixed with the cd value in Equation (7). This is actually quite stupid as sd is supposed to measure compactness, which has nothing to do with intercluster separation. Luckily this design flaw will be eliminated by itself, and the design remains valid. However, Equation (7) should be revised as follows:
sd/com = sd / min(sd,cd)
IF sd < cd: sd / sd = 1
ELSE sd > cd: sd / cd
This can be rewritten much simpler and more meaningful way as:
MPV = min( sd/cd, 1)
In other words, the strange formulation merely upper bounds the value by 1, and operates merely on case when sd > cd. Not sure if this is even needed, but at least define the equation as explained above.
3.
Organization of Section 3 is poor. Materials and Methods? Come on, you can do better than that. Do not use such meaningless generic titles. First, define Density peaks and all related equations in its own section (3. Density peaks), and then the proposed measure in its own (4. MPV), followed by analysis (5. Analysis). No need to have third level headings like 3.2.1.
Have sub-sections for 4.1 Separation between clusters, and 4.2 Compactness within cluster, instead of using Definitions 1, 2, 3, 4. Define the overall index (MPV=sd/cd) in the beginning of this section already (not at the end), and only then describe the two parts (top-down approach).
Remove the monster equation (9). It is useless. Modular definition as described above is reader friendlier.
4.
Why is it called by MPV anyway? Maybe give the full name also in case it is acronym.
5.
Is Algorithm 1 really needed? The only meaningful information I found here is the choice using kmax=SQRT(n). At least, do not write full sentences in Algorithm. It is already clear that Density Peaks is used as the algorithm, although I do not see any reason why limit the measure to one specific algorithm only.
6.
The datasets consist of two types of data: (1) artificial rings and other well-separated clusters, (2) classification data. Neither is typical for clustering. The first one is easy to cluster with proper measure, whereas classification data may have different number of classes than there are clusters (Iris has 3 classes but 2 clusters). In addition, the measure should be tested with normal datasets as well, like the following 11 sets from the clustering basic benchmark [3]: S1, S2, S3, S4, A1, A2, A3, Birch1, Birch2, Unbalance, DIM32. I expect it might fail with some of them as it is heavily heuristic. Anyway, it should be tested, and the reader is entitled to know the truth.
7.
Figures require improvements:
Figure 1 is essentially demonstrating sd measure. Legend could be simplified, no need to explain all variables, just once (myy, cend, cld, e). However, the lines would be better to have different styles like solid for intercluster and dashed for intracluster (also width and thickness can be varied) to make it easier to demonstrate the key points.
Consider making similar picture should be also made for the cd measure.
Figure 2 is also ok, but no need for Figure 3. The idea is already clear without it.
Figure 8: use “clusters” for x-axis (simpler) and have enough big font size that reader can see it. Current makes reader guessing.
8.
References [24]-[28] seem relevant but are not cited in the text. Please fix.
9.
Finally, the paper is full or writing errors. They are not serious and do not prevent understanding the key points, but nevertheless, it is embarrassing that the authors may not have applied grammar checking. Maybe the checking did not reach the equation (9) (“diatance”), and may not detect anything wrong with “counting cluster number”, but professional proof-reading, or free-service Grammarly.com would help a lot here.
There are also many errors in equations:
- Extra ) and } in Equation (1)
- i <= i in Equation (2)
- copy-paste error in Equation (3)
- wrongly formulated Equation (5)
- falsely mixing sd and cd in Equation (7)
- useless monster equation (9)
All variables should also be written using italic font, and exponents as superscripts, and also subindexes using subscripts. These cannot be let for typesetter task alone.
References:
1. Q. Zhao and P. Fränti, "WB-index: a sum-of-squares based index for cluster validity", Data & Knowledge Engineering, 92, 77-89, July 2014.
2. M.I. Malinen and P. Fränti, "All-pairwise squared distances lead to more balanced clustering", Applied Computing and Intelligence, 3 (1), 93-115, 2023.
3. P. Fränti and S. Sieranoja, "K-means properties on six clustering benchmark datasets", Applied Intelligence, 48 (12), 4743-4759, December 2018.
**PeerJ Staff Note:** It is PeerJ policy that additional references suggested during the peer-review process should only be included if the authors are in agreement that they are relevant and useful.
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.