All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
Both reviewers are happy with the revision. As a result, the paper is ready to be accepted.
No comment.
No comment.
No comment.
No comment.
The structure and readability of the manuscript have been improved a lot.
The procedure of the experiment and the settings of the parameters are described in sufficient detail.
no comment
The revised manuscript has been greatly improved and is suitable for publication.
The concerns highlighted must be addressed. Please see the reviewers' comments.
[# PeerJ Staff Note: Please ensure that all review and editorial comments are addressed in a response letter and any edits or clarifications mentioned in the letter are also inserted into the revised manuscript where appropriate. It is a common mistake to address reviewer questions in the response letter but not in the revised manuscript. If a reviewer raised a question then your readers will probably have the same question so you should ensure that the manuscript can stand alone without the response letter. Directions on how to prepare a response letter can be found at: https://peerj.com/benefits/academic-rebuttal-letters/ #]
The paper's readability has improved significantly. However, I did not intend to imply that the paper needed an editorial service; and I am sorry the authors had to retain one.
N/A.
N/A.
The authors have addressed my previous comments; and the paper has been extensively reworked. At this point I am amenable towards this paper's acceptance.
The structure of manuscript is not well-organized, and the content is not coherent, making it less readable.
The procedure of the experiment and the settings of the parameters are described with sufficient detail. Meanwhile, the meaning of each metric is illustrated in detail.
The dpi of the figures in the manuscript is too low. After downloading the figures from the attachment, some results can be verified. However, the reviewer cannot correspond to the ”BNC=10” in line 532, where did the authors get it? It should be emphasized that the discussion is difficult to be well understood, and it is necessary to report the results in a more structured manner.
1) The merits and demerits of the four touchstone similarity equations should be introduced in detail.
2) There are 8 formulas in the section of experimental design, the authors only analysed the PCC, why not discuss the other optimal BNCs?
3) On line 202 page 10, the way of citation is inconsistent with the previous content, please double-check.
4) Some abbreviations are defined repeatedly (e.g., PCC, MRC, COS, JAC in section ITEM INDEPENDENCY ANALYSES)
Both reviewers raised major concerns. As a result, an essential revision is recommended.
[# PeerJ Staff Note: Please ensure that all review and editorial comments are addressed in a response letter and any edits or clarifications mentioned in the letter are also inserted into the revised manuscript where appropriate. It is a common mistake to address reviewer questions in the response letter but not in the revised manuscript. If a reviewer raised a question then your readers will probably have the same question so you should ensure that the manuscript can stand alone without the response letter. Directions on how to prepare a response letter can be found at: https://peerj.com/benefits/academic-rebuttal-letters/ #]
[# PeerJ Staff Note: The review process has identified that the English language must be improved. PeerJ can provide language editing services - please contact us at copyediting@peerj.com for pricing (be sure to provide your manuscript number and title) #]
The manuscript contains a lot of grammatical inadequacies which makes it practically impossible for me to logically follow its flow.
I will recommend that the authors should seek the assistance of a native English speaker to help them in editing the manuscript as it cannot be publish in its present form.
The entire manuscript should be written in good English language and re-submitted for review.
Manuscript has to be re-written first in good English language before comments can be made in this section.
Manuscript has to be re-written first in good English before comments can be made in this section.
Manuscript has to be re-written first in good English language before comments can be made in this section.
The language needs to be improved. A non-exhaustive list of stylistic glitches and suggestions are:
-- Abstract
- 24 "recommendations that.." --> "finding recommendations which appeal to each user varies"
- 26 "it is measured" --> "we measure the appropriateness of the recommendation in terms of"
-- Introduction
- 58 "there are loads of" --> "there are many reported RS implementations"
- 58 "is blurry" --> "it is unclear, how the "
--- Getting started to experiments --> "Experimental design"
Other issues similar to the above.
The figures Fig 1 to 5 are illegible and it is impossible to verify the conclusions drawn from them.
The authors have championed the cause of leave one out methods, however, there should be a discussion of standard statistical augmented methods like ANOVA; that is, the approach discussed here is only true under the severe assumption that each covariate is independent of the others.
It is unclear how the median is considered to reduce the outliers; standard techniques to identify (like checking residuals and standardized residuals) should be compared. The median values are indeed better than the mean under certain scenarios, however, it is best to recall, that there are no general unbiased estimators for determining the population median.
Note that the Pearson correlation is based on the mean and so the estimator is actually an unbiased estimate of the population statistics.
It is unclear why the smooth coloring of the JAC with SW is of merit. In general, the distribution of metrics is unclear as a statistic of interest, as it is sensitive to the order of the table and the dataset. If the question is of tracking the metrics themselves, then it would be better to describe the data in terms of a density plot.
The data and code is provided, and this is commendable.
The research question posed is appropriate and the code is sufficient for the analysis of the same. However, the manuscript at this stage is not ready for publication and needs to be reworked for clarity. Several key points raised in section 2 in particular need to be addressed adequately. The paper contains implementation details and shows significant effort in terms of covering the existing metrics. I am certain it will be a suitable addition to the literature after revisions.
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.