Visitors   Views   Downloads

Can editors save peer review from peer reviewers?

View preprint
悪徳レビュアーのブラックリスト化
91 days ago
RT @PeerJPreprints: Can editors save peer review from peer reviewers? Preprint and discussion on referee bias https://t.co/te4USF5jZP #Peer…
Can editors save peer review from peer reviewers? Preprint and discussion on referee bias https://t.co/te4USF5jZP #PeerRevWk17 https://t.co/bH5PgqXnS0
Can editors save peer review from peer reviewers? https://t.co/wMtW2ZXaLP
99 days ago
Nice article in @thePeerJ Preprints. We need someone to save #PeerReview from peer reviewers https://t.co/eg7C1NIDS6
RT @Xenpaper: Can editors save peer review from peer reviewers? https://t.co/EKBrozBEMG
99 days ago
Can editors save peer review from peer reviewers? https://t.co/EKBrozBEMG
RT @raf_dandrea: Check out my article about peer review, co-authored with James O'Dwyer: Can editors protect peer review? https://t.co/La5n…
RT @JHeditor: Can editors protect peer review? https://t.co/dtKn52t33A via @PeerJPreprints
174 days ago
RT @JHeditor: Can editors protect peer review? https://t.co/dtKn52t33A via @PeerJPreprints
175 days ago
Can editors protect peer review? https://t.co/dtKn52t33A via @PeerJPreprints
RT @caz_porter: Can editors protect peer review from bad reviewers? https://t.co/xEoGaOYTgN via @PeerJPreprints
Can editors protect peer review from bad reviewers? https://t.co/xEoGaOYTgN via @PeerJPreprints
Can editors protect peer review? https://t.co/tlHS4K9LiY via @PeerJPreprints
RT @pash22: Can editors protect peer review from bad reviewers? https://t.co/ksluPu2XKW via @PeerJPreprints
RT @LeeJRickard: Can editors protect peer review? https://t.co/JFgfg7iBuX via @PeerJPreprints
Good question on peer reviews. https://t.co/CUyrnwbNHD Another no one has tackled yet is how to protect ourselves from bad editors with COI.
Can editors protect peer review? https://t.co/JFgfg7iBuX via @PeerJPreprints
180 days ago
Can editors protect peer review from bad reviewers? https://t.co/ksluPu2XKW via @PeerJPreprints
RT @sci_journalism: Can editors protect peer review from bad reviewers? https://t.co/Q3asp8yTGv
RT @sci_journalism: Can editors protect peer review from bad reviewers? https://t.co/Q3asp8yTGv
182 days ago
RT @sci_journalism: Can editors protect peer review from bad reviewers? https://t.co/Q3asp8yTGv
Can editors protect peer review from bad reviewers? https://t.co/Q3asp8yTGv
Can editors protect peer review from bad reviewers? @PeerJPreprints https://t.co/DYpPDw1ynA
RT @GrumpybaldProf: Can editors protect peer review? https://t.co/1YGX4V9lpG via @PeerJPreprints good question!!
NOT PEER-REVIEWED
"PeerJ Preprints" is a venue for early communication or feedback before peer review. Data may be preliminary.

Supplemental Information

Appendix S1: Blacklisting referees

We provide the mathematical details of the editorial strategy of blacklisting referees with a high record of disagreements.

DOI: 10.7287/peerj.preprints.3005v4/supp-1

Figure S1: Effect of narcissistic referees

Narcissists accept only manuscripts that are similar enough to their own work to fall within the quality interval covering 95\% of their own scientific production. These are meant to represent referees with a (conscious or unconscious) bias towards endorsing the relevance/importance of manuscripts on their subfield of expertise. Here we plot the effect of narcissistic referees on the quality of accepted (\textbf{A}) and rejected (\textbf{B}, \textbf{C}) papers, as a function of their percentage in the referee pool (the remainder being moving-standard impartial referees). For comparison, we also plot the effect of indifferent selfish referees (described in the main text).

DOI: 10.7287/peerj.preprints.3005v4/supp-2

Figure S2: Two versus three referees

Average quality of accepted papers when two (\textbf{A}) and three (\textbf{B}) referees are assigned per manuscript, in concert with each editorial strategy tested in this study. Outcomes are qualitatively similar but quantitatively different. Three referees leads to better results overall (\textbf{C}), although not by very large percentage points, and the advantage declines with higher incidence of selfish referees in the pool, even reversing in some cases. $Q_{2(3)}$ is the average quality of accepted papers under 2 (3) referees. Under three referees the editor always honors the majority vote, unless dictated otherwise by the editorial strategy at hand.

DOI: 10.7287/peerj.preprints.3005v4/supp-3

Figure S3: Normal versus lognormal quality distribution

Average quality of accepted and rejected papers under normal (\textbf{A, B, C}) and lognormal (\textbf{D, E, F}) distribution of proficiency across authors and quality across a given author's works. No editorial action considered. A normal distribution follows if manuscript quality is the end result of multiple random additive factors. A lognormal distribution occurs under multiplicative random factors. Comparison between the top and bottom rows indicates that our results are robust to relaxing the assumption of normality. Parameters: mean author proficiency 100 (normal, lognormal); standard deviation of proficiency 10 (normal), 0.5 (lognormal); standard deviation of quality per author's works 5 (normal), 0.5 (lognormal).

DOI: 10.7287/peerj.preprints.3005v4/supp-4

Additional Information

Competing Interests

The authors declare that they have no competing interests.

Author Contributions

Rafael D'Andrea conceived and designed the experiments, performed the experiments, analyzed the data, wrote the paper, prepared figures and/or tables, reviewed drafts of the paper.

James P O'Dwyer conceived and designed the experiments, reviewed drafts of the paper.

Data Deposition

The following information was supplied regarding data availability:

GitHub

https://github.com/odwyer-lab/PeerReviewFunctions

Funding

James O'Dwyer is supported by the Simons Foundation Grant #376199 and McDonnell Foundation Grant #220020439. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Feedback on other revisions


Add your feedback

Before adding feedback, consider if it can be asked as a question instead, and if so then use the Question tab. Pointing out typos is fine, but authors are encouraged to accept only substantially helpful feedback.

Some Markdown syntax is allowed: _italic_ **bold** ^superscript^ ~subscript~ %%blockquote%% [link text](link URL)
 
By posting this you agree to PeerJ's commenting policies