Recently, a manuscript I submitted to a journal came back with two reviews. Reading through these positive and brief reports, I was considering each comment and planning changes I would make in response. And there, at the very end of the second review, was the name of Reviewer 2: Albus Dumbledore. Well, it wasn’t actually Professor Dumbledore, but my point will be that the name isn’t important nor should it have been included.
Although I argue in favour of either single-blind (reviewer names removed) or double blind (both reviewer and author names removed) anonymous peer review, it is worth considering why a reviewer might choose to sign a review. Some argue that open peer review promotes integrity and transparency in the dissemination of scientific discoveries. By signing a review and relinquishing anonymity, a reviewer makes a strong statement about being accountable, being confident about the quality of the review, promoting openness, and being constructive in the review process. Some might sign reviews because they might otherwise be more negative and critical behind the protective wall of anonymity. Although Mick Watson published a blog post here that discusses the benefits of signing reviews, and drawbacks of anonymous peer review, I find that most of my rationale in support of anonymous peer review is distinct from his discussion.
Here are several reasons why I advocate for anonymous peer review.
1. Anonymous peer review works. Having read hundreds of reviews of my own research papers, and being involved in the review process of many other articles, I have only flagged a handful of poor reviews, where the reviewer was obviously out to lunch or had an axe to grind. The remainder were well-considered and fair criticisms of the work. As a reviewer, my reviews usually agree with the other reviewers. Rejections, even for many of my own papers, have almost always been well founded, despite the upset and frustration that accompany such negative outcomes. I do not see how the vast majority of my reviews, or the countless others I’ve seen, would have been improved substantially by signatures at the bottom.
2. Accountability is inherent in the system. Editors are leaders in the field who monitor the review process and those who review for them. Strong reviewers garner favour and are eventually asked to become editorial board members or editors themselves. Weak reviewers do not get called on again and, I suspect, risk their professional reputation. A signature is not needed, nor should be needed, to ensure that a scientist performs a review task professionally and in an unbiased manner. If, on the other hand, a scientist would be overly critical or biased in the absence of a signature, this is an ethical concern; I would not want that scientist reviewing my work regardless.
3. The editor as "good cop". Many papers should be rejected. It is not uncommon to see authors demonstrate poor judgment, submit experimental work that was poorly designed or conducted, misjudge the “impact" of their work, neglect to edit their manuscript sufficiently prior to submission, or prepare figures and tables of unacceptable quality. For many reasons, authors often disagree with the critical comments of reviewers, even when the criticisms are well founded. Being critical of the science, reviewers serve as “bad cops”. In contrast, editors are the good cops, the messengers, the bearers of bad news. Editors may take a lot of flack for rejecting papers, but they can stand behind the reviews to justify their decision. Because reviewers are asked to be bad cops in this process, I do not think that they should also be in the firing line of those who stand subjectively behind their questionable manuscripts and data. Worse still, many journals make decisions to reject based on perceived impact. In those cases, the authors are justifiably upset and I would not want them angry with reviewers for not "raving" about their paper sufficiently to the editor of a high impact factor journal. Given the enormous professional stakes involved with publishing (e.g., job security, promotion, salary, reputation among peers, awards, student success), signing reviews adds an unnecessary element of risk of "bad blood" between reviewers and authors.
4. Let my review be unbiased. Environmental Microbiology had an unfortunate tradition of publishing some of the silly things reviewers said in their evaluations. I was critical of this practice and asked the editors that I never be cited like this because it would make me self conscious about my reviews. For example, I might think that if I just tweak this sentence to be a little more funny, or damning, or tongue in cheek, maybe I'd be quoted in the referees’ quotes section next year. Likewise, if my peer review was signed, I might worry just a little more about the wording of my review, what the authors might think, what I will say to them later, whether they might hold a grudge. If reviewing the paper of a senior scientist in my field, one who might evaluate my next paper, grant proposal, or application for promotion, I might be tempted to sugar coat my concerns a little, even when I know that firmness and objective criticism is essential. Because I'm a "nice guy”, it is particularly important to depersonalize the process of evaluating science. In other words, I would worry too much about not being perceived as a jerk, which could detract from my duty to express very serious concerns about the quality of data I see in many manuscripts that cross my desk. In contrast, if I review a manuscript briefly and favourably, a signature could be perceived as a conflict of interest (see point 5).
5. No reviewer kickbacks. In my case, Albus Dumbledore offered a favourable and brief review of my manuscript. Professor Dumbledore is now in my good books and I will certainly suggest this reviewer again. Perhaps I will send a thank you card, bottle of butterbeer, or an invitation to speak at an upcoming conference. Perhaps I will be generous when reviewing the next paper or grant proposal from this colleague. Conversely, any overly critical reviewers will be on my blacklist. I will avoid recommending them as reviewers in the future or, worse yet, I might ask an editor that they not review my next submission. Would I do all of this? No. Could I? Yes, very easily. Preventing kickbacks may be the single most important reason why anonymous peer review is essential.
6. I don’t need it. That Professor Dumbledore reviewed my paper has little value to me as an author. Without the reviewer name, I am still fully able to work with an editor to incorporate recommended changes and respond to criticism. In my recent example, anonymous Reviewer #1 provided a highly similar review to Albus Dumbledore and both reviewers identified two of the same primary weaknesses that needed addressing in our manuscript.
7. There are better ways to improve science publishing. Although there are many things that can be done to improve the scientific publishing process, signing reviews is not a step in the right direction. A poor review that is signed is still a poor review. A common goal here is to improve the quality of the review process and there are steps that scientific mentors and journals can take to improve the system. Supervisors should partner with senior graduate students and postdocs on joint peer reviews, complementing other forms of early career training. I would also like to see increased efforts by journals and academic societies to educate scientists to be effective reviewers. For example, professional workshops organized by academic societies could focus on peer review skills, much as online reviewer training courses could help provide the basics. Editors should work with reviewers to ensure that poor quality reviews do not end up on authors’ desks. Importantly, journals should do much more to provide reviewers and editorial board members with regular performance metrics, seeking ways to incentivize the review process for faster turnaround times and higher quality reviews. Rewarding strong reviewers for their contributions could help, perhaps through a point system that can be used toward offsetting open access publication charges. In other words, providing training and incentives for reviewers to increase their performance would help address the root of the peer review problem better than signing reviews.
For the reasons outlined above, I will include a request in future cover letters that any reviewer names be removed prior to returning my manuscript for revision. In addition, I will no longer review for journals that publish reviewer names alongside each article and, as much as possible, avoid publishing in those same journals. My preference is for reviewers to remain anonymous, for journals and editors to promote and uphold the integrity of anonymous peer review, and for scientists to take the role of reviewer seriously, whether we sign our reviews or not, continually seeking to improve our performance for the betterment of science.