Peer Review

In the movie Jaws, Hooper, Quint and Brody try to one-up each other comparing battle scars. Scientists engage in analogous grandstanding; we try to outdo each other with horror stories of peer review. For many of us, this is one of the most frustrating parts of science. One works hard designing experiments, collecting data and writing a manuscript. Then one submits it to a journal for peer review, and it disappears, sometimes for periods longer than it took to complete the work in the first place. In the meantime, your work on the subject often languishes in limbo, your CV remains unaccredited or contains a line with the explanation (submitted), and your personal gratification remains unsatisfied. I find myself selecting journals for publication based mainly on their "turn-around time".

What is most surprising to me is that many scientists shirk peer review when asked to perform it. I recall one colleague who suggested the following strategy: after acquiescing to a request for peer review, ignore it until after the deadline, until the editor writes asking where it is. Only then should you perform the task and submit your comments. The colleague reasoned, perhaps then the editors would leave you alone and not request future reviews from you. The strategy, in addition to being appalling and uncollegial, is particularly shortsighted. If the strategy spread, it eventually would be applied to your own submissions as well.

In a letter to PLoS Biology, Marc Hauser and Ernst Fehr propose to incentivise the peer review process by punishing transgressors. Here editors would keep databases on when papers were sent to reviewers and when the reviews were returned. The late reviews would be punished accordingly: "For every day since receipt of the manuscript for review plus the number of days past the deadline, the reviewer's next personal submission to the journal will be held in editorial limbo for twice as long before it is sent for review."

Naturally the system is not without bugs. Who is punished when multi-authored papers are submitted? What happens if one of the authors is a timely reviewer and another is a slacker? Hauser and Fehr suggest penalizing only the primary corresponding author, but I think that might easily be gamed by making the least penalized author the correspondent. Also slackers could avoid penalties by refusing requests for reviews. Hauser and Fehr suggest penalizing them by adding a one-week delay their own next submission. However, this also penalizes those who turn down reviews because of time constraints and or because they feel unqualified to do the service.

Hauser and Fehr's suggestion is interesting. Something surely must be done about the broken review process. I would object to monetizing the review process, as some commenters have suggested, but penalizing slackers with embargoes seems like a workable solution. I also like the open peer review system where submitted papers are posted for comments from scientists at large. The internet provides many new avenues to fix the peer review system and I hope journal editors consider them seriously.