Open peer review: What are the benefits and pitfalls?

Should journals publish review reports alongside a paper? At the meeting “The future of scholarly scientific communication” at the Royal Society it seemed that there is a general agreement: yes – we need more transparency. However, opinions were divided if reviewers’ names should be published as well.

Benefits of publishing reviews with a paper

  • Researchers can see the comments by reviewers and make up their mind if they agree or disagree.
  • Reviewers’ suggestions to improve the paper are available to everyone.
  • Scholars might write better reviews if they know they get published.
  • You would see the progress of science, and not just the final output.
  • It would encourage post publication communication and discussion.
  • Students could learn from reading the paper alongside the criticism and feedback.
  • The whole peer review process could gain more trust because everything is transparent.

Should peer reviews reveal the reviewer’s name?

There was a heated discussion among Royal Society Fellows and invited guests about this question. Some feared that many researchers would not openly want to criticize their peers, colleagues or potential future members on their funding and job panels. Especially younger researchers might hesitate to sign a critical review with their name.

On the other hand, a positive development could be that a review with the reviewer name attached would eventually be ‘elevated’ to the status of a published article.* It could be cited and would be ‘on record’ in the scientific communication about that topic. This would hopefully lead to reviews of higher quality. Ideally, in the future, funders, hiring committees, and the REF would take into account that reviewers contribute to science by writing honest and high quality reviews.

Incentives to write reviews

Publishing a review (with the reviewer’s name) together with the article could also be used to incentivize writing such reviews. At the moment, there are few brownie points to be gained from writing a deep, extensive evaluation of a manuscript.

A review that is ‘signed’ with your name could gain you ‘scholarly credit’ in the form of:

  • a citable output
  • visibility in the field
  • recognition on your CV, for funders or hiring committees

Examples of five journals that do open peer review

The Atmospheric Chemistry and Physics journal publishes pdfs with peer reviews alongside the final version of the paper. You can read the “Author’s Response”, “Referee Report”, and “Editor Decision” (example article). Here are some thoughts by the journal editors on why they introduced open peer review.

F1000Research also publishes referee reports (example article).

Molecular Systems Biology publishes a “Review Process File”, which contains all reviews (anonymous), editors’ decision letters, and a time line containing dates for: submission, editorial decision, revision received, accepted (see this example).

Another example is BMC Cancer where the reviewers’ names are published with the review reports (example).

The journal Royal Society Open Science publishes reviewer reports under “Review History” for each paper. Some of these are anonymous (example), other reviewers publish their name (example).

How to implement open peer review?

While many agreed that peer review should be an open system with appropriate rewards and incentives, the question of implemention could not be resolved. There seemed to be willingness to experiment with options such as:

  • Option 1: be bold and make all peer reviews available, including names of reviewers
  • Option 2: make all peer reviews available, but leave it to the reviewers to sign their review or to keep it anonymous
  • Option 3: leave it to the article authors and peer reviewers if the review will be published (anonymously or with the name attached)

Literature

Open peer review at four STEM journals: an observational overview, by Emily Ford.

Follow the discussions from the Royal Society meeting on twitter @royalsociety #FSSC.

*Thank you Martin Dominik for making this point.

Advertisements
Tagged , ,

2 thoughts on “Open peer review: What are the benefits and pitfalls?

  1. Mike Taylor says:

    One more advantage to publishing reviews: it means that good reviewers, who make a real contribution, will get credit for that. (The converse is also true!)

    Like

  2. Mike Taylor says:

    I am intrigued by the approach taken by PeerJ. Manuscript authors make the decision of whether or not the review history is published alongside the final paper; and reviewers make the decision of whether or not to disclose their identity. So reviews may be open and signed, open but anonymous, closed but signed, or closed and anonymous, depending on the two decisions made by the two parties.

    I’m not sure whether not not this is a good system, but I offer it up for contemplation.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: