Abstract

Publishing Peer Review Reports Alongside Articles With Separate DOIs: A Pilot Study of 5 Journals in Different Scientific Disciplines

Bahar Mehmani1

Objective

This study assessed reviewers’, editors’, and authors’ views about the publication of peer review reports (signed or anonymous depending on the reviewer’s consent) alongside articles with separate DOIs to give more credit to reviewers and add more transparency to the process.

Design

Since February 2014, Elsevier has been publishing review reports of accepted manuscripts with separate DOIs on ScienceDirect from 5 journals in different scientific disciplines (Annals of Medicine and Surgery, Agricultural and Forest Meteorology, Engineering Fracture Mechanics, International Journal of Surgery, and Journal of Hydrology). Reviewers can sign their report or remain anonymous. Survey questionnaires were sent (by email without reminder) to 644 reviewers (204 who had agreed to review and 440 who had declined to review by August 2015), 40 editors, and 3774 authors to assess the usefulness of this exercise. Emails were sent with a link to questionnaires for authors, reviewers, and editors, with a mix of response options (yes or no, multiple choice, Likert scale, and open answer).

Results

Of the 204 reviewers who agreed to review, 40 (19.6%) responded to the survey. Of these 40 respondents, 38 (95.0%) indicated that publishing peer review reports did not influence their recommendation, and 39 (97.5%) indicated they would accept further review invitations from the journal. Of the 440 reviewers who declined to review, 100 (22.7%) responded to the survey. Of these 100 respondents, 91 (91.0%) indicated that publication of the review report was not a reason for declining, 68 (68.0%) indicated lack of time as the primary reason, 24 (24.0%) indicated personal reasons, and 23 (23.0%) indicated a mismatch with areas of expertise. Sixteen of 40 editors responded to the survey (40.0%). Five editors indicated that the pilot study made it more difficult for them to find a reviewer, 2 indicated that the pilot study did not have any influence, and 9 were undecided. Ten journal editors said that since the pilot study, they noticed that the reports were more in depth and constructive for authors, 5 did not notice any difference, and 1 did not reply to this question. Six editors mentioned that they used the published review reports of their journal as examples of the reviewing process for younger reviewers. Of the 3774 authors whose manuscripts were accepted by the 5 journals and who were sent the survey, 501 (13.3%) responded (Table). The percentage of authors who indicated that they like the publication of the peer review reports ranged from 48.7% (Engineering Fracture Mechanics) to 60.5% (Annals of Medicine and Surgery), and 51.6% to 63.9% of authors indicated that this policy would not influence their decision where to publish.

Conclusions

Although the findings are limited by the low response rates, the communities served by the journals in this study seem to be open to the practice of publishing peer review reports. Based on the result of this pilot study, Elsevier will make it possible for reviewers of its other journals from similar research areas to publish their review reports.

1

Elsevier, Amsterdam, the Netherlands; b.mehmani@elsevier.com

Conflict of Interest Disclosures:

None reported.

Funding/Support:

This study was funded by Elsevier.

Additional Contributions:

Dr Mehmani acknowledges 5 pilot journal editors for supporting the pilot.