Abstract
Use of Open Review by Discipline, Country, and Over Time: An Analysis of Reviews and Journal Policies Posted on Publons
Sarah Parks,1 Salil Gunashekar,1 Elta Smith1
Objective
To understand what forms of open peer review are being used and how these vary by discipline, by country, and over time.
Design
We used data from Publons to explore our objectives. Publons is a publisher neutral platform that allows users to record peer reviewer activity and their preferences for signing and/or publishing reviews and also records journal policies on peer review. Publons publishes reviews openly if the reviewer has selected this option, provided the journal policy permits this. We focused on the 7458 journals with at least 10 reviews recorded in Publons as of November 2016. Publons also contains information on the peer review policies of approximately 12% of journals (3692 of 30,000 assuming there are 30,000 English-language journals in the world). We split the data by the country of the reviewer (where there are at least 100 distinct reviewers) and the discipline of the journal to look at trends in use of open peer review, including the difference between the wishes of reviewers to publish their reviews and journal policies and the current state of policies for open peer review for journals in different fields. The disciplines assessed included life sciences, natural sciences, engineering sciences, humanities, and social and behavioral sciences.
Results
Of the 3692 journals analyzed, 3.5% (n=130) allow reviewers to sign their reviews, and 2.3% (n=85) allow or require peer reviews to be published. There are 474,036 reviews in Publons from the 7458 journals that have at least 10 reviews recorded in Publons. Of these reviews, 1.7% (n=7857) are published openly. For an additional 7904 reviews (1.7% of total reviews), the reviewers have indicated that they would prefer Publons to publish the review openly, but the journal policy does not permit this. The rate of published open reviews in Publons varies by country, ranging from 0% in Argentina (0 of 1076), Czech Republic (0 of 867), Ireland (0 of 1667), and Romania (0 of 1392) to 14.1% (149 of 1058) in Saudi Arabia, and by discipline, ranging from 0.8% (205 of 26,762) for engineering sciences to 2.9% (18 of 615) for humanities.
Conclusions
Open peer review is happening in different forms on the Publons platform, but it still accounts for a small percentage of reviews carried out. While these data provide insight into the use of open peer review, it is important to note that the data are sourced from Publons and may not be representative of all scholarly disciplines, countries, open reviews, and open reviewer activity. In addition, we were not able to differentiate the posting of full reviews vs the posting of metadata of reviewer activity. Future research could address these limitations and other aspects of open review.
1RAND Europe, Cambridge, UK, sparks@rand.org
Conflict of Interest Disclosures:
None reported.
Funding/Support:
Initial work on this topic was funded by the European Commission as part of a project to develop a monitor for open science.
Role of the Funder/Sponsor:
The funder/sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract.
Acknowledgments:
We would like to thank Publons Limited and Publons UK Limited for providing us with data.