Abstract
A Comparison of Reviewer Contribution Distributions in Publons
Andrew Preston,1 Tom Culley1
Objective
A 2015 study by Kovanis et al used an early Publons data set to develop a mathematical model that showed an imbalance in the distribution of peer review effort across the biomedical research community. They concluded that 20% of researchers performed between 69% and 94% of reviews. Their study focused on biomedical literature, but the Publons data set was not restricted to that specific research field. Publons is a platform that allows peer reviewers to track, verify, and claim credit for their peer review contributions across the world’s journals. Since 2015, the Publons platform has grown significantly. More than 1350 journals from many disciplines now integrate directly with the system, giving a broader sampling of peer reviewer contributions. In this study we present an updated data set to report the distribution of reviewer workload in the biomedical sciences and other research fields, seeking to confirm whether the heavy reviewing workload is borne by a small proportion of reviewers across all research fields.
Design
We used empirical distribution of prepublication review contributions recorded for the 2015 calendar year by researchers on Publons and tagged by discipline (as defined by the All Science Journal Classification provided by SCOPUS) and country. Researchers more likely to have incomplete review records for any given year were filtered out (by only including researchers who signed up ≥3 months after the completion of that year but had added their reviews from the previous year) as a means of controlling for incomplete Publons records and bias.
Results
Overall, there were 51,482 reviews completed by 35,248 reviewers in Publons in 2015 (Table). Of these, 23,600 reviewers (67%) were excluded because they were likely to have incomplete records. This left 11,648 reviewers from 26 research fields and 126 countries. There was a more even distribution of reviews performed by reviewers than that previously reported by Kovanis et al (20% of reviewers performed 59% of the reviews compared with the 69% to 94% reported by Kovanis et al). Reviews in biomedical fields (n = 4583) were more evenly distributed, with 20% of reviewers performing 51% of the reviews compared with the 41,719 reviews in the nonbiomedical fields (20% of reviewers performed 58% of reviews). Analysis by country is ongoing and will be presented.
Conclusions
If the distribution of peer review records added by researchers active as reviewers on Publons is representative of the distribution of global prepublication peer review effort, then the peer review burden appears more evenly spread than previously suggested by Kovanis et al. The distribution is still skewed, with a small proportion of researchers shouldering much of the reviewing load, but researchers in the biomedical field seem to shoulder a more equitable load than those in other fields. Nevertheless, funders, research institutions, and publishers may want to recognize overburdened reviewers or encourage other researchers to contribute more to ensure that the quality and supply of peer review keeps up with the increasing number of article submissions.
1Publons, London, UK, andrew@publons.com
Conflict of Interest Disclosures:
Andrew Preston is the managing director and a cofounder of Publons; Tom Culley is the marketing director. Publons is a limited liability company with offices in London, UK, and Wellington, New Zealand, and was acquired by Clarivate Analytics in 2017.