Abstract

Modeling the Effects of Jointly Implemented Peer Review Systems on Scientific Publication

Michail Kovanis,1,2 Philippe Ravaud,1-5 Raphael Porcher1-3

Objective

To estimate the effects of joint implementation of different peer review (PR) systems on overall PR efficiency, reviewer effort, and scientific dissemination.

Design

We used a previously developed model of the scientific publication system with data inputs from surveys of researchers (conducted by us and Elsevier) and from journals (Journal Citation Reports 2013). In the model, researchers were characterized by their resources and scientific level, papers by their scientific value, and journals by their reputation and acceptance/rejection thresholds. Papers could be reviewed and revised, then accepted or rejected, then resubmitted or abandoned if rejected. We considered the conventional PR system and 5 alternatives: (1) up to 1 round of reviews and revisions allowed (re-review opt-out), (2) rejected papers resubmitted with their past reviews to another journal (portable PR), (3) rejected papers resubmitted with their past reviews to journals of lower reputation within 1 publisher network (cascade PR), (4) articles available online on submission and editors considered both invited reviews and comments from the community (crowdsourcing PR), and (5) a similar system without online comments (immediate publication). We modeled the effects of joint implementation of the 5 systems (Figure) on PR (measured as the separation between distributions of scientific value of published and unpublished papers [Hellinger distance] and as the mean relative increase in scientific value), on reviewer effort (measured as the total time spent in PR), and on scientific dissemination (measured as the median time from submission to final decision, and the average weekly release of scientific information [paper scientific value × journal reputation]) varied by journal adoption rates.

Results

Compared with conventional PR, scenarios in which review-sharing systems (alternative 2 or 3) were dominant (50% adoption) had the greatest impact on scientific publication: the separation of the distributions of scientific value was unchanged, the mean changes in papers’ scientific value increased by 1.3% to 13.8% (range of means across scenarios), the total time devoted to PR decreased by 33.5% to 64.3%, the median time to final decision decreased by a relative 31.4% to 47.3% and the release of scientific information increased by 3.1% to 36.5%. The scenario with equal (16.7%) adoption of all alternative systems was almost as beneficial.

Figure. Model Results for Peer Review Systems

Conclusions

In this simulation of joint implementation of different PR systems, review-sharing systems seemed most promising at increasing PR efficiency and decreasing PR effort, and may be further tested in real-world trials.

1INSERM U1153, Paris, France, michail.kovanis@aphp.fr; 2Université Paris Descartes–Sorbonne Paris cité, Paris, Franc; 3Assistance Publique–Hôpitaux de Paris, Hôpital Hôtel–Dieu, Centre d’Epidémiologie Clinique, Paris, France; 4Cochrane France, Paris, France; 5Department of Epidemiology, Columbia University Mailman School of Public Health, New York, NY, USA

Conflict of Interest Disclosures:

None reported.