Editorial Evaluation, Peer Review, and Publication of Research Papers With and Without Online-Only Supplements: Quality vs Superior Tonnage

Annette Flanagin,1 Stacy Christiansen,1 Chris Borden,1 Demetrios Kyriacou,1 Caroline Sietmann,1 Elaine Williams,1 Larry Bryant,1 Jamie Reickel,2 Allison Newton2


Online-only supplements are a useful option for publication of content or data that may not fit in the space allotted to a scientific article, but the value of these supplements is not known. We conducted this study to determine if research papers with supplements have a higher likelihood of submission, peer review, acceptance, and online usage after publication than those without supplements.


Cohort study that assessed the numbers and types of supplements submitted with research manuscripts to the 3 medical journals (1 general and 2 specialty) in The JAMA Network with the highest volume of research submissions in 2016: JAMA (n=4416), JAMA Internal Medicine (JIM) (n=1858), and JAMA Pediatrics (JPED) (n=1711). We compared rates of rejection without review vs peer review and acceptance vs rejection for papers with supplements vs papers without supplements and analyzed the peer reviewer comments for the supplements. In addition, we evaluated all research articles published in these 3 journals in 2016 (n=374) and the numbers and types of supplements published. For all articles published with supplements, we compared online usage (views and downloads) of articles vs the supplements for a defined period, January 2016-March 2017. For analysis of research submissions, we calculated risk differences with 95% CIs as well as2-sided P values by χ² test.


Overall, 7985 research manuscripts were submitted to the 3 journals: 4868 (61%) with supplements and 3117 (39%) without supplements (Table 60). Papers with supplements were more likely to be peer reviewed than those without supplements: JAMA 34.1% vs 16.5% (difference, 17.6%; 95% CI, 15.1%-20.1%); JIM 17% vs 9% (difference, 8.0%; 95% CI, 4.8%-11.1%); JPED 32.9% vs 22.9% (difference, 10%; 95% CI, 5.8%-14.3%). Papers with supplements also more likely to be accepted than those without supplements: JAMA 3.9% vs 0.3% (difference, 3.6%; 95% CI, 2.9%-4.4%); JIM 8.4% vs 1.3% (difference, 7.1%; 95% CI, 5.4%-8.9%); JPED 8.7% vs 0.3% (difference, 8.4%; 95% CI, 6.6%-10.2%). Similar results were seen after excluding clinical trials and meta-analyses (for which the majority had supplements). Of 1421 papers with supplements that were sent for peer review, 484 (34%) were commented on by 538 reviewers. Of these comments, 155 (29%) were substantive comments, 63 (12%) indicated moving the supplemental content to the main paper, and 59 (11%) asked for more information about the supplement. Of 374 published research articles, 372 had supplements. Median online usage of articles far exceeded that of the supplements (JAMA: 10,918 vs 37 views or downloads; JIM: 3573 vs 12; JPED: 1386 vs 5). For articles, the majority of online usage (75%) occurred within 2 months of publication; for supplements, the majority of online usage (78%) took 7 months to occur.


Although less valuable to peer reviewers and readers, online supplements appear to be a marker for complexity and quality of research manuscripts submitted and published in these 3 medical journals.

1The JAMA Network, Chicago, IL, USA, annnette.flanagin@jamanetwork.org; 2eJournalPress, Rockville, MD, USA

Conflict of Interest Disclosures:

Other than affiliations with The JAMA Network journals included in the study and eJournalPress (provider of the manuscript submission and production systems used in the study), no other conflicts of interest are reported. Annette Flanagin is the Peer Review Congress Executive Director but was not involved in the review of or decision for this abstract.


No external funding provided.