Abstract

Authors’ Assessment of the Impact and Value of Statistical Review in a General Medical Journal: 5-Year Survey Results

Catharine Stack,1 Alicia Ludwig,2 A. Russell Localio,3 Anne Meibohm,1 Eliseo Guallar,4 John Wong,5 Deborah Cotton,1 Cynthia Mulrow,1 Jaya Rao,1 Mary Beth Schaeffer,1 Darren Taichman,1 Christine Laine1

Objective

Statistical methods for biomedical research are increasingly complex. At our general medical journal, the statistical editors, in concert with the senior editors, perform statistical review of all provisional acceptances and revisions. Adding to our previously reported 2012 survey results, we sought authors’ views annually for an additional 4 years regarding the impact of statistical review on the quality of their articles.

Design

We conducted anonymous online annual surveys of corresponding authors of all articles published in 2012-2016 that underwent statistical review. We asked authors about the effort needed to respond to the statistical editors’ requests and the impact of statistical review on the quality of both specific sections and the overall published article. In 2014-2016 we also asked authors about the usefulness of optional conference calls with the editors, and an offer of such calls was included in all provisional acceptance letters sent during this period. The survey excluded rejected articles because they rarely receive full statistical review.

Results

Of 489 authors (about 100 each year) surveyed, 337 (69%) responded. Response rates varied from 60% in 2016 to 79% in 2012. Studies included reports of original research (69%), systematic reviews and meta-analyses (24%), and decision analyses (7%). Fifty-seven percent of authors (range by year, 52%-61%) reported a moderate or large increase in the articles’ overall quality as a result of the statistical editorial process; 54% noted improvements to the statistical methods section, 53% to the results section, and 32% to the conclusions section. Fewer authors reported no impact (15%) or a negative impact (2%) on the article. Fifty-eight percent of authors reported considerable effort to respond to the statistical editors’ comments. A similar proportion (54%; range, 45%-65%) found that the effort required was worth the improved quality. Ten percent found the effort was not worth the improved quality. In 2014-2016, 19% (37/196) of respondents participated in conference calls with the statistical editors; 95% (35) found these calls extremely or somewhat helpful.

Conclusions

In this survey conducted at a single, general medical journal, the majority of authors of articles published in the past 5 years reported improvements to their articles as a result of the statistical review and felt the effort required to respond to the statistical editors’ comments was worth the improved quality. Telephone conferences were almost universally considered helpful.

1Annals of Internal Medicine, American College of Physicians, Philadelphia, PA, USA, cstack@acponline.org; 2American College of Physicians, Philadelphia, PA, USA; 3University of Pennsylvania, Philadelphia, PA, USA; 4Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA; 5Tufts University, Medford, MA, USA

Conflict of Interest Disclosures:

Catharine Stack reports stock holdings in Pfizer and Johnson & Johnson. A. Russell Localio, Anne Meibohm, Eliseo Guallar, and John Wong report that funding from the Annals of Internal Medicine is provided to them or to their institution. Jaya Rao reports stock holdings/options in Eli Lily and Pfizer. Christine Laine is a member of the Peer Review Congress Advisory Board but was not involved in the review of or decision for this abstract.

Funding/Support:

No external funding was provided for this study. Contributions of staff time and resources came from Annals of Internal Medicine.