Reporting of Confounder Selection in Observational Studies in High-Impact Medical and Epidemiological Journals, 2003-2023

Luis C. L. Correia,¹ Rafael F. Mascarenhas,² Felipe S. C. de Menezes,² Jeronimo S. Oliveira Júnior,² Marcus V. Almeida,² Caio F. Azevedo,² Naieli M. de Andrade,² Viola Vaccarino,¹ Joseph S. Ross,³ Joshua D. Wallach¹

Objective

Several approaches exist for selecting confounders for adjustment in observational studies, including subjective judgment, statistical criteria, or the use of a causal model, usually represented by directed acyclic graphs (DAGs). We assessed the characteristics and trends in the reported methods for selecting confounders to control for in observational studies published in the highest impact factor medical and epidemiological journals.

Design

We identified the 10 highest Impact Factor medical (n = 5) and epidemiological (n = 5) journals according to InCites Journal Citation Reports. For each journal, we reviewed all PubMed-indexed articles published in 2003, 2013, and 2023 to identify observational studies evaluating exposure-outcome relationships in which confounder adjustment would be expected. We excluded articles that were descriptive, predictive, and quasi-experimental (ie, that primarily address confounding through design-based approaches). We randomly selected half of the articles in each journal and publication year for full-text evaluation and identified key study characteristics. We classified the methods reported by each study to select confounders: no confounder adjustment; adjustment but confounders not specified; confounders selected without justification; confounders selected based on an established association with the outcome; confounders selected based on statistical criteria (ie, imbalance between exposure groups, change-in-estimate strategy, or application of a stepwise regression); or confounders selected based on a causal model, either depicted by a DAG or explained in the text. We followed the STROBE reporting guideline for cross-sectional studies.

Results

We identified 623 eligible observational studies, including 197 (31.6%) published in medical journals and 426 (68.4%) published in epidemiological journals. Of these, 22 (3.5%) did not report adjusting for confounders, 18 (2.9%) did not specify which confounders were selected, 281 (45.1%) reported selection of confounders without justification, 139 (22.3%) reported selection of confounders based on an established association with the outcome, 121 (19.4%) reported selection of confounders based on statistical criteria, and 42 (6.7%) reported selection of confounders based on a causal model (35 used a DAG and 7 provided an explanation in their text) (Table 25-1006). The selection of confounders without justification remained relatively stable between 2003 and 2023 (from 111 of 228 [48.7%] to 68 of 164 [41.5%]), while the use of a causal model to identify confounders increased (from 0 of 228 to 37 of 164 [22.6%]). Differences in the methods used to select confounders were observed across journal type and study design, but not exposure type or funding source.

Conclusions

Although our analysis was limited to high-impact medical and epidemiological journals, we found that the reporting of causal models—such as DAGs—has increased over time. However, fewer than one-fourth of studies reported such models in 2023, raising concerns about how confounders are selected and justified in observational research.

¹Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, US, joshua.wallach@emory.edu; ²Bahiana School of Medicine and Public Health, Salvador, BA, Brazil; ³Department of Medicine, Yale School of Medicine, New Haven, CT, US.

Conflicts of Interest

Joseph S. Ross reported receiving grants from the US Food and Drug Administration; Johnson & Johnson; the Medical Device Innovation Consortium; the Agency for Healthcare Research and Quality; the National Heart, Lung, and Blood Institute; and Arnold Ventures outside the submitted work and is also is an expert witness at the request of relator attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc that was settled in September 2022. Dr Ross is a Deputy Editor at JAMA. Joshua D. Wallach is supported by Arnold Ventures, Johnson & Johnson through the Yale Open Data Access project, and the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under award 1K01AA028258 and previously served as a consultant to Hagens Berman Sobol Shapiro LLP and Dugan Law Firm APLC. Dr Wallach is a Specialty Associated Editor at JACC.