Veronica Yank,1 Sanjhavi Agarwal,1 Rhea Red,2 Amy Lozano1
Some of the most highly marketed mobile applications (“apps”) focus on weight loss and physical activity. The US Food and Drug Administration has declined to regulate them. It is unknown whether app research studies that receive financial support from commercial entities are similar to studies without commercial support in the degree to which they protect against bias. We sought to determine whether app studies with commercial financial ties were more or less likely than others to have discordant results and conclusions or other characteristics pertinent to assessing bias.
We performed a retrospective cohort study of English-language randomized clinical trials of weight loss or physical activity apps published through October 2016. We searched PubMed, the Cochrane Database, EMBASE, and Web of Science using standardized approaches to identify articles that met initial literature search criteria. Two reviewers blinded to author name, affiliation, financial support, and conflict of interest disclosures performed title, abstract, and full-text review (as necessary) to determine final study eligibility. For included studies, 2 blinded reviewers independently assessed direction of study results (favorable or not favorable toward study app), direction of conclusions, and whether trial registration information and 7 design elements that protect against bias (from Cochrane Risk of Bias tool) were present. Financial ties were extracted after other assessments were complete. Financial ties were categorized as commercial (eg, ties to an app company) vs noncommercial (eg, ties to government or a foundation). Results and conclusions were defined as discordant if they disagreed in direction (results, not favorable; conclusions, favorable). Comparisons using Fisher exact tests and t-tests were performed using Stata statistical software (version 14.2; StataCorp).
Among 876 unique articles identified on initial literature search, 17 met inclusion criteria: 7 (41%) with commercial ties and 10 (59%) with noncommercial ties (Table). A smaller percentage of commercial trials reported favorable results than did noncommercial trials (14% vs 60%, respectively; P = .13), whereas a larger percentage reported favorable conclusions (86% vs. 70%; P = .60). As a result, commercial trials had significantly greater discordance between results and conclusions than noncommercial trials (71% vs. 10%; P = .04). They also were less likely to report trial registration (14% vs 100%; P = .001) and had fewer design elements that protect against bias (mean [SD], 1 [0.4] vs 4 [1.2]; P < .001).
Randomized clinical trials of weight loss and physical activity apps with commercial financial ties were more likely to have discordance between results and conclusions and less likely to meet registration and design standards.
1University of California, San Francisco, San Francisco, CA, USA, firstname.lastname@example.org; 2San Francisco State University, San Francisco, CA, USA
Conflict of Interest Disclosures:
Dr Yank is supported by a career development award from the National Institute of Diabetes and Digestive and Kidney Diseases (K23DK097308).
Role of Funder/Sponsor:
The National Institute of Diabetes and Digestive and Kidney Diseases had no role in the design, performance, or interpretation of findings of this study.