Abstract
Comparative Analysis of Expert, Clinician, and Consumer Interactions With Summary of Findings Tables: A Quasi-Experimental Study
Nina Vitlov,1 Nensi Bralić,1 Tina Poklepović Peričić,2 Daniel Garcia-Costa,3 Emilia López-Iñesta,4 Elena Álvarez-García,3 Francisco Grimaldo,3 Ana Marušić1
Objective
To explore how GRADE and Cochrane experts, clinicians, and health care consumers interact with Summary of Findings (SoF) tables from systematic reviews of evidence when answering questions about table content.
Design
A quasi-experimental study was conducted from February to June 2025, using the Read&Learn platform, an online tool designed for monitoring the interaction with a word problem or a table.1,2 The sample size was calculated at 25 participants per group to detect a mean (SD) of 4 (5) points’ difference in the number of correct answers (range, 0-16), with 80% power and 5% α level. The study intervention was blurring of SoF table cells, which the participants were required to navigate and open in order to answer questions about 4 different SoF tables, alternating between the question screen and table cells. All necessary data for answering the questions were contained within the target cells, requiring no additional calculations. We measured the number of correct answers to the questions about SoF tables, total number of table cells visited, number of target table cells visited, number of nontarget table cells visited, total time spent on the quiz from beginning to end, time spent on initial reading, total time spent reading table cells, time spent on reading target table cells, and time spent on reading nontarget table cells. The study was approved by the Ethics Committee of the University of Split School of Medicine, and all participants gave informed consent.
Results
We collected the data from 40 experts, 40 clinicians, and 40 health care consumers (Table 25-0884). There were no significant differences between experts, clinicians, and health care consumers in terms of quiz performance, with all groups achieving a similar number of correct answers. Additionally, total time spent on the quiz, initial reading, and quiz questions did not differ between the groups. Significant differences were observed in their interaction with the SoF tables. Experts visited fewer numbers of table cells in total and fewer nontarget segments and spent the least amount of time reading the nontarget segments, compared with clinicians and health care consumers. They also spent the least amount of time reading the question statement.
Conclusions
Experts in evidence synthesis, Cochrane systematic review authors, and GRADE experts showed different interaction patterns with the SoF table, possibly reflecting their greater familiarity with the content. Future studies, including qualitative approaches, should further explore how different users approach and use the information presented in SoF tables.
References
1. Matas J, Tokalić R, García-Costa D, et al. Tool to assess recognition and understanding of elements in Summary of Findings table for health evidence synthesis: a cross-sectional study. Sci Rep. 2023;13:18044. doi:10.1038/s41598-023-45359-x
2. Sanz MT, López-Iñesta E, García-Costa D, Grimaldo F. Measuring arithmetic word problem complexity through reading comprehension and learning analytics. Mathematics. 2020;8(9):1556. doi:10.3390/math8091556
1Department of Research in Biomedicine and Health, Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia, nina.vitlov@mefst.hr; 2Department of Prosthodontics, School of Dental Medicine, University of Split School of Medicine, Split, Croatia; 3Department of Computer Science, Universitat de València, València, Spain; 4Department of Didactics of Mathematics, Universitat de València, València, Spain.
Conflict of Interest Disclosures
Nina Vitlov is funded by the Croatian Science Foundation under the Programme for Career Development of Early Career Researchers-Training of New Doctoral Students, NPOO (C3 2 R2-I1). Ana Marušić is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.