Abstract
Association Between the Journal Evaluation Program of the Korean Association of Medical Journal Editors (KAMJE) and Change in Quality of Member Journals
Hee-Jin Yang,1,2 Se Jeong Oh,1,3 Sung-Tae Hong1,4
Objective
In 1997 the Korean Association of Medical Journal Editors (KAMJE) instituted a program to evaluate Korean medical journals. Journals were evaluated on criteria such as timeliness, quality of editorial work, and adherence of bibliography and citations to high standards. Journals that passed the initial evaluation process were indexed in KoreaMed, the Korean version of PubMed. Here, we report changes in measures of quality among the KoreaMed-indexed journals that were associated with the evaluation program after 7 years.
Design
Quality measures used in the study comprised self-assessment by journal editors and assessment of the journals by KAMJE reviewers and by Korean health science librarians. Each used detailed criteria to score the journals on a scale of 0 to 5 or 6 in multiple dimensions. We compared scores at baseline evaluation and after 7 years for 129 journals and compared improvements in journals indexed vs. not-indexed by Web of Science.
Results
Among 251 KAMJE member journals at the end of 2015, 227 passed evaluation criteria and 129 (56%) had both baseline and 7-year follow-up assessment data. The journals showed improvement overall (increase in mean [SD] score from baseline, 0.53 [0.48]; 95% CI, 0.44-0.61; P < .001) and within each category of evaluation (mean [SD] increase by editor’s assessment, 0.14 [0.58]; 95% CI 0.04-0.26; P = .007; reviewer’s, 0.43 [0.76]; 95% CI, 0.29-0.57; P < .001; and librarian’s, 1.98 [1.15]; 95% CI, 1.77-2.18, P < .001) (Table). Before the foundation of KAMJE in 1996, there were only 5 Korean medical journals indexed in the MEDLINE and 1 indexed by Web of Science (SCI). By 2016, there were 24 journals listed in MEDLINE and 34 journals indexed in Web of Science (SCI). There was no statistically significant difference in scores on initial assessments between 21 SCI-indexed and 108 non-indexed journals, but the scores of the SCI-indexed journals were significantly higher on follow-up assessments (mean [SD], 3.99 [0.37] vs 3.38 [0.43]).
Conclusions
These results suggest an association between a program of assessment by editors, reviewers, and librarians and improvement in quality of KAJME member journals. The increase in the number of KAJME member journals indexed in international databases also suggests that the KAMJE program is successful at improving journal quality.
1The Korean Association of Medical Journal Editors, Seoul, Korea, hst@snu.ac.kr; 2Department of Neurosurgery, Seoul National University Boramae Hospital, Seoul, Korea; 3Department of Surgery, The Catholic University of Korea, Incheon St. Mary”s Hospital, Incheon, Korea; 4Department of Parasitology and Tropical Medicine, Seoul National University College of Medicine, Seoul, Korea
Conflict of Interest Disclosures:
None reported.
Funding/Support:
Financial support by a grant from the Korean Association of Medical Journal Editors (KAMJE), 2015.
Role of the Funder/Sponsor:
The funder supplied data of the journal evaluation program but was not otherwise involved in the study.