Abstract
Indicators of Small-Scale and Large-Scale Citation Concentration Patterns
Iakovos Evdaimon,1 John P. A. Ioannidis,2 Giannis Nikolentzos,3 Michail Chatzianastasis,1 George Panagopoulos,4 Michalis Vazirgiannis1,5
Objective
Citation counts are widely used to evaluate research influence.1 The h-index, while popular, does not account for how citations are distributed across articles or authors. This is critical when citations are strategically concentrated to boost perceived influence. We studied extreme citation concentration patterns that may distort bibliometric evaluations and merit further scrutiny.2
Design
We analyzed Scopus data up to March 2023, focusing on 1,496,680 authors with more than 5 full articles and at least 1000 citations. We used Scopus-calculated h-indices and citation counts and introduced 3 indicators of extreme citation concentration: C/h2 ratio, A50%C , and A50 . For C/h2 ratio, a small value (first percentile, 2.45) indicates that the citations of the author may have been strategically placed to increase the h-index. This metric normalizes citation volume by what would be expected if the h-index reflected broad citation distribution. A50%C measured the number of distinct citing authors contributing at least 50% of an author’s total citations. Extremely low values (first percentile, 5) suggest reliance on few citing sources, possibly from self-citation or closed groups. A50 measured the number of coauthors with whom an author has published more than 50 articles. Extremely high values (99th percentile, more than 7) may indicate large-scale collaborations and dense citation clusters. Thresholds were based on the 1% extremes of each metric’s distribution. Authors in physics and astronomy were excluded from A50 and A50%C analyses due to the influence of nuclear and particle physics. Field definitions followed the ScienceMetrix classification.
Results
In the C/h2 analysis, 14,967 authors (1%) fell below 2.45. These authors had a median of 63 articles, all with h-indices exceeding 21. Fields such as chemistry and earth and environmental sciences showed 1.5-fold higher representation of authors with extremely low C/h2 scores (Table 25-0974). In the A50%C analysis, 11,277 authors (1%) had 5 or fewer citing authors accounting for 50% of their citations. The median number of articles was 104, and most authors had an h-index greater than 20. Fields with the highest enrichment included chemistry and mathematics and statistics. In the A50 analysis, 12,015 authors (1%) had more than 7 highly recurrent coauthors (50 or more shared publications). These authors were exceptionally prolific (median, 596 articles) and had very high h-indices (median, 56; 12.8% with an h-index greater than 100). A total of 73.3% were in clinical medicine. Co-occurrence analysis revealed a strong association between low C/h2 and low A50%C values (odds ratio, 6.4; 95% CI, 5.9-6.9). There was modest coexistence for extremely low A50%C and extremely high A50 values (odds ratio, 1.5; 95% CI, 1.3-1.7). Extremely low C/h2 very rarely co-occurred with extremely high A50 values (odds ratio, 0.09; 95% CI, 0.05-0.16).
Conclusions
Extreme patterns of citation concentration can substantially distort citation-based metrics, especially the h-index. While not implying misconduct, these metrics highlight outliers where citation behavior diverges from norms. Such metrics may serve as red flags for further analysis.
References
1. Bornmann L. How are excellent (highly cited) papers defined in bibliometrics? a quantitative analysis of the literature. Res Eval. 2014;23(2):166-173. doi:10.1093/reseval/rvu002
2. Waltman L, van Eck NJ, van Leeuwen TN, Visser MS, van Raan AFJ. Towards a new crown indicator: some theoretical considerations. J Informetrics. 2011;5(1):37-47. doi:10.1016/j.joi.2010.08.001
1LIX, École Polytechnique, Institut Polytechnique de Paris, Palaiseau, France; 2Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, US, jioannid@stanford.edu; 3Department of Informatics and Telecommunications, University of Peloponnese, Tripoli, Greece; 4Department of Computer Science, University of Luxembourg, Esch-sur-Alzette, Luxembourg; 5Mohamed bin Zayed University of Artificial Intelligence, Masdar City, Abu Dhabi, United Arab Emirates.
Conflict of Interest Disclosures
John P. A. Ioannidis is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.
Additional Information
Iakovos Evdaimon is a cocorresponding author (ievdaimon7@gmail.com).