The Scientific and Altmetric Impact of Retracted Systematic Review Studies Open and Closed Access

Document Type : Research Paper

Authors

1 Assistant professor, Department of Medical Library and Information Sciences, School of Allied Medical Sciences, Zahedan University of Medical Sciences, Zahedan, Iran

2 M.A. Student, Medical Library and Information Sciences, School of Allied Medical Sciences, Tehran University of Medical Sciences, Tehran, Iran

Abstract

Purpose: Retraction of articles is highly necessary to maintain the integrity and credibility of published scientific articles and to inform all audiences about potential serious scientific and ethical issues within these articles. Studies indicate that, in the field of medicine, many articles continue to receive citations even after being retracted. Altmetrics is a method based on scientific data that evaluates the impact of research and studies on social media by analyzing interactions and contributions from researchers. This study aims to investigate the scientific impact and Altmetrics of retracted systematic reviews with open and closed access.
Methodology: PubMed, Scopus, and Web of Science databases were searched using appropriate keywords. A total of 433 articles were initially retrieved; 143 duplicates were removed due to duplicates, and 81 articles were excluded for reasons such as being non-systematic reviews, not retracted, or unrelated to the medical field. Ultimately, 209 articles met the inclusion criteria for this study. Data were extracted on access type, reasons for retraction, Altmetric scores, and article citations. Retracted articles were independently reviewed by two researchers. All required data were recorded using a checklist designed in Excel, with the dataset updated as of June 2023. Firstly, EndNote version 9 was used to screen the data. Then, an Altmetrics and citation data extraction checklist was designed using Excel 2019. Finally, the data were analyzed using Excel 2019 and SPSS version 21 software. Descriptive statistics and Pearson's correlation coefficient test were employed for analytical statistics.
Findings: The findings showed that more than 67% of the articles were open access, with an average age of 6.39 ± 3.15 years. The publication period for all reviewed articles spanned from 2001 to 2022, while open-access articles were published between 2008 and 2022. These articles were retracted between 2011 and 2022. The most common reasons for retraction were honest errors for open-access articles and fake peer review for closed-access articles. The average Altmetrics score for all retracted articles was 42.44. The average Altmetrics score of retracted systematic review articles with open access was approximately 78.83 ± 463.56. The altmetrics analysis revealed that 64 of the reviewed articles had no Altmetrics score, 33 of which were closed-access articles. According to the Web of Science (WOS) database, the total number of citations for the retracted systematic review articles was 3,713. Open-access articles received more citations than closed-access articles, with an average of 17.78 ± 45.99 citations. Most of the Twitter users sharing the reviewed systematic review/meta-analysis (SR/MA) articles were members of the public, regardless of access type. It should be noted that some articles were mentioned by multiple readers on social networks. Detailed Mendeley access data, including country names, was available for only 28% of the retracted SR/MA articles. Most readers of retracted OA and CA articles on two social networks, Twitter and Mendeley, were from the United States. The results of the Pearson correlation test showed a significant correlation between the Altmetrics score and the total number of citations of the reviewed articles in the Scopus and Web of Science (WOS) databases (p-value = 0.000).
Conclusion: Providing timely information to the audience and using new technologies can lead to a reduction in the formal and informal use of these types of articles. Our findings indicate that retracted articles may continue to be used within both scientific and social communities even after their retraction. Specifically, retracted systematic review studies—often withdrawn due to scientific errors in their results and methods—may still have parts of their findings disseminated through social networks. This practice can influence clinical decision-making and potentially endanger public health. Since many researchers share their scientific work on academic social networks such as ResearchGate and Academia.edu, as well as on public platforms like X, Facebook, and YouTube, these networks should incorporate intelligent software designed to analyze and evaluate the validity of the scientific studies and sources referenced within their content. This issue requires informing senior social network managers about the need for increased monitoring of published scientific content. Providing timely information to the audience, along with utilizing new technologies, will help reduce the dissemination of such articles. Additionally, the future application of artificial intelligence techniques may minimize the number of citations to retracted articles, especially reputable articles.

Keywords

Main Subjects

References

جنوی، ا.، و مرادی، ش. (1398). سرنوشت مقالات سلب اعتبارشده کشورهای خاورمیانه در حوزه سلامت. رهیافت، 29(2)، 53-64. https://rahyaft.nrisp.ac.ir/article_13766.html
قربی، ع.، فهیمی فر، س. (1399). ابعاد و الگوهای همکاری آثار سلب اعتبارشده به‌عنوان مصداق سوء رفتار پژوهشی در سطح بین‌المللی و ایران. پژوهش‌نامه علم‌سنجی، 6(1)، ۱۴۹-۱۷۲.     
    https://doi.org/10.22070/RSCI.2019.4392.1287
قربی، ع.، فهیمی فر، س.، و نوروزی، ع. (1402). تحلیل استنادی و دلایل سلب اعتبار آثار در کشورهای خاورمیانه. پژوهش‌نامه علم‌سنجی، 9(1)، ۹۹-۱۲۴. https://doi.org/10.22070/rsci.2021.13819.1472
مرادی، ش.، و جنوی، ا. (1397). مطالعه علم‌سنجی مقاله‌های سلب اعتبارشده ایرانی. پژوهش‌نامه پردازش و مدیریت اطلاعات، 33(4)، ۱۸۰۸-۱۷۸۹. https://jipm.irandoc.ac.ir/article_699497.html?lang=fa  
Araujo, A. C., Gonzalez, G. Z., Nascimento, D. P., & Costa, L. O. P. (2021). The impact of low back pain systematic reviews and clinical practice guidelines measured by the Altmetric score: Cross-Sectional Study. Brazilian Journal of Physical Therapy, 25(1), 48-55.       
https://doi.org/10.1016/j.bjpt.2020.01.002
Bar-Ilan, J., & Halevi, G. (2017). Post retraction citations in context: a case study. Scientometrics, 113(1), 547-565. https://doi.org/10.1007/s11192-017-2242-0
Bar-Ilan, J., & Halevi, G. (2018). Temporal characteristics of retracted articles. Scientometrics, 116(3), 1771-1783. https://doi.org/10.1007/s11192-018-2802-y
Budd, J. M., Coble, Z., &  Abritis, A. (2016). An Investigation of Retracted Articles in the Biomedical Literature. Proceedings of the Association for Information Science and Technology, 53(1), 1–9. https://doi.org/10.1002/pra2.2016.14505301055
Cho, J. (2021). Altmetrics analysis of highly cited academic papers in the field of library and information science. Scientometrics, 126(9), 7623-7635.       
https://doi.org/10.1007/s11192-021-04084-w
Feng, L., Yuan, J., & Yang, L. (2020). An observation framework for retracted publications in multiple dimensions. Scientometrics, 125(2), 1445-1457.       
https://doi.org/10.1007/s11192-020-03702-3
Gaudino, M., Robinson, N. B., Audisio, K., Rahouma, M., Benedetto, U., Kurlansky, P., & Fremes, S. E. (2021). Trends and characteristics of retracted articles in the biomedical literature, 1971 to 2020. JAMA Internal Medicine, 181(8), 1118-1121.       
https://doi.org/10.1001/jamainternmed.2021.1807
Ghorbi, A., & Fahimifar, S. (2020). Aspects and collaboration patterns of retracted papers as evidence of research misconduct in Iran and foreign countries. Scientometrics Research Journal, 6(1), 149-172. https://doi.org/10.22070/rsci.2019.4392.1287 [In Persian].
Ghorbi, A., Fahimifar, S., & Noruzi, A. (2023). Citation analysis and reasons for retraction in Middle Eastern countries. Scientometrics Research Journal, 9(1), 99-124.      
https://doi.org/10.22070/rsci.2021.13819.1472 [In Persian].
Gotink, R. A., Chu, P., Busschbach, J. J. v., Benson, H., Fricchione, G. L., & Hunink, M. G. (2015). Standardised mindfulness-based interventions in healthcare: an overview of systematic reviews and meta-analyses of RCTs. PlOS One, 10(4), e0124344.       
https://doi.org/10.1371/journal.pone.0124344
Hill, A., Garratt, A., Levi, J., Falconer, J., Ellis, L., McCann, K., Pilkington, V., Qavi, A., Wang, J., & Wentzel, H. (2021). Retracted: meta-analysis of randomized trials of ivermectin to treat SARS-CoV-2 infection. Open Forum Infectious Diseases, 8(11).       
https://doi.org/10.1093/ofid/ofab358
Janavi, E., & Moradi, S. (2019). The fate of Middle Eastern countries' retracted articles on health. Rahyaft, 29(2), 53-64. https://rahyaft.nrisp.ac.ir/article_13766.html?lang=en        
[In Persian].
Khan, H., Gupta, P., Zimba, O., & Gupta, L. (2022). Bibliometric and altmetric analysis of retracted articles on COVID-19. Journal of Korean Medical Science, 37(6).       
https://doi.org/10.3346/jkms.2022.37.e44
Kolahi, J., & Khazaei, S. (2018). Altmetric analysis of contemporary dental literature. British dental journal, 225(1), 68-72. https://doi.org/10.1038/sj.bdj.2018.521
Mansourzadeh, M. J., Ghazimirsaeid, J., Motamedi, N., Najafi, A., Abubakar, A. A., & Dehdarirad, H. (2021). A survey of Iranian retracted publications indexed in PubMed. Iranian Journal of Public Health, 50(1), 188. https://doi.org/10.18502/ijph.v50i1.5086
Mas-Bleda, A., Thelwall, M., Kousha, K., & Aguillo, I. F. (2014). Do highly cited researchers successfully use the social web?. Scientometrics, 101(1), 337-356.      
https://doi.org/10.1007/s11192-014-1345-0
Miller, G. W., Stoner, R., Boissaud‐Cooke, M. A., Lim, J., Furness, H., Putt, O., & Lewis, T. L. (2021). Open access availability of anatomy papers presented at meetings of the American and British Associations of Clinical Anatomists. Clinical Anatomy, 34(5), 660-667.
https://doi.org/10.1002/ca.23580
Momeni, F., Fraser, N., Peters, I., & Mayr, P. (2019). From closed to open access: A case study of flipped journals [Preprint]. arXiv (arXiv:1903.11682).       
https://doi.org/10.48550/arXiv.1903.11682
Moradi, S., & Janavi, E. (2018). A scientometrics study of Iranian retracted papers. Iranian Journal of Information Processing and Management, 33(4), 1789-1808.     
https://jipm.irandoc.ac.ir/article_699497.html?lang=en [In Persian].
Moylan, E. C., & Kowalczuk, M. K. (2016). Why articles are retracted: a retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open, 6(11), e012047.       
https://doi.org/10.1136/bmjopen-2016-012047
Neale, A. V., Dailey, R. K., & Abrams, J. (2010). Analysis of citations to biomedical articles affected by scientific misconduct. Science and engineering ethics, 16(2), 251-261.       
https://doi.org/10.1007/s11948-009-9151-4
Nocera, A. P., Boyd, C. J., Boudreau, H., Hakim, O., & Rais-Bahrami, S. (2019). Examining the correlation between altmetric score and citations in the urology literature. Urology, 134, (December), 45-50. https://doi.org/10.1016/j.urology.2019.09.014
Panahi, S., & Soleimanpour, S. (2021). The landscape of the characteristics, citations, scientific, technological, and altmetrics impacts of retracted papers in hematology. Accountability in Research, 30(7), 363-378. https://doi.org/10.1080/08989621.2021.1990049
Patel, R. B., Vaduganathan, M., Bhatt, D. L., & Bonow, R. O. (2018). Characterizing high-performing articles by Altmetric score in major cardiovascular journals. JAMA Cardiology, 3(12), 1249-1251. https://doi.org/10.1001/jamacardio.2018.3823
Serghiou, S., Marton, R. M., & Ioannidis, J. P. (2021). Media and social media attention to retracted articles according to Altmetric. PLOS One, 16(5), e0248625.       
https://doi.org/10.1371/journal.pone.0248625
Shamsi, A., Lund, B. D., & SeyyedHosseini, S. (2022). Sharing of retracted COVID-19 articles: An altmetric study. Journal of the Medical Library Association, 110(1),  97-102.      
https://doi.org/10.5195/jmla.2022.1269
Stillman, D. (2019). Retracted item notifications with Retraction Watch integration. Zotero [Internet], 14. Retrieved November 20, 2024, from: https://www.zotero.org/blog/retracted-item-notifications/
Tenopir, C., King, D. W., Clarke, M. T., Na, K., & Zhou, X. (2007). Journal reading patterns and preferences of pediatricians. Journal of the Medical Library Association, 95(1), 56-63. https://pmc.ncbi.nlm.nih.gov/articles/PMC1773049/
Van Der Vet, P. E., & Nijveen, H. (2016). Propagation of errors in citation networks: a study involving the entire citation network of a widely cited paper published in, and later retracted from, the journal Nature. Research Integrity and Peer Review, 1(3).      
https://doi.org/10.1186/s41073-016-0008-5
Wager, E., Barbour, V., Yentis, S., & Kleinert, S. (2009-a). Retractions: Guidance from the Committee on Publication Ethics (COPE). Maturitas, 64(2), 201-203.       
https://doi.org/10.1016/j.maturitas.2009.09.018
Wager, E., Barbour, V., Yentis, S., & Kleinert, S. (2009-b). Retractions: Guidance from the Committee on Publication Ethics. Journal of Critical Care, 24(4), 620–622.      
https://doi.org/10.1016/j.jcrc.2009.10.009
Wang, T., Xing, Q. R., Wang, H., & Chen, W. (2019). Retracted publications in the biomedical literature from open access journals. Science and engineering ethics, 25, (March), 855-868. https://doi.org/10.1007/s11948-018-0040-6
Wang, Z., Shi, Q., Zhou, Q., Zhao, S., Hou, R., Lu, S., Gao, Xia., & Chen, Y. (2022). Retracted systematic reviews were continued to be frequently cited: a citation analysis. Journal of Clinical Epidemiology, 149, (September), 137-145.       
https://doi.org/10.1016/j.jclinepi.2022.05.013
Yu, H., Murat, B., Li, L., & Xiao, T. (2021). How accurate are Twitter and Facebook altmetrics data? A comparative content analysis. Scientometrics, 126(5), 4437-4463.      
https://doi.org/10.1007/s11192-021-03954-7

Files

Share

How to cite

Statistics