Study of the Statistical Relationship Between the Iranian Journals Citation Index (JCI) and Some Scientometric Indices in JCR 2020

Document Type : Research Paper

Authors

Assistant Professor, Department of Information Management, Islamic World Science & Technology Monitoring and Citation Institute (ISC), Shiraz, Iran.

Abstract

Purpose: The main objective of this research is to rank Iranian-indexed journals in the Journal Citation Reports (JCR) 2020 based on the Journal Citation Index (JCI). Additionally, the study aims to investigate the significant correlation between the JCI and other metrics such as Eigenfactor, Article Influence Score (AIS), Quartile, and Journal Impact Factor (JIF). Among them, forty Iranian journals are indexed in three main citation indexes and have received Journal Impact Factor (JIF) and Quartile rankings for the first time, which also reported their Journal Citation Index (JCI). However, the ranking of these Iranian journals based on the new JCI is not specified. More importantly, the study does not examine the significant correlation between JCI and other indices such as Eigenfactor, AIS, Quartile, and JIF. Therefore, this research aims to rank Iranian journals indexed in JCR 2020 based on the Journal Citation Indicator (JCI) and investigate the significant correlation of this index with Eigenfactor, Article Influence Score (AIS), Quartile, and Journal Impact Factor (JIF). The main focus of this research is to determine whether changes in the value of the new index (JCI) will correspond to changes in the old indices (Eigenfactor, AIS, Quartile, and JIF). Therefore, this article aims to investigate the mentioned issues and provide scientific and accurate answers.
 
Methodology: The present research is applied and has been conducted using scientometric indices and an analytical approach. The required data for the research was extracted from JCR 2020. All Iranian journals in JCR 2020 with an Impact Factor (IF) and Quartile, totaling forty journals, formed the research population. In this article, five research questions have been posed. Iranian journals and their corresponding indices were extracted from JCR 2020. They were then ranked and reported in a table based on JCI to address the first question. To address the second to fifth research questions, which investigate the significant relationship between JCI and Eigenfactor, AIS, Quartile, and JIF, the Shapiro-Wilk test was initially used to assess the normality of the distribution of the relevant data for Eigenfactor, AIS, Quartile, and JIF. The results of the Shapiro-Wilk test indicated that the data related to Eigenfactor and AIS followed a normal distribution, while the data related to JIF did not. Therefore, to investigate the significant relationship between JCI and Eigenfactor and AIS, the Pearson correlation coefficient was used. Additionally, to study the significant relationship between JCI and JIF, Spearman's rho correlation coefficient was employed. Additionally, when examining the relationship between JCI and Quartile, it is important to note that Quartile only has four levels (Q1, Q2, Q3, and Q4). Therefore, the Kendall's tau b test was used. This test is used when the data under investigation is on an ordinal scale.
 
Findings: "Iranian Journal of Fuzzy Systems," "International Journal of Health Policy and Management," and "BioImpacts" ranked first, second, and third, respectively, in JCI with scores of 65.1, 15.1, and 99.0. A positive and significant correlation between JCI and Eigenfactor (R=44.0, P<0.006) indicates a statistical relationship. There is a positive and significant correlation between JCI and Article Influence Score (AIS) (R=73.0, P<0.0001), suggesting that an increase in JCI is associated with an increase in AIS. There is a negative and significant correlation between JCI and Q (R=-64.0, P<0.0001), implying that an increase in JCI is associated with a decrease in Q. In summary, the results suggest that JCI is positively related to Eigenfactor and AIS but negatively related to Q. These findings can have implications for evaluating the impact and influence of academic journals.
 Conclusion: The JCI can be a complementary indicator with the JIF. Many indicators have been introduced to cover JIF's weaknesses, but JIF is still the most famous indicator. The passage of time will show that the JCI will also suffer the fate of the previous indicators or, as a new and unique indicator, will remove the authority and acceptance of the JIF after half a century.

Keywords

References

جمالی، ج.، دهقانی، م.، و افضل آقایی، م. (1393). بررسی کیفیت مجلات حوزه‏ زنان و مامایی در پایگاه ISI و Scopus بر اساس شاخص‏های رتبه‏بندی مجلات. مجله زنان، مامایی و نازایی ایران، 17 (108)، 9-20.  https://doi.org/10.22038/ijogi.2014.3175
ستوده،‌ ه. (1389). گذری بر جی.آی.اف.  مجله‌ها و دلایل ناکارآمدی آن در ارزیابی پژوهش در رشته‌های مختلف. رهیافت، 20 (47)، 33-43. https://dorl.net/dor/20.1001.1.10272690.1389.20.47.3.6
فلاح‏زاده، ع.، ابراهیمی، س.، و سلیمی، ق. (1399). سنجش سطح انطباق شاخص‌های استنادی و سنجه‌های جایگزین نرمال‌شده در رتبه‌بندی مجلات علمی. پژوهش‌نامه علم‌سنجی، 6 (1)، 99-126. https://doi.org/10.22070/rsci.2019.4147.1270
مطلبی، د.، حاجی الهیار، ن.، و حریری، ن. (1399). ارزیابی مقایسه‌ای کارکرد شاخص‌های جی.آی.اف.  (IF) IPP، SJR، SNIP در معرفی مجلات معتبر حوزه‌های علوم اجتماعی، فنی‌‌ـ‌ مهندسی و پزشکی. پژوهش‌نامه علم‌سنجی، 6 (1)، 195‌-‌222. https://doi.org/10.22070/rsci.2019.3257.1191
 
Ali, M. J. (2021a). Questioning the impact of the impact factor. A brief review and future directions. Seminars in Ophthalmology, 37(1), 91-96. https://doi.org/10.1080/08820538.2021.1922713  
Ali, M. J. (2021b). Impact factor under attack! Are the criticisms justified? Indian Journal of Ophthalmology, 69(4), 790. https://doi.org/10.4103%2Fijo.IJO_281_21
Bornmann, L. (2020). How can citation impact in bibliometrics be normalized? A new approach combining citing-side normalization and citation percentiles. Quantitative Science Studies, 1(4), 1553-1569. https://doi.org/10.1007/s11192-019-03071-6 .
Bornmann, L., & Haunschild, R. (2016). Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator. Journal of Informetrics, 10(3), 875-887. https://doi.org/10.1016/j.joi.2016.07.002
………………. ., & Mutz, R. (2011). Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field normalization. Journal of Informetrics, 1(5), 228-230. http://‌doi.org/10.1016%2Fj.joi.2010.10.009
…………….. ., Tekles, A., & Leydesdorff, L. (2019). How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators. Scientometrics, 119(2), 1187-1205. https://doi.org/10.1007/s11192-019-03071-6
 Brody, S. (1995). Impact factor as the best operational measure of medical journals. Lancet, 346(8985), 1300-1301. https://doi.org/10.1016/s0140-6736(95)91904-x
Clarivate. (2022). Web of Science Journal Citation Reports 2022 Release Updates. Available at: https://clarivate.com/webofsciencegroup/wp-content/uploads/sites/2/2021/05/JCR-2021-Changes_May21.pdf  (accessed 20 July 2022)
Davis, P. (2021). Journal Citation Indicator. Just another Tool in Clarivate’s Metrics Toolbox? Available at: https://scholarlykitchen.sspnet.org/2021/05/24/journal-citation-indicator/ (accessed 25 June 2021)
De Bellis, N. (2009). Bibliometrics and citation analysis: from the science citation index to cybermetrics. United States of America, Scarecrow Press. https://doi.org/10.12775/TSB.2012.009
Fllahzadeh, A., Ebrahimi, S., & Salimi, G. (2020). Interdisciplinary Approach in the Survey of Adaptation Level of Citation Indices and Normalized Altmetrics in Scientific Journal Rankings. Scientometrics Research Journal, 6(1), 99-126. https://doi.org/10.22070/rsci.2019.4147.1270 [In Persian].
Garcia, M. (2021). For the 2021 Journal Citation Reports, a new visual experience. Available at: https://clarivate.com/blog/for-the-2021-journal-citation-reports-a-new-visual-experience/ (accessed 18 June 2021)
Garfield, E. (1955). Citation indexes for science. Science, 122(3159), 108-111. https://doi.org/10.1126/science.122.3159.108
............................. (1956). Citation indexes: new paths to scientific knowledge. The Chemical Bulletin, 43(4), 11. http://www.garfield.library.upenn.edu/papers/31.html
.................................. (1972). Citation analysis as a tool in journal evaluation. Science, 178(4060), 471-479. https://doi.org/10.1126/science.178.4060.471
..................................... (1998). Long-term vs. short-term journal impact: does it matter? Scientist, 12(3), 11-12. https://garfield.library.upenn.edu/commentaries/tsv12(03)p10y19980202.pdf
.......................................... (1999). Journal impact factor: a brief review. Cmaj, 161(8), 979-980.
https://www.cmaj.ca/content/161/8/979
 
Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., & Rafols, I. (2015). Bibliometrics: the Leiden Manifesto for research metrics. Nature News, 520(7548), 429. https://doi.org/10.1038/520429a
Hubbard, S. (2020). What’s next for JCR: defining ‘Early Access? Available at: https://clarivate.com/blog/whats-next-for-jcr-defining-early-access/ (accessed 2 June 2021)
Huh, S. (2021). The Journal Citation Indicator has arrived for Emerging Sources Citation Index journals, including the Journal of Educational Evaluation for Health Professions, in June 2021. Journal of Educational Evaluation for Health Professions, 18(20), 1-4. https://doi.org/10.3352/jeehp.2021.18.20
Jamali, J., Dehghani, M., & AfzalAghaee, M. (2014). Quality of Journal of obstetrics and Gynecology in ISI and Scopus databases based on indices of ranking journals. The Iranian Journal of Obstetrics, Gynecology and Infertility, 17(108), 9-20. https://doi.org/10.22038/ijogi.2014.3175 [In Persian].
Kamat, P. V. (2021). 2020 Citation Analysis of Energy Journals. ACS Energy Letters, 6(8), 2969-2970. https://doi.org/10.1021/acsenergylett.1c01510
Kamath, P. S., & Bologna, G. (2009). Impact factor: Misused and overhyped? Hepatology, 49 (6), 1787-1789. https://doi.org/10.1002/hep.23040 
Lippi, G. (2009). The impact factor for evaluating scientists: the good, the bad, and the ugly. Clinical chemistry and laboratory medicine, 47(12), 1585-1586. https://doi.org/10.1515/CCLM.2009.352
MacRoberts, M. H., & MacRoberts, B. R. (2010). Problems of citation analysis: A study of uncited and seldom‐cited influences. Journal of the American Society for Information Science and Technology, 61(1), 1-12. https://doi.org/10.1002/asi.21228
Matlabi, D., Haji Allahyari, N., &. Hariri, N. (2020). A comparative Evaluation of the Impact Factor Indicators Function of IF, IPP, SJR, and SNIP in Introducing valid journals in social sciences, engineering, engineering, and medicine fields. Scientometrics Research Journal, 6(1), 195-212.  https://doi.org/10.22070/rsci.2019.3257.1191[In Persian]. 
Mingers, J., & Leydesdorff, L. (2015). A review of theory and practice in scientometrics. European journal of operational research, 246(1), 1-19. https://doi.org/10.1016/j.ejor.2015.04.002
Moed, H., Burger, W., Frankfort, J., & Van Raan, A. (1985). The application of bibliometric indicators: important field-and time-dependent factors to be considered. Scientometrics, 8(3-4), 177-203. https://doi.org/10.1007/bf02016935
O’Grady, C. (2021). Journal impact factor gets a sibling that adjusts for the scientific field. Scientific Community, Available at: https://www.sciencemag.org/news/2021/06/journal-impact-factor-gets-sibling-adjusts-scientific-field (accessed 2 July 2021)
Potter, R. W., Szomszor, M., & Adams, J. (2020). Interpreting CNCIs on a country-scale: The effect of domestic and international collaboration type. Journal of Informetrics, 14(4), 101075. https://doi.org/10.1016/j.joi.2020.101075
Rogers, G., Szomszor, M., & Adams, J. (2020). Sample size in bibliometric analysis. Scientometrics, 125(1), 777-794. doi: https://doi.org/10.1007/s11192-020-03647-7
Sotudeh, H. (2010). The effect of magazines on the impact of research on different fields of research and its inefficiencies. Rahyaft, 20(47), 33-43. https://dorl.net/dor/20.1001.1.10272690.1389.20.47.3.6 [In Persian].
Torres-Salinas, Daniel and Valderrama-Baca, Pilar and Arroyo-Machado, Wenceslao. (2022). Is there a need for a new journal metric? Correlations between JCR Impact Factor metrics and the Journal Citation Indicator—JCI. Journal of Informetrics, 16 (3), 1-11. https://doi.org/10.1016/j.joi.2022.101315
Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365-391. https://doi.org/10.1016/j.joi.2016.02.007
Wang, D., Song, C., & Barabási, A. L. (2013). Quantifying long-term scientific impact. Science, 342(6154), 127-132. https://doi.org/10.1126/science.1237825
Wang, X., & Zhang, Z. (2020). Improving the reliability of short-term citation impact indicators by taking into account the correlation between short- and long-term citation impact. Journal of Informetrics, 14(2), 101019. https://doi.org/10.1016/j.joi.2020.101019
Zitt, M., & Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, 59(11), 1856-1860. https://doi.org/10.1002/asi.20880

Files

Share

How to cite

Statistics