Interdisciplinary Approach in the Survey of Adaptation Level of Citation Indices and Normalized Altmetrics in Scientific Journal Rankings

Document Type : Research Paper

Authors

1 M.Sc., Department of Knowledge and Information Science Stuides, Shiraz Universtiy, Iran

2 Assistant Professor, Department of Knowledge and Information Science Stuides, Shiraz Universtiy, Iran

3 Assistant Professor, Department of Educational Management, Shiraz Universtiy, Iran

Abstract

Purpose: The main objectives of the present study are to estimate the correlation between the ranking based on normalized altmetrics and citation indices, and to conduct interdisciplinary comparisons at the level of scientific journals.
Methodology: With a quantitative approach, this research has used scientific methods and techniques such as analysis of web data, citation analysis, and data-based analysis of indicators.
Findings: The findings indicated that NJAM normalized altmetrics did not have any significant relation with citation indices (i.e., IF, SJR and SNIP). However, NJRM and NJCM normalized altmetrics had a significant positive relationship with citation indices (i.e., IF, SJR, SNIP). In addition, there was a significant correlation between disciplines, in ranking journals based on altmetrics (i.e., JAM, JRM and JCM). However, there was no significant difference between disciplines, in ranking journals based on normalized altmetrics (i.e., NJAM, NJRM and NJCM).
Conclusion: If we accept that various citation indices and altmetrics measure aspects of the quality of scientific journals, the observed significant positive correlations can be explained by referring to the stable quality of journals. Therefore, we can conclude that all altmetrics, except for JAM and NJAM, can be used for ranking scientific journals

Keywords

References

جمالی، ج.، دهقانی، م؛ و افضل آقایی، م. (1393). بررسی کیفیت مجلات حوزه زنان و مامایی در پایگاه ISI و Scopus بر اساس شاخص‌های رتبه‌بندی مجلات. زنان، مامایی و نازایی ایران، 17(108)، 9-20.
http://dx.doi.org/10.22038/ijogi.2014.3175
ستوده، ه.، روایی، م؛ و میرزابیگی، م. (1397). مقایسه فرصت‌های دگرسنجی و تحلیل استنادی در ارزیابی پژوهش. پردازش و مدیریت اطلاعات، 34(1)، 113-138.
سعادت، ر.، شعبانی، الف؛ و عاصمی، ع. (1390). بررسی میزان استناد مقالات ISI web of science به مجلات دسترسی آزاد پایگاه DOAJ در دو حوزه علوم بهداشت و پزشکی و علوم پایه. مدیریت اطلاعات سلامت، 11(18)، 165-175.
سلاجقه، م؛ و دیاری، س. (1395). رابطه بین دگرسنجه‌ها و شاخص‌های استنادی: اسنیپ، رتبه‌بندی نشریات سایماگو، ایگن‌فاکتور و ضریب تأثیر نشریات علوم پزشکی. فصلنامه مطالعات ملی کتابداری و سازمان‌دهی اطلاعات، 27(2)، 167-180.
صالحی، ک؛ و نوروزی، ع. (۱۳۸۵). ارزشیابی ضریب تأثیرگذاری مجله‌های بین‌المللی ایران. اطلاع‌شناسی.
عرفان‌منش، م. (1397). رابطه میان شاخص‌های فعالیت آلتمتریک و کیفیت مجله‌های علم اطلاعات و کتابداری در اسکوپوس. مطالعات ملی کتابداری و سازمان‌دهی اطلاعات، 29(2)، 7-26.
فضایلی، ر. (1387). شاخص‌های ارزیابی مجلات و مقالات علمی از دیدگاه .ISI تولید علم، 9،
معتمدی، ف؛ و رمضانی پاکپور، ف. (1394). بررسی تطبیقی IF و SJR مجلات کتابداری و اطلاع‌رسانی. مجله علم‌سنجی کاسپین، 2(1)، 50-56.
نادربیگی، ف.، اسفندیاری مقدم، ع؛ و سهیلی، ف. (1394). جستاری بر دگرسنجی: سنجه‌ای جدید برای ارزیابی تأثیر بروندادها و تعاملات علمی. مجله علم‌سنجی کاسپین، 2(2)، 55-67.
نوروزی چاکلی، ع.، قضاوی، ر؛ و نورمحمدی، ح. (1394). نرمال‌سازی، ارزش‌گذاری و اعتبارسنجی شاخص‌های ارزیابی عملکرد پژوهشی در علوم پزشکی نسبت به سایر حوزه‌های علمی. مدیریت اطلاعات سلامت، 12(4)، 445-456.
Barbaro, A., Gentili, D., & Rebuffi, C. (2014). Altmetrics as New Indicators of Scientific Impact. Journal of the European Association for Health Information and Libraries, 10 (1), 3-6.
Bar- Ilan, J., & et al. (2013). Altmetrics: Present and Future. In: Proceedings of the 76th ASIS&T Annual Meeting: Beyond the Cloud: Rethinking Information Boundaries 2013: 78.
Bazrafshan, A., Haghdoost, A. A., & Zare, M. (2015). A Comparison of Downloads, Readership and Citations Data for the Journal of Medical Hypotheses and Ideas. Journal of Medical Hypotheses and Ideas, 9(1), 1.
Beall, J. (2015). The “Metric” System: Yet More Chaos in Scholarly Publishing. The journal of physical chemistry letters, 6(11), 2020-2021.
Bornmann, L. (2014). Do Altmetrics Point to the Broader Impact of Research? An Overview of Benefits and Disadvantages of Altmetrics. Journal of Informetrics, 8(4), 895-903.
Bornmann, L., & Haunschild, R. (2016a). How to Normalize Twitter Counts? A First Attempt based on Journals in the Twitter Index. Scientometrics, 1-29.  http://dx.doi.org/10.1007/s11 192 -016-1893-6
Bornmann, L., & Haunschild, R. (2016b). Normalization of Mendeley Reader Impact on the Reader- and Paper-side: A Comparison of the Mean Discipline Normalized Reader Score (MDNRS) with the Mean Normalized Reader Score (MNRS) and Bare Reader Counts. Journal of Informetrics, 10(3), 776–788.
Bornmann, L., & Haunschild, R. (2017). Measuring Field-Normalized Impact of Papers on Specific Societal Groups: An Altmetrics Study Based on Mendeley Data. Research Evaluation, 26(3), 230–241. https://doi.org/10.1093/reseval/rvx005.
Clarivate Analytics. (n.d.). Retrieved 2017, Sep. 21, from http://mjl.clarivate.com
Costas, R., Haustein, S., Zahedi, Z., & Lariviére, V. (2016). Exploring Paths for the Normalization of Altmetrics: Applying the Characteristic Scores and Scales. Retrieved 2ج017, Sep. 18, from http://altmetrics.org/wp-content/uploads/2016/09/altmetrics16_paper _6.pdf
Figa-Talamanca, A. (2007). Strengths and Weaknesses of Citation Indices and Impact Factors. Portland Press.
Hammarfelt, B. (2014). Using Altmetrics for Assessing Research Impact in the Humanities. Scientometrics, 101(2), 1419-1430.
Haunschild, R., & Bornmann, L. (2016). Normalization of Mendeley Reader Counts for Impact Assessment. Journal of Informetrics, 10(1), 62–73. http://doi.org/10.1016/ j.joi.2015.11.003
Haunschild, R., & Bornmann, L. (2017). Normalization of Zero-Inflated Data: An Empirical Analysis of a New Indicator Family. In: Proceedings of ISSI 2017: The 16th International Conference on Scientometrics and Informetrics (pp. 448–459). China: Wuhan University.
Haunschild, R., & Bornmann, L. (2018). Field- and Time-Normalization of Data with Many Zeros: An Empirical Analysis Using Citation and Twitter Data. Scientometrics, 116, 997-1012. https://doi.org/10.1007/s11192-018-2771-1
Haustein, S., Bowman, T. D., & Costas, R. (2015). Interpreting "Altmetrics": Viewing Acts on Social Media Through the Lens of Citation and Social Theories. arXiv preprint arXiv:1502.05701.
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014 a). Coverage and adoption of Altmetrics Sources in the Bibliometric Community. Scientometrics, 1, 468-483.
Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014 b). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of The Association for Information Science and Technology, 65(4), 656-669.
Kyvik, S. (1989). Productivity Differences, Fields of Learning, and Lotka's Law. Scientometrics, 15(3), 205-214.
Lapinski, S., Piwowar, H., & Priem, J. (2013). Riding the Crest of the Altmetrics Wave How Librarians Can Help Prepare Faculty for the Next Generation of Research Impact Metrics. College and Research Libraries News, 74(6), 292-300.
Lariviére, V., Archambault, É., Gingras, Y., & Vignola- Gagné, É. (2006). The Place of Serials in Referencing Practices: Comparing Natural Sciences and Engineering with Social Sciences and Humanities. Journal of the American Society for Information Science and Technology, 57(8), 997-1004.
Loach, T. V., & Evans, T. S. (2015). Ranking Journals Using Altmetrics. In: Proceedings of the ISSI 2015, The 15th International Society of Scientometrics and Informetrics Conference held in Istanbul on 30th June- 2nd July 2015. http://dx.doi.org/10.6084/m9.figshare.1461693
Mas- Bleda, A., Thelwall, M., Kousha, K., & Aguillo, I. F. (2014). Do Highly Cited Researchers Successfully Use the Social Web?. Scientometrics, 101(1), 337-356.
Mazov, N. A., & Gureev, V. N. (2015). Alternative Approaches to Assessing Scientific Results. Herald of the Russian Academy of Sciences, 85(1), 26–32.
Mingers, J., & Leydesdorff, L. (2015). A Review of Theory and Practice in Scientometrics. European Journal of Operational Research (in press). Retrieved 21 Apr. 2017, from: http://arxiv.org/abs/1501.05462.
Mingers, J., & Yang, L. (2017). Evaluating Journal Quality: A Review of Journal Citation Indicators and Ranking in Business and Management. European Journal of Operational Research, 257, 323-337.
Mohammadi, E., & Thelwall, M. (2014). Mendeley Readership Altmetrics for the Social Sciences and Humanities: Research Evaluation and Knowledge Flows. Journal of the Association for Information Science and Technology. Wiley Online Library, 65(8), 138-162.
Mohammadi, E., Thelwall, M., Haustein, S., & Larivière, V. (2015). Who Reads Research Articles? An Altmetrics Analysis of Mendeley User Categories. Journal of the Association for Information Science and Technology, 66(9), 1832-1846. DOI: 10.1002/asi.23286.
Nielsen, F. (2007). Scientific Citations in Wikipedia. First Monday, 12(8), Retrieved 21 Apr. 2017, from: http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/ 1997/1872.
Pajić, D. (2015). On the Stability of Citation-based Journal Rankings. Informetrics, 9, 990-1006.
Pendlebury, D. A. (2009). The Use and Misuse of Journal Metrics and Other Citation Indicators. Archivum Immunologiae et Therapiae Experimentalis, 57(1), 1-11.
Plum Analytics. (2015). Overview: Plum Metrics. Retrieved 2017, Sep. 22, from http://www.plumanalytics.com/metrics.html
Priem, J. (2014). Altmetrics. In: B. Cronin and C. Sugimoto (Eds.), Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact (pp. 263-288). London: MIT Press.
Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New Metrics of Scholarly Impact on the Social Web. First Monday, 15(7).
Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A Manifesto. Retrieved 22 Sep. 2017, from: http://altmetrics.org/manifesto
Rasmussen, P. G. & Andersen J. P. (2013). Altmetrics: An Alternate Perspective on Research Evaluation. Sciecom info, 9(2).
Ringelhan, S., Wollersheim, J., & Welpe, I. M. (2015). I Like, I Cite? Do Facebook Likes Predict the Impact of Scientific Work?. PloS one. (DOI: 10.1371/journal.pone.0134389).
Rodgers, E. P., & Barbrow, S. (2013). A Look at Altmetrics and its Growing Significance to Research Libraries. The University of Michigan University Library. Available at: http://hdl.handle.net/2027.42/99709.
SCImago. (2013). SCImago Journal & Country Rank Spain: SCImago Research Group. Retrieved 2017, Sep. 21, from http://www.scimagojr.com
Sud, P., & Thelwall, M. (2014). Evaluating Altmetrics. Scientometrics, 98(2), 1131-1143.
Thelwall, M. (2017). Three Practical Field Normalised Alternative Indicator Formulae for Research Evaluation. Journal of Informetrics, 11(1), 128–151. https://doi.org/10.1016/ j.joi.2016.12.002.
Torres- Salinas, D., Cabezas- Clavijo, Á., & Jiménez- Contreras, E. (2013). Altmetrics: New Indicators for Scientific Communication in Web 2.0. Comunicar, XXI(41), 53-60.
Trueger, N. S., Thoma, B., Hsu, C. H., Sullivan, D., Peters, L., & Lin, M. (2015). The Altmetric Score: A New Measure for Article-Level Dissemination and Impact. Annals of emergency medicine.
Verbeek, A., Debackere, K., Luwel, M., & Zimmermann, E. (2002). Measuring Progress and Evolution in Science and Technology–I: The Multiple Uses of Bibliometric Indicators. International Journal of Management Reviews, 4(2), 179-211.
Zahedi, Z., Costas, R., & Wouters, P. (2014). How Well Developed are Altmetrics? A Cross-Disciplinary Analysis of the Presence of ‘Alternative Metrics’ in Scientific Publications. Scientometrics, 101(2), 1491-1513.

Files

Share

How to cite

Statistics