A Model for Evaluating the Impact of Humanities Research in Iran

Document Type : Research Paper

Authors

1 PhD Student in Knowledge and Information Science, Islamic Azad University, Tehran North Branch.

2 Associate Professor, Department of Knowledge and Information Science, Islamic Azad University, Tehran North Branch.

3 Assistant professor, Policy Evaluation and Science, Technology and Innovation Monitoring Department, National Research Institute for Science Policy (NRISP).

4 . Associate Professor, Department of Knowledge and Information Science, Islamic Azad University, Science and Research Branch.

Abstract



Purpose: This study aimes to develop and present a model for evaluating the impact of humanities research in Iran.
Methodology:  In the present study, the mixed method approach, which is a combination of quantitative and qualitative methods, was used. The research was conducted in two phases. The first phase used the Meta-Synthesis method to identify the dimensions and indicators of the impact of humanities research. In the second phase, the goal was to validate the indicators from the perspective of experts and a model was developed via the fuzzy Delphi method. The statistical population in the first phase included 58 documents related to measuring and evaluating the impact of humanities research, and in the second phase, it was experts in the field of research assessment based on sampling.
Findings:  Using seven Meta-Synthesis steps, 6 dimensions, 25 criteria and 123 indicators were identified as model elements for evaluating the impact of humanities research. Fuzzy Delphi was performed in two stages to validate the model elements at the level of dimensions, criteria and indicators and the opinions of the experts were applied. Analyzing the results of the fuzzy Delphi demonstrated that the experts agreed on all dimensions, criteria and indicators, while eight indicators scored very low and were thus removed at this stage. Accordingly, a multidimensional model was designed. Analyzing the results of the fuzzy Delphi demonstrated the policy dimension had the highest, and the social and cultural dimensions had the lowest level of agreement among others. Prioritizing the criteria, in the scientific dimension, “scientific contribution" was the most important followed by, "education" in the dimension of capacity, "innovation" in the economic dimension, "social justice" in the social and cultural, "production of goods and services" in the performance, and "demand" in the policy dimensions were the most important criteria, respectively.
Conclusion: This multidimensional model provides an efficient coherent system for evaluating the impact of research in the field of humanities for policymakers and researchers, and facilitates decision making regarding scientific investments.

Keywords

References

بابااکبری ساری، امیر، قهرمانی، محمد، فتحی اجارگاه، کوروش و موتمنی، علیرضا. (1395). ارائه الگوی ارزشیابی اثرات پژوهش‌های مدیریتی. پژوهش‌های مدیریت در ایران، ۲۱ (۱)، ۹۳-۱۱۹.
حبیبی، آرش، ایزدیار، صدیقه و سرافرازی، اعظم. (1393). تصمیم‌گیری چندمعیاره فازی، رشت، کتیبه گیل.
خسروی، مریم و پورنقی، رویا. (1398). ابعاد اثرگذاری پژوهش: مطالعه مرور سیستماتیک، پژوهش‌نامه علم‌سنجی، 5 (1).203-224.
ساکی، رضا. (1385 ). تفکر نظام‌گرا؛ پیشنیاز توسعه پژوهش در حوزه علوم انسانی. کنگره ملی علوم انسانی. تهران: پژوهشگاه علوم انسانی و مطالعات فرهنگی.
ستوده، هاجر، روایی، معصومه و میرزابیگی، مهدیه. (1397). مقایسه فرصت‌های دگرسنجی و تحلیل استنادی در ارزیابی پژوهش. پژوهش‌نامه پردازش و مدیریت اطلاعات، ۳۴ (۱)، ۱۱۳-۱۳۸.
عابدی جعفری، عابد و امیری، مجتبی. (1398). فراترکیب، روشی برای سنتز مطالعات کیفی. روش‌شناسی علوم انسانی، 25 (99)، 73-87.
عرب، سید محمد، ابراهیم‌زاده پزشکی، رضا و مروتی شریف‌آبادی، علی. (1393). طراحی مدل فراترکیب عوامل مؤثر بر طلاق با مرور نظام‌مند مطالعه‌های پیشین. مجله اپیدمیولوژی ایران، 10 (4)، 10-22.
فیض‌آبادی، منصوره، فهیم‌نیا، فاطمه، نقشینه، نادر، توفیقی، شهرام و موسوی جراحی، علیرضا. (1395). مروری بر شاخص‌های سنجش اثر در پژوهش‌های پزشکی. مدیریت اطلاعات سلامت. 13 (6). 432-437.
قاسمی، علی‌اصغر و امامی میبدی، راضیه. (1394). نقش و جایگاه مطالعات میان‌رشته‌ای در رشد و توسعه علوم انسانی کشور. فصلنامه مطالعات میان‌رشته‌ای در علوم انسانی. 7 (4). 1-19.
قنادی‌نژاد، فرزانه و حیدری، غلامرضا. (1399). روش‌ها و شاخص‌های ارزیابی تولیدات علمی در علوم انسانی و اجتماعی: مرور نظام‌مند، پژوهش‌نامه علم‌سنجی (زودآیند).
لطیفی، سمیه، راحلی، حسین، یادآور، حسین، سعدی، حشمت‌اله و شهرستانی، سید علی. (1397). شناسایی و تبیین مراحل اجرایی توسعه کشاورزی حفاظتی در ایران با رویکرد دلفی فازی. مهندسی بیوسیستم ایران، 49 (1), 107-12.
محمدی، علی و شجاعی، پیام. (1395). ارائه مدل جامع مؤلفه‌های مدیریت ریسک زنجیره تأمین: رویکرد فراترکیب. پژوهش‌نامه مدیریت اجرائی، 8 (15)، 112-93.
منتظر، غلامعلی و جعفری، نیلوفر. (1387). استفاده از روش دلفی فازی برای تعیین سیاست‌های مالیاتی کشور. پژوهش‌های رشد و توسعه پایدار (پژوهش‌های اقتصادی)، ۸ (۱) ، ۹۱-۱۱۴.
Cheng, C. H., & Lin, Y. (2002). Evaluating the best main battle tank using fuzzy decision theory with linguistic criteria evaluation. European journal of operational research, 142(1), 174-186.
Chen, K. H., Tang, M. C., Wang, C. M., & Hsiang, J. (2015). Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan. Scientometrics, 102(1), 97-112
Cohen, G., Schroeder, J., Newson, R., King, L., Rychetnik, L., Milat, A. J., ... & Chapman, S. (2015). Does health intervention research have real world policy and practice impacts: testing a new impact assessment tool. Health research policy and systems, 13(1), 3.
Finfgeld, D. L. (2003). Metasynthesis: The state of the art—so far. Qualitative health research, 13(7), 893-904.
Hicks, D. (2004). The four literatures of social science. In Handbook of quantitative science and technology research (pp. 473-496). Springer, Dordrecht
Hsu, J. W., & Huang, D. W. (2011). Correlation between impact and collaboration. Scientometrics, 86(2), 317-324.
Hug, S. E., Ochsner, M., & Daniel, H. D. (2014). A framework to explore and develop criteria for assessing research quality in the humanities. Int'l J. Educ. L. & Pol'y, 10, 55.‏
Ishikawa, A., Amagasa, M., Shiga, T., Tomizawa, G., Tatsuta, R., & Mieno, H. (1993). The max-min Delphi method and fuzzy Delphi method via fuzzy integration. Fuzzy sets and systems, 55(3), 241-253
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational researcher, 33(7), 14-26.
Joly, P. B., Gaunand, A., Colinet, L., Larédo, P., Lemarié, S., & Matt, M. (2015). ASIRPA: A comprehensive theory-based approach to assessing the societal impacts of a research organization. Research Evaluation, 24(4), 440-453.
Keeney, S., Hasson, F., & McKenna, H. P. (2001). A critical review of the Delphi technique as a research methodology for nursing. International journal of nursing studies, 38(2), 195-200.
Klautzer, L., Hanney, S., Nason, E., Rubin, J., Grant, J., & Wooding, S. (2011). Assessing policy and practice impacts of social science research: the application of the Payback Framework to assess the Future of Work programme. Research Evaluation, 20(3), 201-209.
Kulczycki, E., Engels, T. C. E., Polonen, J., Bruun, K., Duskova, M., Guns, R., et al. (2018). Publication patterns in the social sciences and humanities: evidence from eight European countries. Scientometrics 116, 463–486. doi: 10.1007/s11192-018-2711-0
Morton, S. (2015). Progressing research impact assessment: A ‘contributions’ approach. Research Evaluation, 24(4), 405-419.‏
Nederhof, A. J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81-100.‏
Newson, R., King, L., Rychetnik, L., Bauman, A. E., Redman, S., Milat, A. J., Schroeder, J., Cohen, G., & Chapman, S. (2015). A mixed methods study of the factors that influence whether intervention research has policy and practice impacts: perceptions of Australian researchers. BMJ open, 5(7), 1-14.
Ochsner, M.; Hug, S. E.; Daniel, H. D. (2012). Four types of research in the humanities: Setting the stage for research quality criteria in the humanities. Research Evaluation, 22(2), 79– 92.
Ochsner, M., Hug, S. E., & Daniel, H. D. (2014). Setting the stage for the assessment of research quality in the humanities. Consolidating the results of Four empirical studies. Zeitschrift für Erziehungswissenschaft, 17(6), 111-132.
Ochsner, M., Hug, S., & Galleron, I. (2017). The future of research assessment in the humanities: bottom-up assessment procedures. Palgrave Communications, 3(1), 1-12.
Pedersen, D. B., Grønvad, J. F., & Hvidtfeldt, R. (2020). Methods for mapping the impact of social sciences and humanities—A literature review. Research Evaluation, 29(1), 4-21.‏
Penfield, T., Baker, M. J., Scoble, R., & Wykes, M. C. (2014). Assessment, evaluations, and definitions of research impact: A review. Research evaluation, 23(1), 21-32
Puuska, H. M., Muhonen, R., & Leino, Y. (2014). International and domestic co-publishing and their citation impact in different disciplines. Scientometrics, 98(2), 823-839.
Reale, E., Avramov, D., Canhial, K., Donovan, C., Flecha, R., Holm, P., & Primeri, E. (2018). A review of literature on evaluating the scientific, social and political impact of social sciences and humanities research. Research Evaluation, 27(4), 298-308.
Sandelowski, M., & Barroso, J. (2007). Handbook for synthesizing qualitative research. New York: Springer Publishing Company.
Sivertsen, G., & Larsen, B. (2012). Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: An empirical analysis of the potential. Scientometrics, 91(2), 567-575.
Shewchuk, S., & Cooper, A. (2018). Research Impact, the ‘New Academic Capital’: An Environmental Scan of Research Impact Indicators and Resources for the Humanities and Social Sciences across 32 Countries.  Journal of Social Sciences, 14, 55-64.
Toledo, E. G. (2018). Research assessment in Humanities and Social Sciences in review. Revista española de Documentación Científica, 41(3), 1-14.
 

Files

Share

How to cite

Statistics