Harms and Solutions for Evaluating Humanities Research Outputs: A Case Study of Language and Literature

Document Type : Research Paper

Authors

1 Ph.D. Candidate, Department of Knowledge and Information Science, University of Isfahan, Isfahan, Iran.

2 Associate Professor, Department of Knowledge and Information Science, University of Isfahan, Isfahan. Iran .

3 Professor, Department of Persian Literature, University of Isfahan, Isfahan. Iran .

4 Professor, Department of Knowledge and Information Science, University of Isfahan, Isfahan. Iran .

Abstract

Purpose: In addition to the distinction between the field of humanities and other sciences in terms of their nature and research goals, different disciplines within the humanities also possess distinct characteristics. This study aims to identify the disadvantages of evaluating humanities research outputs, with a specific focus on the fields of language and literature. It also aims to provide solutions to improve the evaluation of researchers in these areas.
 
Methodology: The present study is an applied type research with a qualitative content analysis method. The study's statistical population included faculty members from language and literature fields with various Persian and non-Persian orientations (English, Arabic, French, German, and Chinese) in Iran. The research sample consisted of 24 individuals, including 19 men and 5 women who were at least Assistant professors from 8 universities: Isfahan, Tarbiat Modarres, Tehran, Al-Zahra, Shahid Chamran Ahvaz, Shiraz, Allameh Tabatabaei, and Ferdowsi of Mashhad. The participants were selected using targeted sampling and the snowball method. The semi-structured interview was used as a research tool, and MAXQDA 2020 qualitative data analysis software was utilized for coding. Theoretical coding involves three stages: open coding, axial coding, and selective coding. To ensure credibility, the opinions of two coders were sought. Also, the results of the coding were sent to three interviewees who played a significant role in expressing injuries and proposing solutions. They were asked to comment on the compatibility of the extracted codes with their own experiences. The terms and details of the work are explained in detail within the plan for the transferability criterion. In order to measure dependability, all the stages and processes of the research were recorded and reported accurately from the beginning to the end, mentioning the details of the data analysis, and excerpts from the text of the interviews were presented. For confirmability, the findings were provided to 5 interviewees and peers to express their opinions about the results.
 
Findings: Evaluation harms were identified through 62 codes. The concept of science production and the promotion of knowledge foundations were investigated in two general areas (humanities) and a specific area (language and literature) with different components. The general area includes the establishment of policies for evaluating research output, the inclusion of research output ratings in promotion regulations, and considerations of research ethics. The specific area includes the evaluation of research outputs in the promotion regulations, publications, international article publications, theoretical contributions, research style of faculty members, and literary and creative works. The concept of practicality and addressing societal issues also encompasses two aspects: disconnecting the relationship between language and literature fields and society, and teaching language and literature fields. The solutions were presented in the form of 34 codes. The concept of science production and the promotion of knowledge foundations were investigated in two general areas (humanities) and a specific area (language and literature) with different components. The general area includes the development of policies for evaluating research output, the inclusion of research output ratings in promotion regulations, and the improvement of research facilities and resources. The specific area includes the evaluation of research outputs in the promotion regulations, the research approach of faculty members, and research facilities. The concept of practicality and addressing societal issues also encompasses two aspects of applied research: the connection between the disciplines of language and literature and society.
 
Conclusion: If the evaluation is conducted accurately and comprehensively, and it clearly indicates the gap between the current situation and the desired outcome, it will minimize trial and error in this field and reduce costs associated with ineffective and unnecessary tasks. It accelerates the steps towards success and progress. The essence of evaluation is optimizing performance. The competition to obtain the best scientific position among researchers, educational groups, and research institutes has gained great importance and momentum. The health of such a competition and its correct orientation will be possible through planning for accurate evaluation, and a review of the evaluation policy of academic faculty members of universities and higher education is one of the fundamental needs in this direction. The results of the present study can be utilized as an effective measure to enhance the current situation.

Keywords

References

اسلامی اردکانی، ح. (۱۳۹۲). فربهی یا آماس؛ تحلیل ساختار مقالات (نا) علمی- پژوهشی در علوم انسانی. روش‌شناسی علوم انسانی، ۱۹ (۷۴)، 119-91. https://ensani.ir/fa/article/322512
پاریاد، ر. (1387). چالش‌های فراروی تحقیقات در حوزه علوم انسانی. کنگره ملی علوم انسانی، تهران: پژوهشگاه علوم انسانی و مطالعات فرهنگی.
تقی‌زاده کرمان، ن.، حسین‌قلی‌زاده، ر.، و جاویدی کلاته جعفرآبادی، ط. (۱۳۹۴). آسیب‌شناسی تولید دانش در هسته‌های پژوهشی علوم انسانی. راهبرد فرهنگ، ۸ (۳۰)، 188-161. https://www.jsfc.ir/article_14951.html
جمالی مهموئی، ح. (1389). کتاب‌شناسی علم‌سنجی. پژوهشگاه علوم و فناوری اطلاعات ایران. 79 ص.
حسین‌پور، م. (1391). آسیب‌شناسی پژوهشی اعضای هیئت علمی رشته‌های علوم انسانی. مطالعات معرفتی در دانشگاه اسلامی، 16 (50)، 45-64. https://ensani.ir/fa/article/304672/
داورپناه، م.، ر. (1386). چالش‌های علم‌سنجی در حوزه علوم انسانی. مطالعات تربیتی و روا‌ن‌شناسی، 8 (2)، 125-155. https://doi.org/10.22067/IJAP.V8I2.6799
رسولی، ب.، و شهریاری، پ. (۱۴۰۰). چاله‌ها و چالش‌های پژوهش در علوم انسانی در ایران؛ پنجره‌ای به روی سیاست‌گذاری علم. پردازش و مدیریت اطلاعات، ۳۷ (۲)، ۳61-۳33. https://doi.org/10.52547/JIPM.37.2.333
شیخ‌زاده، ح. (1394). علوم انسانی تحول و ارتقاء در راستای منافع ملی ایران. مقاله ارائه‌شده در کنفرانس بین‌المللی رویکردهای نوین در علوم انسانی، مالزی. https://civilica.com/doc/439581/
صفرپور، ا.، عباس‌پور، ج.، و سلیمی، ق. (1400). کاوش تجربیات و دیدگاه‌های اعضای هیئت علمی حوزه علوم انسانی و هنر دانشگاه شیراز در مورد آیین‌نامه ارتقاء. مطالعات دانش‌پژوهی، 1 (1)، 29-54. https://doi.org/ 10.22034/jkrs.2021.47805.1006
صمدی، ل. (1399). تحلیل ضرورت‌های هم‌ترازسازی ارزش تولیدات علمی حوزه‌های تخصصی در علم‌سنجی و آسیب‌شناسی سنجش و ارزیابی همسان: مطالعه موردی حوزه هنر. رهیافت، 30 (77)، 1-24. https://doi.org/ 10.22034/RAHYAFT.2020.13817
صمدی، ل.، ناخدا، م.، نوروزی چاکلی، ع.، و اسدی، س. (1397). استناد و استناددهی در ارزیابی‌های حوزۀ هنری: مسائل و چالش‌ها؛ مطالعه موردی رشته نقاشی. مطالعات کتابداری و علم اطلاعات، 10 (23)، 77-94. https://doi.org/10.22055/slis.2017.18543.1252
صمدی، ل.، ناخدا، م.، نوروزی چاکلی، ع.، و اسدی، س. (1398). چالش‌های آئین‌نامه ارتقای اعضای هیئت علمی برای هم‌ترازسازی سنجش و ارزیابی بهره‌وری علمی پژوهشگران رشته موسیقی در ایران. پژوهشنامه کتابداری و اطلاع‌رسانی، 9 (1)، 59-84. https://doi.org/10.22067/RIIS.V9I1.75959
طایفه‌باقر، د.، اباذری، ز.، مرادی، ش.، و باب‌الحوائجی، ف. (1401). مدل ارزیابی اثربخشی پژوهش‌های علوم انسانی در ایران. پژوهش‌نامه علم‌سنجی، 8 (15)، 1-34. https://doi.org/10.22070/RSCI.2020.13384.1450
قنادی‌نژاد، ف.، و حیدری، غ. (1399). روش‌ها و شاخص‌های ارزیابی تولیدات علمی در علوم انسانی و اجتماعی: مرور نظام‌مند. پژوهش‌نامه علم‌سنجی، 6 (12)، 203-230. https://doi.org/10.22070/RSCI.2020.4998.1341
کورس، سارا. (تابستان1383). ادبیات و جامعه. ترجمة غلامرضا ارجمندی. پیک نور، 6، 13-25. https://ensani.ir/fa/article/192009/
نامداریان، ل.، کلانتری، ن.، و علیدوستی، س. (1396). ارزیابی علم، فناوری و نوآوری: مروری بر شاخص‌ها و سازمان‌های فعال این حوزه. تهران: چاپار. 404 ص. 978-964-7519-73-1
نوروزی چاکلی، ع (1390). آشنایی با علم‌سنجی (مبانی، مفاهیم، روابط و ریشه‌ها). تهران: سمت. 502 ص 978-600-6121-10-9
یاسینی، ع.، نوروزی کوهدشت، ر.، و تابان، م. (۱۳۹۶). آسیب‌شناسی پژوهش‌های دانشگاهی حوزه‌های ادبیات و علوم انسانی، الهیات و معارف اسلامی با روش آمیخته تشریحی. مطالعات معرفتی در دانشگاه اسلامی، ۲۱ (۷۱)، ۲50-۲31. https://www.magiran.com/paper/1741979/
 
Abramo, G., Aksnes, D. W., & D’Angelo, C. A. (2020). Comparison of research performance of Italian and Norwegian professors and universities. Journal of Informetrics, 14(2), 1-23. https://doi.org/10.1016/j.joi.2020.101023
Campbell, D. T. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2(1), 67–90. https://www.sciencedirect.com/science/article/abs/pii/014971897990048X
Cicero, T., & Malgarini, M. (2020). On the use of journal classification in social sciences and humanities: evidence from an Italian database. Scientometrics, 125(2), 1689–1708. https://doi.org/10.1007/s11192-020-03581-8
Course, S. (2000). Literature and society. Translated by Gholamreza Arjamandi. Peyk Noor, 6 (Summer), 13-25. [In Persian]
Davarpanah, M.R. (2007). Scientometric Challenges in Humanities and Social Sciences. Studies in Education and Psychology, 8(2), 125. https://doi.org/10.22067/IJAP.V8I2.6799 [In Persian].
Eslami Ardakani, H. (2013). Fatness or swelling; an analysis of (non)scholarly articles in humanities. Methodology of Social Sciences and Humanities, 19(74-75), 91-118. https://ensani.ir/fa/article/322512 [In Persian]
Gadd, E. (2019). Influencing the changing world of research evaluation. Insights, 32(1), 1–9. https://doi.org/10.1629/uksg.456
Ghanadinezhad, F. & Heidari, G. (2020). Methods and Indicators for the Evaluation of Scientific Production in the Humanities and Social Sciences: A Systematic Review. Scientometric research journal, 6(12), 203-230.  https://doi.org/10.22070/RSCI.2020.4998.1341 [In Persian]
Hosseinpour, M. (2012). The Research Pathology of Faculty Members in Human Sciences. Irainian Journal of The Knowledge Studies in The Islamic University, 16(50), 45. https://ensani.ir/fa/article/304672/ [In Persian]
Jamali Mahmouei, H.R. (2010). Bibliography of scientometrics. Iranian Research Institute for Information Science and Technology (IranDoc). 79 p. [In Persian]
Ku, M.-C. (2019). A Comparative Analysis of English Abstracts and Summaries of Chinese Research Articles in Three Library and Information Science Journals Indexed by the Taiwan Social Science Citation Index. Journal of Library & Information Studies, 17(1), 37–81. https://doi.org/10.6182/jlis.201906_17(1).037
Lin, C.-S. (2018). An analysis of citation functions in the humanities and social sciences research from the perspective of problematic citation analysis assumptions. Scientometrics, 116(2), 797–813. https://doi.org/10.1007/s11192-018-2770-2
Namdarian, L., Kalantari, N. & Alidousti, S. (2017). Evaluation of science, technology and innovation: a review of indicators and active organizations in this field. Tehran: Chapar. 404 p. 978-964-7519-73-1 [In Persian]
Noroozi Chakoli, A. (2011). Introduction to scientometric: foundations, concepts, relations & origins. Tehran: Samt. 502 p.  [In Persian]
Ochsner, M., & Hug, S. E. (2016). Indicators for Research Performance in the Humanities? The Scholars’ View on Research Quality and Indicators. [International Conference on Science and Technology Indicators], Valencia: Spain. https://doi.org/10.13140/RG.2.2.11431.14245
Ochsner, M., Hug, S. E., & Daniel, H.-D. (2014). Setting the stage for the assessment of research quality in the humanities. Consolidating the results of four empirical studies. Z Erziehungswiss, 17(6), 111–132. https://doi.org/10.1007/s11618-014-0576-4
Olmos-Peñuela, J., Benneworth, P., & Castro-Martinez, E. (2014). Are ‘STEM from Mars and SSH from Venus’?: Challenging disciplinary stereotypes of research’s social value. Science and Public Policy, 41(3), 384–400. https://doi.org/10.1093/scipol/sct071
Paryad, R. (2008). Challenges facing research in the field of human sciences. National Congress of Human Sciences, Tehran: Institute for Humanities and Cultural Studies. [In Persian]
Rasuli, B., & Shahriari, P. (2021). Barriers to Research in the Humanities in Iran: a window to Science Policy. Information Sciences & Technology, 37(2), 333-361. https://doi.org/ 10.52547/JIPM.37.2.333 [In Persian]
Safarpour, E., Abbaspour, J., & Salimi, G. (2021). Exploring the Experiences and Views of Shiraz University’s Art and Humanities faculty members on Promotion Regulation. Journal of Knowledge-Research Studies, 1(1), 29-54. https://doi.org/10.22034/jkrs.2021.47805.1006 [In Persian]
Samadi, L. (2020). Analyzing the Necessities of Equating the Value of Scientific Outputs of Specialized Fields in Scientometrics and Pathology of Similar Measurement and Evaluation: Case Study: Field of Art. Rahyaft Journal, 30(77), 1-24. https://doi.org/ 10.22034/RAHYAFT.2020.13817 [In Persian]
Samadi, L., Nakhoda, M., Noroozi Chakoli, A., & Asadi, S. (2018). Citing and citations Issues in the assessments of Art: Challenges and Problems: Case Study Painting. Journal of Studies in Library and Information Science, 10(1), 77-94. https://doi.org/10.22055/slis.2017.18543.1252 [In Persian]
Samadi, L.; Nakhoda, M., Noroozi Chakoli, A. & Asadi, S. (2019). The Challenges of the Regulations for the Promotion of Faculty Members for the Equated Comparison of the Assessment and Evaluation Process of Scientific Productivity of Music Researchers in Iran. Library and Information Science Research, 9(1), 59-84. https://doi.org/ 10.22067/RIIS.V9I1.75959 [In Persian]
Schreibman, S., Siemens, R., & Unsworth, J. (2004). A Companion to Digital Humanities. Oxford: Blackwell. https://doi.org/10.1002/9780470999875
Sheikhzade, H. (2015). Human sciences, transformation and promotion in line with national interests. [Paper presented at the International Conference on New Approaches in Human Sciences, Malaysia]. https://civilica.com/doc/439581/ [In Persian]
Sigurðarson, E. S. (2020). Capacities, capabilities, and the societal impact of the humanities. Research Evaluation, 29(1), 71–76. https://doi.org/10.1093/reseval/rvz031
Sīle, L., Pölönen, J., Sivertsen, G., Guns, R., Engels, T. C. E., Arefiev, P., Dušková, M., Faurbæk, L., Holl, A., Kulczycki, E., Macan, B., Nelhans, G., Petr, M., Pisk, M., Soós, S., Stojanovski, J., Stone, A., Šušol, J., & Teitelbaum, R. (2018). Comprehensiveness of national bibliographic databases for social sciences and humanities: Findings from a European survey. Research Evaluation, 27(4), 310–322. https://doi.org/10.1093/reseval/rvy016
Taqizadeh Kerman, N., Hosseinqolizadeh, R., & Javidi Kalate Ja’farabadi, T. (2015). A Pathological Study of Knowledge Creation in Humanities Research Centers. Strategy for culture, 8(30), 161-188. https://www.jsfc.ir/article_14951.html [In Persian]
Tayefehbagher, D., Abazari, Z., Moradi, S. & Babalhavaeji, F. (2022). A Model for Evaluating the Impact of Humanities Research in Iran. Scientometric research journal, 8(15), 1-34.  https://doi.org/10.22070/RSCI.2020.13384.1450 [In Persian]
Toledo, E. G. (2018). Research assessment in Humanities and Social Sciences in review. Revista española de Documentación Científica, 41(3), 1-14. https://doi.org/10.3989/redc.2018.3.1552
Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Jones, R., Kain, R., Kerridge, S., Thelwall, M., Tinkler, J., Viney, I., Wouters, P., Hill, J. & Johnson, B. (2015). The metric tide: report of the Independent Review of the Role of Metrics in Research Assessment and Management. Higher Education Funding Council for England. https://doi.org/https://doi.org/10.13140/RG.2.1.4929.136
Yang, S., Zheng, M., Yu, Y., & Wolfram, D. (2021). Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities? Journal of Informetrics, 15(1), 1–21. https://doi.org/10.1016/j.joi.2020.101120
Yasini, A., Norouzi Kuhdasht, R., & Tabban, M. (2017). Pathology of Academic Research Using Anatomical Mixed Method (The Case of Literature and Humanities, and Theology Fields). Irainian Journal of The Knowledge Studies in The Islamic University, 21(71), 231-250. https://www.magiran.com/paper/1741979/ [In Persian]

Files

Share

How to cite

Statistics