نوع مقاله : مقاله پژوهشی
نویسندگان
1 دانشجوی کارشناسی ارشد علم اطلاعات و دانش‌شناسی؛ دانشکده علوم تربیتی و روان‌شناسی، دانشگاه شیراز، شیراز، ایران.
2 دکتری علم اطلاعات و دانش‌شناسی؛ استاد؛ دانشکده علوم تربیتی و روان‌شناسی؛ دانشگاه شیراز، شیراز، ایران.
3 دکتری علم اطلاعات و دانش‌شناسی؛ استادیار؛ گروه پژوهشی سنجش علم و فناوری؛ موسسه استنادی و پایش علم و فناوری جهان اسلام (ISC)، شیراز، ایران.
چکیده
کلیدواژه‌ها
موضوعات
عنوان مقاله [English]
نویسندگان [English]
Purpose: This study aims to evaluate university ranking systems by analyzing the attitudes of their social stakeholders. It mined opinions expressed in tweets about universities and analyzed their correlations with universities’ performance scores. The rationale lies in the importance of the evaluation of the academic performance of higher education institutions, which is essential, yet challenging, due to the lack of universally accepted standards. For decades, global university rankings have been striving to develop methodologies to measure academic performance. However, they face criticism, for example, for overestimating certain features, underestimating or ignoring others, and bias due to reliance on surveys or citation databases. Consequently, their results are sometimes considered unrealistic and misleading. Achieving realistic ranking results is crucial as they can trigger a "ranking for ranking" phenomenon, causing individuals and institutions to focus more on ranking criteria than on fulfilling their primary missions and service quality. This could lead universities to deviate from their main missions due to unreasonable external evaluations. Therefore, it is necessary to assess the quality of ranking system results. Studies on ranking systems have mostly focused on the correlations of their results. They also dealt with the methodological similarities between the world ranking systems. The findings confirmed partial similarities in both results and methodologies. However, given the commonalities in methodologies, evaluating results based on external benchmarks is needed. As universities aim to meet the needs and interests of their stakeholders to improve their societal standing and ensure survival, the attitudes of stakeholders can serve as a desirable benchmark. Social media provides a platform for the general public to discuss and comment on different subjects, including university capabilities, services, and activities. Thus, they offer opportunities to analyze the opinions of universities stakeholders worldwide.
Methodology: The study used a quantitative content analysis method with opinion mining and altmetric approaches. It focused on a sample of universities ranked in the Times Higher Education (THE) and THE Impact Ranking systems. A distinct, specific name was an inclusion criterion to ensure precise searches on X (formerly Twitter). A sample of 355 universities ranked in THE in 2019-2021 was identified, and their coverage in THE Impact ranking was checked, leading to 174 universities ranked in both systems. Tweets about these universities were extracted using Mozdeh Big Data Text Analysis, and opinion scores were calculated using the SentiStrength opinion mining tool. The tweet data was collected through an extensive search on Twitter from January 3, 2022, to August 8, 2022. Given the non-normality of the data distributions, Spearman correlation analysis was used to analyze the data.
Findings: The research findings indicated weak-to-moderate significant correlations between the frequency of tweets about universities and their overall scores in THE and THE Impact ranking systems, as well as their dimension scores in THE. Tweet counts showed the strongest correlation with the overall score in THE Impact and the weakest with industry income in THE. A strong correlation between total tweet counts and tweet counts in opinion polarities was observed, suggesting that an increase in a university's performance score corresponds with an increase in expressed opinions on Twitter and vice versa. This finding held for both total scores in THE and THE Impact ranking systems and the latter’s dimension scores. Additionally, the overall THE score correlated directly with positive opinion strength, indicating that better performance in the United Nations Sustainable Development Goals (SDGs) led to more positive opinion scores in tweets. However, no significant correlation was found between performance scores in THE Impact and negative opinions in tweets. Meanwhile, overall performance scores in THE correlated with both positive and negative opinion strengths.
Conclusion: The results imply that while social views about universities strongly align with their performances in sustainable development goals, they partially align with performances in THE at the overall level and in specific dimensions. Thus, THE ranking system and social attention may have a complementary relationship, providing a better and deeper understanding of university performance. It is, therefore, possible to improve ranking systems by incorporating social surveys. However, given the challenges of altmetrics, extensive research is needed to pave the way for the practical application of this metric in evaluations. This research also contributes to the altmetrics literature by reaffirming the difference between quantitative and content-based approaches in altmetric studies.
کلیدواژه‌ها [English]
)