Evaluation Indicators of Science, Technology, and Innovation: A Comparative Study of Iran, Australia, and New Zealand

Document Type : Research Paper

Author

Associate Professor, Information and Society Research Department, Iranian Research Institute for Information Science and Technology (IranDoc), Tehran, Iran.

Abstract

purpose: In today’s rapidly evolving global landscape, effectivly evaluating and monitoring research and technology capabilities is crucial for fostering innovation, economic growth, and competitiveness. This study aims to comprehensively analyze and compare of science, technology, and innovation (STI) evaluation indicators across three distinct national contexts: Iran, Australia, and New Zealand. By developing a robust thematic framework, the research provides actionable insights and recommendations for policymakers seeking to strengthen their research and innovation ecosystems. Aligning evaluation methods with global standards is of paramount importance, particularly as nations increasingly compete in the global knowledge economy. Australia and New Zealand were selected as benchmarks due to their leadership in STI evaluation practices. This choice enables a nuanced understanding of how different national contexts shape STI policies and outcomes. Through this comparative approach, the study highlights best practices and contextual adaptations that can inform global policymaking.
Methodology: This study employs a systematic and qualitative methodology to compare science, technology, and innovation (STI) evaluation indicators in Iran, Australia, and New Zealand. Data collection involved library research, document reviews, and analysis of governmental and institutional reports. A total of 445 indicators were identified and categorized into six thematic groups: (1) scientific publications, (2) human capital, (3) technology, (4) budget, (5) institutional environment, and (6) innovation. These categories include various subcategories, such as R&D expenditure, education funding, venture capital investment, patents, and technology transfer, offering a multidimensional perspective on STI evaluation. A thematic analysis was conducted to identify patterns, strengths, and weaknesses within each national system, ensuring a robust comparative framework. By intergrating on both qualitative and quantitative metrics, this methodology highlights the importance of aligning evaluation systems with global best practices while considering local contexts. Furthermore, the study adopts an interdisciplinary approach, incorporating insights from science policy, economics, and innovation studies to enrich the comparative analysis.
Findings: The findings reveal significant differences in the prioritization and implementation of STI evaluation indicators across the three countries. In Iran, the focus is primarily on scientific productivity and human capital development, emphasizing national research outputs and workforce expansion. In contrast, Australia and New Zealand prioritize technology commercialization, venture capital investment, and international collaborations, reflecting their advanced innovation ecosystems and integration into the global markets. The indicators were systematically classified into six overarching categories, with subcategories providing insights into R&D funding, innovative business environments, and knowledge transfer mechanisms. For example, the technology category includes  measures such as patents, industrial designs, and trademarks, while the human capital category examines metrics related to researchers, students, and workforce development. Through thematic analysis, this study identifies both shared practices and unique strategies in the STI evaluation frameworks of  each country. Iran's Science, Technology, and Innovation (STI) evaluation framework demonstrates notable strengths in fostering national research initiatives; however, it faces challenges in integrating these efforts with broader innovation policies. In contrast, the systems of Australia and New Zealand employ a balanced approach, intergrating quantitative metrics with qualitative assessments to comprehensively capture STI activities. The differences among the three countries underscore the influence of governance models, funding structures, and regional priorities on shaping STI evaluation outcomes. This analysis highlights the role of contextual factors—such as economic priorities, governance frameworks, and cultural norms— in shaping the design and effectiveness of STI evaluation systems.
Conclusion: This study contributes a significantly to the field of STI evaluation by providing a comprehensive comparative analysis of three distinct national contexts. The findings highligh the importance of aligning evaluation frameworks with international standards while also adapting them to local needs and priorities. For policymakers, the study offers a roadmap to foster dynamic innovation ecosystems, optimize resource allocation, and enhance global competitiveness. It also underscores the need for flexible evaluation systems capable of adapting to emerging technologies, shifting priorities, and new trends. Furthermore, the research identifies gaps in existing systems and proposes targeted strategies to improve the efficiency and impact of STI initiatives across diverse contexts. The thematic framework developed in the study serves as a valuable tool for other countries seeking to assess and strengthen their Science, Technology, and Innovation (STI) evaluation systems. It emphasizes the importance of balancing quantitative indicators, such as publication outputs and patents, and qualitative measures, such as collaboration quality. Ultimately, accurately assessing and strategically enhancing STI capabilities is critical for maintaining a competitive edge in the global knowledge economy. This research offers practical insights for nations seeking to align innovation policies with global standards while promoting sustainable development and long-term economic resilience. By providing a dynamic and adaptable evaluation framework, this study not only advances academic discourse in science, technology, and innovation (STI) evaluation but also offers practical guidance for practitioners and policymakers around the globe. The findings highlight the importance of robust evaluation systems in driving innovation and ensuring that research efforts translate into meaningful societal and economic outcomes. They also underscore the potential to scale successful practices across different regions while addressing context-specific challenges. As global challenges continue to evolve, this study emphasizes the necessity for continuous refinement and innovation in Science, Technology, and Innovation (STI) evaluation practices to meet the demands of an increasingly interconnected world.

Keywords

Main Subjects

References

 
ستاد راهبری اجرای نقشه جامع علمی کشور. (1394). روند تحولات شاخص‌های علم و فناوری در جمهوری اسلامی ایران (1391-1380)، تهران: شورای عالی انقلاب فرهنگی. https://sccr.ir/Files/6910.pdf
شورای عالی انقلاب فرهنگی. (1389). نقشه جامع علمی کشور. تهران: شورای عالی انقلاب فرهنگی. https://dl3.takbook.com/pdf2/ebook7405[www.takbook.com].pdf
هیئت نظارت و ارزیابی شورای عالی انقلاب فرهنگی. (1382). ارزیابی علم و فناوری در جمهوری اسلامی ایران: اولین ارزیابی کلان. تهران: شورای عالی انقلاب فرهنگی، هیئت نظارت و ارزیابی. https://ketabnak.com/reader/21319
Archibugi, D., & Filippetti, A. (2015). The handbook of global science, technology, and innovation, Wiley.
      https://www.wiley.com/en-us/The+Handbook+of+Global+Science%2C+Technology%2C+and+Innovation-p-9781118739068
Australia’s Department of Industry, Innovation, and Science. (2017). Australian innovation system report-2017. https://www.industry.gov.au/publications/australian-innovation-system-report-2017
Australia’s Department of Industry, Science and Resources. (2022). Innovation Metrics Review. https://www.industry.gov.au/publications/innovation-metrics-review
Australia’s Office of the Chief Scientist. (2014). Benchmarking Australian science, technology, engineering and mathematics. https://www.chiefscientist.gov.au/2014/12/benchmarking-australian-science-technology-engineering-mathematics
Braun, V., & Clarke, V. (2008). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.
      https://www.tandfonline.com/doi/epdf/10.1191/1478088706qp063oa?needAccess=true
Board of Supervision and Assessment of Cultural and Scientific Affairs, the Supreme Council of Cultural Revolution. (2003). Assessment of Science and Technology in the Islamic Republic of Iran: The First Macro-Assessment. https://ketabnak.com/reader/21319 [In Persian].
Braun, V., & Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health, 11(4), 589-597.
      https://doi.org/10.1080/2159676X.2019.1628806
Crespi, G., & Zuniga, P. (2012). Innovation and productivity: evidence from six Latin American countries. World development, 40(2), 273-290.
      https://doi.org/10.1016/j.worlddev.2011.07.010
Callaghan Innovation. (2022). Callaghan Innovation Annual Report 2022.
      https://www.callaghaninnovation.govt.nz/assets/Uploads/Documents/FINAL-Callaghan-Innovation-Annual-Report-2022_23-website-version.pdf
Dodgson, M. (2013). The Oxford handbook of innovation management. Oxford University Press.  https://doi.org/10.1093/oxfordhb/9780199694945.001.0001
Dutta, S., Lanvin, B., Rivera León, L., & Wunsch-Vincent, S. (Eds.). (2023). Global Innovation Index 2023: Innovation in the face of uncertainty. WIPO. wipo.int/edocs/pubdocs/en/wipo-pub-2000-2023-en-main-report-global-innovation-index-2023-16th-edition.pdf
Department of Education. (2023). Australian Curriculum: Mathematics and Science Focus. https://www.education.gov.au/australian-curriculum
Edquist, C. (2010). Systems of innovation perspectives and challenges. African Journal of Science, Technology, Innovation and Development, 2(3), 14-45.
      https://journals.co.za/doi/abs/10.10520/EJC10560
Fathollah-Nejad, A. (2020). The Islamic Republic of Iran four decades on: The 2017/18 protests amid a triple crisis. Doha, Qatar: Brookings Doha Center, April, 9, 2019-2020. https://www.brookings.edu/wp-content/uploads/2020/04/The-Islamic-Republic-of-Iran-Four-Decades-On-English-Web.pdf
Fagerberg, J. (2018). Innovation, economic development and policy: Selected essays. Edward Elgar Publishing.
https://books.google.com/books/about/Innovation_Economic_Development_and_Poli.html?id=CilWDwAAQBAJ
Fagerberg, J., Martin, B. R., & Andersen, E. S. (2013). Innovation studies: Evolution and future challenges. Oxford University Press.
      https://doi.org/10.1093/acprof:oso/9780199686346.001.0001
Freeman, C., & Louçã, F. (2002). As time goes by: From the industrial revolutions to the information revolution. Oxford University Press. https://doi.org/10.1093/0199251053.001.0001
Freeman, C., & Soete, L. (2009). Developing science, technology and innovation indicators: What we can learn from the past. Research policy, 38(4), 583-589.
      https://doi.org/10.1016/j.respol.2009.01.018
Freeman, C. (1987). Technology policy and economic performance: Lessons from Japan. Science Policy Research Unit University of Sussex and Pinter Publishers.  
      https://archive.org/details/technologypolicy00free/page/n5/mode/2up
Godin, B. (2017). Models of innovation: The history of an idea. MIT press.
      https://direct.mit.edu/books/monograph/2281/Models-of-InnovationThe-History-of-an-Idea
Godin, B. (2009). The making of science, technology and innovation policy: conceptual frameworks as narratives, 1945-2005.
      https://espace.inrs.ca/id/eprint/9915/1/Godin_2009_412.pdf
Galindo-Rueda, F. (2019, July). Oslo Manual 2018: Guidelines for Collecting, Reporting and Using Data on Innovation. In National Bureau of Statistics of China, OECD-NBS International Training Workshop on Innovation Statistics. Xi’an, China (pp. 16-18).
      https://www.stats.gov.cn/english/pdf/202010/P020201012342666850167.pdf
Godin, B. (2004). The obsession for competitiveness and its impact on statistics: The construction of high-technology indicators. Research Policy, 33(8), 1217-1229.
      https://doi.org/10.1016/j.respol.2004.07.005
Grupp, H., & Schubert, T. (2010). Review and new evidence on composite innovation indicators for evaluating national performance. Research Policy, 39(1), 67-78.
      https://doi.org/10.1016/j.respol.2009.10.002
Ghazanfari, R., & Aliahmadi, A. (2019). National innovation systems in Iran: challenges and approaches. International Journal of Industrial Engineering and Management Science, 6(1), 1-23. https://www.ijiems.com/article_86814_0f877b2cc198bb536c9326b8847c37de.pdf
Godin, B. (2020). The idea of technological innovation: a brief alternative history. Edward Elgar Publishing. https://doi.org/10.23987/sts.101367
Australian Government. IP Australia. (2023). Australian IP Report 2023.
      https://www.ipaustralia.gov.au/tools-and-reh
Lundvall, B. A. (1992). National systems of innovation: Towards a theory of innovation and interactive learning. Francis Printer.
https://library.oapen.org/bitstream/handle/20.500.12657/31613/626406.pdf?sequence=1#page=102
Lee, K. (2019). The art of economic catch-up: Barriers, detours and leapfrogging in innovation systems. Cambridge University Press. https://www.cambridge.org/core/books/art-of-economic-catchup/680D758F404C9CB321CFB7B5D4DD6D72
LaunchVic. (2023). Victoria welcomes eight new VC funds to back early-stage startups. https://launchvic.org/announcements/victoria-welcomes-eight-new-vc-funds-to-back-early-stage-startups/
Mowery, D. C., & Rosenberg, N. (2000). Paths of innovation: Technological change in 20th-century America. Cambridge University Press.
      https://www.cambridge.org/ae/universitypress/
Nijkamp, P., & Siedschlag, I. (Eds.). (2011). Innovation, growth and competitiveness, Dynamic regions in the knowledge-based world economy. Springer Science & Business Media. https://doi.org/10.1007/978-3-642-14965-8
New Zealand’s Minster of Business, Innovation, and Employment. (2021). The Research, Science and Innovation Report — 2021. https://researchscienceinnovation.nz/
Nelson, R. R. (1982). An evolutionary theory of economic change. Harvard university press. https://archive.org/details/evolutionarytheo0000nels
OECD (2014). Measuring Innovation in Education: A New Perspective. Paris: OECD Publishing. https://www.oecd.org/en/publications.html
OECD. (2005). Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data. OECD Publishing. https://www.oecd.org/en/publications.html
OECD. (2017). OECD Science, Technology and Industry Scoreboard 2017. OECD Publishing. https://www.oecd.org/en/publications.html
OECD. (2021). Science, Technology and Innovation Outlook 2021. OECD Publishing.
      https://doi.org/10.1787/75f79015-en
OECD. (2023). OECD science, technology and innovation outlook 2023: A global perspective. OECD Publishing.  https://www.oecd.org/content/dam/
SCImago. (2021). SCImago Journal & Country Rank.
      https://www.scimagojr.com/countryrank.php
Smith, R. G. (2023). Public Sector Criminological Research: The Australian Institute of Criminology, 1972-2022. Palgrave Macmillan. https://doi.org/10.1007/978-3-031-28356-7
Steering Committee for the Implementation of the Comprehensive scientific Map. (2015). Trends in the Development of Science and Technology Indicators in the Islamic Republic of Iran (2001–2012). Tehran: Supreme Council of the Cultural Revolution.
      https://sccr.ir/Files/6910.pdf [In Persian].
Supreme Council of the Cultural Revolution. (2011) The Comprehensive scientific Map. Tehran: Supreme Council of the Cultural Revolution.
      https://dl3.takbook.com/pdf2/ebook7405[www.takbook.com].pdf  [In Persian].
UNESCO. (2021). UNESCO Science Report: The Race Against Time for Smarter Development. UNESCO Publishing. https://www.unesco.org/reports/science/2021/en/download-report
WIPO. (2021). Global Innovation Index 2021: Tracking Innovation through the COVID-19 Crisis. World Intellectual Property Organization.
      https://www.wipo.int/edocs/pubdocs/en/wipo_pub_gii_2021.pdf
World Bank. (2021). Iran Economic Monitor: Adapting to the New Normal, a protracted pandemic and ongoing sunctions, Spring 2021. World Bank Group, Middle East and North Africa Region.
      https://www.worldbank.org/en/country/iran/publication/iran-economic-monitor-fall-2021

Files

Share

How to cite

Statistics