A research paper points out that some of the major “inconsistencies” in rankings of Indian institutions include over-emphasis on bibliometrics, neglect of teaching quality, lack of methodological transparency, huge fluctuations in rankings and partial assessment of online education.
The study, published by Professor V Ramgopal Rao, former director of IIT Delhi and current vice-chancellor of BITS Pilani campus, aims to suggest ways to increase transparency in the ranking framework.
Union Education Minister Dharmendra Pradhan released the National Institutional Ranking Framework (NIRF) ranking report 2024 on August 12. Higher education institutions in the country.
The study highlights that while the NIRF rankings are designed to promote transparency and accountability, some inconsistencies inherent in the framework raise concerns about its reliability. The study, titled “Revealing inconsistencies in NIRF,” was authored by Rao and Abhishek Singh, also from BITS, Pilani, and published in the journal Current Science.
The research shows that the inconsistencies identified highlight the need for ongoing dialogue and improvements to the ranking framework. “It is important to acknowledge that rankings by their very nature subtly influence perceptions. Therefore, if the identified issues are not addressed, it may affect the credibility and relevance of the NIRF rankings and potentially impact interests such as students, parents and policymakers. Stakeholders’ views.
Challenges faced by NIRF bibliometric methods
According to the study, rankings overemphasize numbers and fail to capture key elements such as relevance, innovation, social impact and contributions beyond traditional publishing. This approach raises concerns about the comprehensive assessment process, particularly in relation to research and professional practice indicators.
“Focusing solely on numbers such as the number of research papers or citations does not reflect an institution’s actual contribution. For example, if a researcher develops a technology that helps improve life in a rural area, the whereabouts of that technology will remain unknown. news18.
“A truly effective assessment of institutional performance should consider a wider range of characteristics to avoid an incomplete and distorted representation of an institution's contribution to the academy. These problems are exacerbated by the bibliometric approach adopted by NIRF, which relies exclusively on commercial repositories Conduct data collection. This reliance reveals shortcomings in the scope, precision, and inclusion of non-traditional research output and therefore risks undermining the diversity of scholarly contributions by overlooking institutions that have made significant contributions in non-traditional ways.
Ignore teaching quality
The study pointed out that the NIRF ranking lacks a specific mechanism to directly evaluate teaching quality and ignores key aspects such as classroom observation, student evaluation and alumni feedback.
“The absence of these assessment methods hinders a comprehensive assessment of teaching effectiveness, resulting in an incomplete picture of a school's educational strength. In addition, the NIRF rankings pay limited attention to practical training elements such as practical projects and internships, leading to a prioritization of experiential learning institutions are undervalued.
Professor Rao said student feedback was crucial in assessing the quality of teaching. “Another important way to assess the quality of teaching is to look at placements, the quality of work students achieve and median salaries,” he said.
Some reviews of online education
The document states that while the online education sub-indicator includes data related to online completion of syllabuses and exams and Swayam, an Indian government initiative to achieve access, equity and quality in education, the assessment specifically focuses on development on the Swayam platform. and number of courses offered. This can be seen in the examples of some institutions with extensive online courses not receiving marks despite providing data on online completion of syllabuses and exams.
“The ranking only covers content hosted on the Swayam platform, which is only possible with IITs as it was developed in this way. This completely ignores the online education offered by other institutions. For example, BITS conducts work-integrated learning programmes. , one of the largest such programs in the country, is not currently considered in the ranking framework.
Insufficient methodological transparency
According to the paper, while NIRF does provide insights into its rating methodology, more thorough and easy-to-understand documentation is needed. To reduce ambiguity and potential misunderstandings, clear and unambiguous definitions of metrics must be established, especially those that capture financial data. Establishing clear and unambiguous rules and standards is critical to ensuring standardized and fair assessment. This proactive initiative not only promotes consistent and fair assessments but also eliminates discrepancies that may arise due to different interpretations.
Limited global benchmarks
The study highlights that a significant limitation of the NIRF rankings is limited global benchmarking. The lack of robust international comparison mechanisms hinders institutions that aspire to global recognition. “In an era of increasing cross-border collaboration and exchange of academic ideas, the NIRF rankings cannot provide a comprehensive assessment that transcends national borders,” the report states.
Singh, a co-author of the paper, said that unlike the global rankings, the Indian rankings are still evolving and are expected to have global influence over the years. “Resetting the parameters and incorporating a more detailed view of the institution is critical. It should also increase the number of stakeholders participating in the survey, including students, alumni and industry,” Singh said.
