October 3, 2025

IRU Feedback on “A new approach to research evaluation: The ARC’s Research Insights Capability”

The Innovative Research Universities (IRU) has provided feedback on the Consultation Draft – New Australian Research Council Research Insights Capability.

The IRU supports:

• The ARC taking a greater role in improving the understanding of the research ecosystem and highlighting strengths, the ARC having a role to examine the “excellence, depth and impact” of Australian research, and this being enshrined in the ARC Act. The ARC is an ideal position to analyse, publish and link national datasets to provide public understanding of research strengths and benefits. With a more sophisticated approach, the Research Insights Capability could map and align the output and capacity of Australian research with national priorities, Socio-Economic Objectives (SEO), industry needs, locations and scale.
• The intention for light touch reporting (not ranking, scoring or rating institutions), combining sector data with relevant government data, alongside increased linking of publicly available data. We also support the proposed focus on responsible research assessment approaches and on FAIR data principles and practices. Detail on how these will be implemented should be designed in close consultation with the research sector, including the content and process for annual assessment.
• The role of the ARC in improving public confidence (excellence and return on investment through impact) and quality improvement. The targeted insights and tailored reports will need to be dynamic and have value to beneficiaries/partners and potential users. To support this, we recommend an explicit process to draw on lessons learned from the previous Engagement and Impact Assessment exercise, and from the Pathways to Impact programs run by individual universities.
• Using higher education data already collected by institutions and facilitating its sharing or use by the ARC. The sharing of institutional data could also improve internal research evaluation and strategy by increasing the ability to use sectoral (or sub-sector/field of research) benchmarks for appropriate research performance evaluation and improvement.
• Data-driven research evaluation and dynamic real-time insights. These are possible with existing data sets for traditional research outputs, especially if the ARC encourages greater uptake of persistent identifiers of research (e.g. Digital Object Identifiers, DOIs) and researchers (e.g. ORCID).

Our areas of critical feedback or concern are:

• The goal of identifying needs and future opportunities needs to be aligned with the data the ARC has available. Non-traditional outputs (and inputs) will be required to measure impact and collecting this data may prove challenging. The data sought by the ARC is mostly lagged, rather than leading indicators. These indicators may not be sufficient to drive the behaviours needed (ie. encouraging and supporting researchers to focus on future impacts).
• Despite the ARC’s intentions, evaluation reports will likely be used to publicly rank, compare or benchmark universities at single points in time. The ARC will need to consider if additional metadata may need to be published alongside the research data to enable interpretation between institutions (e.g. scaling factors for students, staff numbers or size; context for mission-based compacts). To encourage improvement, the exercises and reports should provide baseline and longitudinal comparisons for institutions “themselves” so they can track and aim for intra-institutional improvements over time.
• Understanding the research ecosystem and highlighting strengths needs to extend beyond the higher education sector and traditional research outputs, including government, hospital, business and other research organisations, and non-traditional outputs. It may also need to consider strengths (and weaknesses) within (and across) fields of research, such as research funding and workforce issues. There should be a balance between commercial/economic impacts and broader social and public good impacts.
• The role of Higher Degree by Research (HDR) training, which is unbalanced across fields and in decline for domestic candidates, requires attention. The ‘data dashboards’ component of ARC Research Insights Capability could be an essential tool for aligning HDR candidate positions/opportunities with industry-based projects, as well as demonstrating the contribution of HDR training to the broader system. Implementation of a DOI for HDR projects would be in line with the Australian National Persistent Identifier (PID) Strategy.
• The research ecosystem also includes State and Territory-based research priorities and funding opportunities, which are highly variable, and extensive international collaboration.We recommend that the proposed State of the Research Environment report include regularly-updated assessments of relevant global trends and their implications for the Australian research system.
• Research evaluation, impact and responding to government and sector needs are live discussions in the Strategic Examination of R&D (SERD), draft National Health and Medical Research Strategy and the Australian Universities Accord recommendations, especially the role of mission-based compacts (including for research). The role of the ARC Research Insights Capability needs to align with these concurrent reviews. Currently TEQSA’s assessment of research for Australian University (re)registration is the main standard that universities are accountable for. The ARC needs to be explicit in how the Research Insights Capability reports will feed into TEQSA’s assessment. More immediately, the ARC Research Insights Capability should directly inform the SERD discussions and decisions on Focus Areas, which should be evidence-based.