June 21, 2016

Breaking the impasse – an effective process to assess Engagement and Impact: IRU Submission

The strong support for measuring the value of university research for end-users has been hindered by the lack of a viable way to do so.  Discussion is caught between the difficulty of ascribing meaning to an array of data sets, many of which are highly subject to manipulation, and the lack of breadth in an assessment of particular case examples.

Previously, in its policy statement ‘Measuring Research Value for End Users’, the IRU argued  that judgment by a panel of relevant people is crucial to creating a plausible rating of performance with the panel working through the data and individual examples put forward.  That proposal still lacked clarity about the focus of the assessment.

At page 15, the Australian Research Council (ARC)’s consultation paper gets to the essence of how to resolve the impasse through its proposal that a “focus on the processes or approaches to impact used by universities may be more appropriate”.  Under this approach, to make an assessment, the panel would consider a set of information about how the university is engaged with end-users as well as evidence of the value of those interactions for the end-users. The inclusion of short vignettes highlighting individual and verifiable impact stories would work as exemplars demonstrating the pathways to impact.

The approach would move away from the researcher-centric attempt to track individual research outputs over a period from idea to application.  If Excellence in Research for Australia (ERA) assesses the value of the research as judged by fellow researchers, an engagement and impact measure should focus on the value for end-users. 

To be meaningful, this engagement and impact measure must rise above a set of data indicators to encompass a knowledge-based judgement of the value of research for end-users.

A number of iterations over different rounds will probably be necessary until an optimal measure is developed.

Overview

The IRU submission to the ARC Engagement and Impact Assessment Consultation Paper covers:

• the purpose of creating an assessment of engagement and impact with end-users;
• the importance of having a focus on end-users in designing the assessment;
• key aspects for the assessment process;
• amendments to the proposed principles; and
• a response to the various questions the Consultation Paper poses.

Full submission attached below (7 pages).