Cross-posted from the Higher Education Policy Institute (HEPI) website.

James Cleaver, Gemma Derrick and Simon Hettrick take a look at the potential implications for research assessment, were more open research practices adopted and a wider range of research outputs recognised.

It is pleasing to see a broad consensus that REF2021 has encouraged steps to be taken in the direction of more Open Research, and that the Future Research Assessment Programme has been open-minded about what the future of the REF could look like. And there is plenty more that could and should be done in all areas of Open Research, especially to create a more equitable and diverse research culture. To take one example, an opportunity for future iterations of the REF would be to consider how to assess the contributions and improve the recognition of the large number of supporting roles behind the researchers whose publications are assessed for the REF. This includes the likes of lab technicians, research managers, research software engineers, facilities managers, data stewards, lived-experience contributors and many more roles that are vital to research but overlooked by the academic system. 

Open Research is a broad term that includes all the ways it is now possible to open up and share more of the research process publicly, with the aim of improving visibility, quality, transparency, impact and trust in all elements of the research process. Whilst of course Open Access to research publications is a crucial part of this and has been the focus of policy objectives to date, Open Research also encompasses open data and materials, open software and code, open models of peer review, and a drive for a more diverse and equitable research culture. However, in terms of the REF it is also important to be transparent and open as to how research is done – who contributes what aspect and why – as this is also part of the Open Research agenda and research’s accountability to the public.

Academia has long held publications as the only metric of research success and this culture is evident in the domination of the REF results by publications. The question is whether this is an accurate portrayal of the actual work being conducted within universities. Outputs submitted to ‘Publications’ and ‘Books’ made up 94 per cent of all outputs submitted to the 2008 RAE (the precursor to the REF). This increased to 97 per cent with REF2014 and is now 98 per cent with REF2021. This leaves a mere two per cent of outputs to represent every other possible research activity conducted in the highly diverse UK research community. 

It is evidently a distortion of the reality within the UK’s universities. Software is a prime example: around 70 per cent of research across all disciplines is underpinned by software, yet of the 191,000 outputs submitted in total to REF2014, only 38 software outputs were registered (this level of detail is not yet available for REF2021). 

Further, it is still rare for those in ‘non-REFable’ roles to be named on research outputs assessed by the REF. Current REF rules require only those with a contractual requirement to research above 30 per cent of their time to be ‘REFable’.  However, since credit is still linked to authorship on publications or books, this overlooks a large proportion of research contribution. Potential solutions to this issue already exist, such as CRediT, a standardised taxonomy that can be used to represent the roles played by contributors to research outputs, including data curation, project administration, and provision of resources – parts often played by those ‘behind the scenes’. Whilst some publishers have implemented this taxonomy for research journal articles, universities and institutions could be encouraged to adopt CRediT or create a system based on this approach. It would be good if future REF exercises could embrace a broader Open Research agenda by extending acknowledgement to these roles that, although not strictly academic, still contribute to the environment that made this research possible.

The Hidden REF

Beyond publications, even though the REF guidelines allow a wide range of output types to be submitted for assessment, the often-conservative approach of universities, concerned by the pressures and risks around obtaining funding, mean that the REF has barely scraped the surface in understanding and recognising the contributions of these ‘hidden’ people. And yet these roles are vital – without them, it would harm the ability for the UK’s universities to conduct the high-quality research that has been celebrated most recently by REF2021. It is this oversight that has spurred the development of the grassroots organization, The Hidden REF. 

Formed by a committee of volunteers, The Hidden REF established a competition that allowed those individuals and groups in hidden, ‘non-REFable’ roles to be nominated by their colleagues for recognition for the crucial work they do in numerous crowd-sourced categories. All submissions and supporting evidence were assessed by panels of volunteers drawn from across the research community and after challenging deliberations and discussions, winners and highly commended runners-up were chosen and invited to an online awards ceremony in September 2021, whilst all those nominated received a certificate in recognition of the work they do. 

There are plans for the Hidden REF competition to run again in 2023, and the initiative welcomes supporters from universities and other research institutes, funding bodies, learned societies and scholarly publishers.

The Hidden REF is just one example of many that draws attention to how a large exercise like the REF could positively change academic culture and push forward different elements of Open Research. Ultimately, if the REF was successful in demonstrating the quality of the work and providing the credit and visibility for all those involved in research, we would not need initiatives like The Hidden REF to exist. It is important that this issue is developed in the implementation of the Research and Development People and Culture Strategy, which has already recognised that an additional 400,000 people are needed in an array of different roles to develop the UK’s research sector. Creating an environment where all those contributing to research are recognised for it officially would surely make those roles more attractive and encourage talented individuals to apply for and thrive in them.