Researchers both rely on software - and overlook it

Posted by s.hettrick on 31 October 2014 - 9:24am

By Simon Hettrick, Deputy Director.

This article originally appeared in Research Fortnight.

With few exceptions, every significant advance in research over at least the past 30 years would have been impossible without computer software. Research software—used to produce results rather than for, say, word processing or web searches—has spread far beyond traditionally computational fields such as particle physics and bioinformatics to achieve near ubiquity in all disciplines. In my role at the Software Sustainability Institute, I have worked with everyone from fusion physicists to choreographers.

The institute, which helps researchers with software and promotes a better understanding thereof, is conducting a survey of researchers selected at random from 15 Russell Group universities. Early indications from about 400 respondents are that almost 90 per cent rely on research software. About 70 per cent report that their research would be impossible without it, and almost 60 per cent develop their own software.This dependence on software is making it impossible to assess the reliability and reproducibility of results without access to the code used to generate them. Yet researchers often overlook the importance of producing well-written, accessible computer code.

One reason for this might be software’s intangibility—out of sight, out of mind. In one sense, such an attitude is perfectly logical. Academics are judged on their papers, not their code. As a consequence, time-pressed researchers have few incentives for good practice and, too often, software development is left as an unresourced problem to be solved by junior team members, who tend to be untrained and transient. This is not the way to develop reliable software.

We found that in the 2013-14 financial year, the UK research councils and Innovate UK spent at least £840 million on software-reliant research. That’s a huge sum, but it’s only a third of the overall research budget and so significantly less than the software use reported by researchers. Software, it seems, has become such a ubiquitous part of the research process that providing for its development and maintenance is usually simply not mentioned in funding calls.

As the growth of software use in research seems unstoppable, a change in academic practice is inevitable. Things are already on the move: in the UK, a growing number of researchers and research software engineers, as well as our institute, are helping to promote better software practices.

At the institute, for example, we have taken a number of successful practical steps such as providing training and helping researchers with software development problems. The research councils are also looking at ways to support research software development, for example through the Engineering and Physical Sciences Research Council’s Collaborative Computational Projects and the Biotechnology and Biological Sciences Research Council’s Tools and Resources Development Fund.

But there is still a need to raise awareness of the issues around software use. We at the institute believe that all researchers need a rudimentary knowledge of software development, but we would not argue that all researchers should become coders. The obvious solution is to hire people with the skills that are needed.

Many academics, however, believe that they cannot obtain funding for software developers. The result is that software developers are hidden as postdoctoral researchers. This locks them into a catch-22: despite being postdocs in name only, their work is judged against expected postdoctoral duties. Many of the research councils have argued that software developers are an allowable expense on a grant, but researchers have yet to take this on board.

Faced with the need to raise the profile of software among researchers, the institute has decided to follow in the footsteps of other revolutionaries and raise a petition: "We must accept that software is fundamental to research, or we will lose our ability to make groundbreaking discoveries."

Rather than describing the incremental changes that will lead to better software practices, we have concentrated on the emotive issue of what is at risk. We hope that this message will resonate with researchers and encourage them to reassess their work with software.

Researchers cannot continue to simultaneously rely on and overlook software. Although the UK leads the world in adapting to the requirements of modern, software-reliant research, there are still many changes to be made. The first step is to question your own use of software: if you find that you could not do your research without it, then it’s time to sign our petition.