If we value software, we need "a fundamental sea-change" in academia

CW12Robed.JPGBy Alice Tobin, science writer.

This article by Alice Tobin was fundamental in garnering interest in the research software engineer campaign, which among other things led to our article in the Times Higher Education.

Combining the skills of a scientific researcher and a software developer, the research software engineer is ideally placed to bring scientific software up to scratch. An ongoing discussion that began at the Collaborations Workshop 2012 asks what obstacles need to be removed to clear the way.

Beneath the arched ceilings and elaborate chandeliers of Queen’s College, at the University of Oxford, an odd mix of researchers and software developers congregated last March. The 2012 Collaborations Workshop, organised by the Software Sustainability Institute (the Institute), brought together people from all sorts of backgrounds to discuss the future of scientific software development. And it was here, under the gaze of beautifully robed portraits, that they began to map out a new career path in the difficult terrain of academia.

Developing software for research doesn’t just require the person to be a skilled developer. They also need to have an in-depth knowledge of the science, so they can fully grasp the scientific lingo and the problems they are trying to solve. Some come from a software engineering background and choose to specialise in science, whereas others are researchers at heart who have learnt to develop their own software. While these people can go by many names, the Institute calls them research software engineers.

Software has become increasingly important for scientific research over the past few years, and Neil Chue Hong, Director of the Institute, says that there are few fields left untouched by its mark. From genomics to quantum physics, the demand for custom made software in academia is on the rise, but the numbers of research software engineers has not changed. According to Ilian Todorov, a research software engineer at the Science and Technology Facilities Council (STFC) and one of the speakers at the conference, this now means many are finding themselves overloaded. “There is more work than people, that is the problem.”

An outdated attitude

Research software engineers in academia are forced to live a life of pretence. Under the façade of a scientific researcher, they must publish academic papers to sustain their careers, yet the code they produce counts for very little. As Dirk Gorissen, a research software engineer at the University of Southampton, argues, “It’s all about the publications and grants, and software doesn’t give you points”.

This absurd situation leaves these people with no clear career path, and many are left with little choice but to abandon their job in academia and switch to a career in industry. This is precisely what Gorissen has done, who has left his position in academia in search of greener pastures. Coming from a background in computer science, he laments the state of software in academia, but notes that, while researchers are very diligent in their labs, they do not appear to care about the quality of the software they use.

But why should academic institutions care? Simon Hettrick, policy leader at the Institute, explains that academic research groups currently have two options; “They might develop the software themselves in a throw-away sense, using some terribly badly hung together bit of code, or they might go the opposite end and actually employ a qualified software developer to develop something professional.” Neither option is very satisfactory; the first option is limited by project funding and the researcher is unlikely to have the necessary expertise, whereas the second is extremely expensive and the professional developer is unlikely to have the necessary background scientific knowledge.

The current culture in academia is restrictive. Code is still widely viewed by researchers as a means to an end rather than an end in itself. Hettrick says that researchers tend to be focussed on scientific results, but not how they get those results.

Gorissen reports that a lack of time and funding mean that most researchers just “cobble together a few scripts to get the job done” to the detriment of the software. If researchers took this approach to other aspects of their scientific methodology, their research would be shunned. Yet software often avoids scrutiny, as there is a distinct lack of transparency in a lot of academic code.

Imagine a manufacturing company designed a hammer which could only ever be used once. Not only would it be a terrible waste, it would also cost builders a fortune. Yet this is effectively how researchers treat custom software in science. “They use the software, get the results they need, publish a paper, and that’s it, that software’s forgotten about,” says Hettrick. It is true that sustainable, reusable software is more expensive to develop initially, but he argues that it is a cheaper option in the long run.

Such a slapdash attitude can undermine the quality of research, and some think it is only a matter of time before the cracks begin to show. “It’s my view that there’s a crunch coming,” says James Hetherington, head of the Research Software Development Team at University College London, “where people start to realise that their otherwise good research is based on code which is unreliable and potentially incorrect.”

Carving out a career path

The way forward is far from clear, but everyone agrees on one point. Stuart Rossiter, a research fellow at the University of Southampton, echoes many others when he argues that there has to be a "fundamental sea-change" in the culture of academia.

Some incentive must be provided before researchers can begin to improve their software. “There is enough talent out there,” says Todorov, “It’s about nourishing and harnessing the talent of the people who would like to do it as a job”.

Rossiter argues that if the research councils take the lead, universities should follow, although he accepts the substantial changes required from universities could be problematic. Others disagree, suggesting instead that universities should drive the change. According to Chue Hong, research councils have already done a considerable amount, and the responsibility should instead fall on the shoulders of the top university academics. 

In the same vein, Todorov agrees that there is a need for a top-down approach and it must be brought up to a senior level before research software engineers can be accepted. “It’s about changing the institutional culture so that software engineers find a place within it,” he says, “It’s about how they’re recognised and evaluated, because their outcome is code rather than papers.”

The issue they face is how to convince senior academics that research software engineers deserve more credit. “I think one of the key things that would help is some kind of metric,” says Rossiter, and he isn’t the only one. What they now need, Chue Hong says, is some hard data showing that it is worth the investment before they can present a convincing case.

Exactly how research software engineers should, and can, be incorporated into the existing system has proven a contentious point. While many agree that the ideal situation would be to have just one or two specialised research software engineers working alongside the researchers as an integral part of the group, there is disagreement over whether this is actually possible. Todorov says, “It is now recognised that software development is very expensive, and it is unsustainable to have just one or two people.” Instead, he argues that the creation of this career path can only be achieved through collaboration.

However, Chue Hong says that the EPCC already have such a system in place, with around two research software engineers per group, some staying as long as ten years with the same project. Yet he admits that this is an unusual case, as the centre is large enough to carry on paying their developers during the gaps between projects.

Signs of change

It is not all doom and gloom; there have already been some signs of progress. Take the Research Software Development Team at UCL. This group of research software engineers works closely with researchers at the university to help them develop better software, through projects where they effectively become a temporary member of the research group and a training course for PhD students and postdoctoral researchers. It has received a considerable amount of praise. Todorov remarks, “It’s the first time I’ve seen a university pulling their act together to create something that will try to preserve the value of the software that they develop.”

Rossiter makes the distinction between the changes that can be reasonably expected to occur in the short-term and the long-term. He says that a model similar to the one at UCL is likely to be an excellent model for others in the short-term, but in the long term it may be more complicated. “In the longer term, for [sustainable software development] to be properly embedded in research, you have to change this whole culture of how people are assessed, how people are promoted; effectively the nature of what gives you kudos, what gives you respect, has to change.”

Chue Hong thinks the distinction should instead be made according to the institution’s size. For larger teams of researchers, it should be possible for a couple of researcher software engineers to integrate into the group, but it would not be practical for smaller teams, and therefore a model resembling the one at UCL might be more appropriate.

One thing is certain. Until academia learns to appreciate the work of research software engineers, they will continue to lose some of the most talented members of their projects.
 

Posted by s.hettrick on 14 May 2013 - 1:02pm

Add new comment

The content of this field is kept private and will not be shown publicly.
By submitting this form, you accept the Mollom privacy policy.