Policy research

ResearchFishBy Simon Hettrick, Deputy Director.

Researchfish® allows researchers to record the impact of their research outside of the standard metric of how many papers I have written. These outcomes, as they are called, cover publications, but also collaborations, events, awards and other metrics including - and of most interest to me - software.

Researchfish® was established with the support of MRC and initially focused on collecting outcomes from medical research. It has since been adopted by a broad range of funders, including the UK’s seven Research Councils. I recently had an interesting talk with the EPSRC’s Louise Tillman about what these outcomes might say about research software in the UK and, thanks to her, a week later I found myself in possession of a spreadsheet containing the research outcomes related to software for EPSRC researchers.

Just having the outcomes is pretty exciting, but to make things more interesting, I decided that I would write the analysis code myself. I’m not a software developer, but it’s getting progressively more difficult to stay that way when I spend my life surrounded by Research Software Engineers. Hence this post not only reports an investigation into Researchfish…

Continue Reading

A word cloud of the software used in researchBy Simon Hettrick, Deputy Director.

Over the last couple of years, we’ve had occasion to ask people about the software they use in their research. We’re about to start a long-running survey to collect this information properly, but I thought it might be fun to take a rough look at the data we’ve collected from a few different surveys.

It would be easy to survey people if there existed a super-list of all possible research software from which people could choose. But no such list exists. This raises the question of how many different types of software do we expect to see in research? Hundreds, thousands, more? The lack of this list is rather annoying, because it means we have to collect freeform text rather than ask people to choose from a drop-down list. Free-form text is the bane of anyone who collects survey data, because it takes so much effort to clean. It is truly amazing how many different ways people can find to say the same thing!

I collected together five of our surveys from 2014 to 2016, which relates to 1261 survey participants. From these, we collected 2958 different responses to the question “What software do you use in your research?”, but after a few hours of fairly laborious data cleaning (using Open Refine to make things easier) these were boiled…

Continue Reading

By Simon Hettrick, Deputy Director.

No one knows how much software is used in research. Look around any lab and you’ll see software – both standard and bespoke – being used by all disciplines and seniorities of researchers. Software is clearly fundamental to research, but we can’t prove this without evidence. And this lack of evidence is the reason why we ran a survey of researchers at 15 Russell Group universities to find out about their software use and background.

Headline figures

  • 92% of academics use research software
  • 69% say that their research would not be practical without it
  • 56% develop their own software (worryingly, 21% of those have no training in software development
  • 70% of male researchers develop their own software, and only 30% of female researchers do so

Data

The data collected during this survey is available for download from Zenodo ("S.J. Hettrick et al, UK Research Software Survey 2014"​​, DOI:10.5281/zenodo.14809). It is licensed under a Creative Commons by Attribution licence (attribution to The University of Edinburgh on behalf of the Software Sustainability Institute).

Software is far more important to research than anyone knows

If we do not know how much we rely on software, we cannot ensure that researchers…

Continue Reading

By Simon Hettrick, Deputy Director.

This article originally appeared in Research Fortnight.

With few exceptions, every significant advance in research over at least the past 30 years would have been impossible without computer software. Research software—used to produce results rather than for, say, word processing or web searches—has spread far beyond traditionally computational fields such as particle physics and bioinformatics to achieve near ubiquity in all disciplines. In my role at the Software Sustainability Institute, I have worked with everyone from fusion physicists to choreographers.

The institute, which helps researchers with software and promotes a better understanding thereof, is conducting a survey of researchers selected at random from 15 Russell Group universities. Early indications from about 400 respondents are that almost 90 per cent rely on research software. About 70 per cent report that their research would be impossible without it, and almost 60 per cent develop their own software.

This dependence on software is making it impossible to assess the reliability and reproducibility of results without access to the code used to generate them. Yet researchers often overlook the importance of producing well-written, accessible computer code.

One reason for this might…

Continue Reading

By Simon Hettrick, Deputy Director.

Two weeks ago, we launched a petition to see how many people agreed that software is fundamental to research and, if we overlook this fact, that we lose our ability to make groundbreaking discoveries. How will the petition help make a difference?

The purpose of the petition is to raise awareness of research software. Every person who signs, everyone who retweets and - importantly - everyone who talks about the petition in their groups or offices, helps make more researchers think about their use of software. The more of those people who sign up, the easier it is to persuade research stakeholders that they must change their policies to support software. And that's not just research funders in the UK, research funders in the Australia, Canada, Germany, Netherlands, New Zealand, Poland and the US are currently working with us to develop similar policies in their own countries.

Profile raising might seem a bit flimsy, especially when you consider the fact that the people we are asking to sign up are working on everything from curing cancer to understanding the very workings of the universe. But it's not the only thing we're doing to change the way that the research community deals with software.

Continue Reading

If you agree that software is fundamental to research, we invite you to sign our petition.

Everyone who signs this petition will add weight to our lobbying of research stakeholders, and will help us prove the fundamental role of software in research.

For more information, read the petition - and don't forget to let your friends and colleagues know!

By Simon Hettrick, Deputy Director.

Over the last few months, we’ve been working on improving our understanding of the size of the research software community. In previous posts, I’ve discussed our plans for this research. Although we've not yet finished our analysis, we thought that it would be interesting to release some early results. First of all, how much money do the Research Councils invest into research that relies on software? The answer: at least a third of the entire RCUK budget - or £840 million in 2013.

Summary

The UK Research Councils and Technology Strategy Board (TSB) have been investing, at a minimum, around 30% of their total budget for project grants into software-reliant research, which is £840 million in the financial year 2013-14. We expect the actual investment to be significantly larger than this figure due to the fact that software is rarely discussed in the title or abstract of a grant - data on which this research relies.

Investment per council into software-reliant research is relatively stable, with an average increase of 3.3% over the last four years. Notable exceptions to this rule are the TSB which, despite a significant increase in total research investment, has invested a lower percentage of those funds…

Continue Reading

Study Title: Study to find the size of the research community.

Researcher: Simon Hettrick

Ethics number: ERGO/FPSE/12521

Please read this information carefully before deciding to take part in this research. If you are happy to participate your consent will be implied by completing the survey.

What is the research about?

Very little information has been collected about researchers' use of software. The lack of this information, makes it is difficult to provide the correct resources and training for researchers.

In this study we will ask a small number of questions about a researcher's use of software. We will extrapolate the results from the survey group to understand the size of the research community, the software that is most popular in research and the level of training that researchers have received. We will make this information freely available to funders, universities and other appropriate research stakeholders.

If you are interested in why we are conducting this survey, you can read out blog posts on the subject. The first provides the reasons why we are conducting the research and the second provides updates and discusses the use of random emailing.

Why have I been chosen?

Participants for this survey were chosen from two groups: postdoctoral students at the…

Continue Reading

Study Title: Study to find the size of the research community.

Researcher: Simon Hettrick

Ethics numberERGO/FPSE/ 12000

Please read this information carefully before deciding to take part in this research. If you are happy to participate your consent will be implied by completing the survey.

What is the research about?

Very little information has been collected about researchers' use of software. The lack of this information, makes it is difficult to provide the correct resources and training for researchers.

In this study we will ask a small number of questions about a researcher's use of software. We will extrapolate the results from the survey group to understand the size of the research community, the software that is most popular in research and the level of training that researchers have received. We will make this information freely available to funders, universities and other appropriate research stakeholders.

If you are interested in why we are conducting this survey, you can read out blog posts on the subject. The first provides the reasons why we are conducting the research and the second provides updates and discusses the use of random emailing.

Why have I been chosen?

Participants for this survey were chosen from two…

Continue Reading

By Simon Hettrick, Deputy Director.

In an earlier post, I discussed our plans for investigating the number of researchers who rely on software. We’ve spent the last month looking at the feasibility of some of our ideas. In this post, I’ll present our findings about one of these approaches and some of the problems that we’ve encountered. If you’ve ever wondered what happens when a clueless physicist starts to dabble in social science, then this is the post for you.

First of all, a quick recap. Anecdotally, it seems that the number of researchers who rely on software for their research is – pretty much – everyone. There are few researchers who don’t use software in some way, even when we discount things like word processing and other non-result-generating software. But without a study, we lack the evidence to make this case convincingly. And it’s not just about the size of the community, it’s also about demographics. Seemingly simply questions are unanswerable without knowing the make up of the research software community. How much EPSRC funding is spent on researchers who rely on software? Is that greater, proportionally speaking, than the AHRC?

We had a few ideas about how to determine the size of the research software community. The one that has progressed furthest is the conceptually most straightforward idea: simply ask the research community. This is…

Continue Reading
Subscribe to Policy research