Event Summary

This post was first published on the Research Software Alliance website.

Join one of our upcoming webinars to discuss community-supported routes for convening the first-ever international research software conference in 2025/26, or provide feedback asynchronously on the options paper, Towards an international research software conference (version 2).

ReSA is engaging with key stakeholders to identify and recommend possible options, with three conference options proposed currently. These all share a common overarching aim of community building but differ in the communities they focus on.

The next stage of the development of these ideas is to gain feedback through open public consultation. This includes identifying stakeholders who would be interested in shepherding the development of at least one of these ideas in 2025/26, including possible pilots in 2025.

How to join

Anyone can provide feedback or ask questions through the following methods:

• Responding to this brief questionnaire on preferred options and potential involvement
• Attending an open webinar. Five webinars are available for anyone to join, each with the same agenda of discussion on this document, to engage multiple time zones:
  • 31 Oct 2024, 20:00-21:00 UTC (see in your time zone)
  • 1 Nov 2024, 09:00-10:00 UTC (see in your time zone)
  • 4 Nov 2024, 21:00-22:00 UTC (see in your time zone)
  • 5 Nov 2024, 05:00-6:00 UTC (see in your time zone)
  • 5 Nov 2024, 09:00-10:00 UTC (see in your time zone)

Zoom link, Meeting ID: 218 787 3236, Passcode: 446688

• Commenting directly on Towards an international research software conference (version 2).
• Emailing questions or comments to info@researchsoft.org.

The closing date for input and feedback is Friday 15 November, 2024.

This blog was originally posted on the Research Software Alliance website and has been cross-posted by the Netherlands eScience Center, and US-RSE. The publication can be accessed on Zenodo (DOI 10.5281/zenodo.13350747).

Abstract

This position paper provides a statement on the criticality of research software in artificial intelligence (AI)-driven research and makes recommendations for stakeholders on how to consider research software in their AI goals. This is needed to ensure that the focus on technological infrastructure to support AI acceleration includes research software and its personnel as a vital part of that infrastructure. This paper discusses both research software that supports generative AI, which is now being explored today as a tool to enable new research, as well as more traditional machine learning, which has demonstrated impact in research in most disciplines (particularly in the last decade).

This paper begins by providing definitions for key terms and demonstrating the importance of research software in AI-driven research. The paper then explores the need for AI strategies to recognise research software as key building blocks of AI-driven research, and to include this element alongside a focus on computing systems, data, and models. The fact that AI is dependent on software (i.e., data preparation and model training are performed by software, models are implemented in software) is not always adequately considered. Alongside this high level need for incorporation of research software in AI strategies, the challenges inherent in software dependencies also need consideration, as research software requires continual maintenance, updating, bug fixes, etc. Consequently, the broader challenges in enabling ongoing support for research software used for any type of research are equally relevant to AI-driven research and presented here.

There is also a need to better support the people who develop and maintain the research software that enables AI-driven research. Because human skills, training, and career paths often feature in AI strategies, it is imperative that stakeholders facilitate practices that support and recognise research software personnel. And while the research community is exploring how to leverage AI to improve how research software is developed and maintained, this is also affecting how research software practices should be taught. The paper also illustrates how some countries are operationalising AI strategies that could support the critical element of research software, and have the potential to do so by building on existing investments in research software.

This paper was written as a collaboration between the Research Software Alliance (ReSA) and the Digital Research Alliance of Canada, one of ReSA’s Founding Members. ReSA is a global organisation that unites decision-makers and influencers across the international research software community.

The Digital Research Alliance of Canada is funded by the Government of Canada to serve Canadian researchers, including the infrastructure and activities required for research software. Other community leaders also provided inputs from institutional and open source perspectives.

1. Introduction

This position paper provides a statement on the criticality of research software in artificial intelligence (AI)-driven research. This is needed to ensure that the focus on technological infrastructure to support AI acceleration includes research software - a vital part of that infrastructure. This paper begins by providing definitions for key terms, demonstrating the importance of research software in AI-driven research, and explaining why this paper was developed. The paper then explores the need for AI strategies to recognise research software as a key building block of the AI-driven research, and to include this element alongside a focus on computing systems, data, and models. To do this, evidence on the issues in supporting research software, including open-source scientific software, are presented.

The paper then explores how to better support the people who develop and maintain the research software in AI-driven research. AI strategies often consider people elements such as skills and career paths, which also reflect broader issues for personnel. The paper then illustrates how some countries are operationalising AI strategies that could support the critical element of research software, and have the potential to do so by building on existing investments in research software. Finally, the paper makes recommendations for stakeholders to consider research software in their AI strategies.

2. What is research software and why is it critical to AI?

Research software is defined in accordance with the FAIR for Research Software Principles (Barker et al. 2022) as “source code files, algorithms, scripts, computational workflows and executables that were created during the research process or for a research purpose. Software components (e.g., operating systems, libraries, dependencies, packages, scripts, etc.) that are used for research but were not created during or with a clear research intent should be considered software in research and not research software. This differentiation may vary between disciplines” (Gruenpeter et al. 2021).

Research software supports AI, both 1) in more traditional machine learning (ML), where models are trained on data and then used to infer knowledge about new data, and 2) in generative AI, which can be defined as “deep-learning models that can generate high quality text, images, and other content based on the data they were trained on” (Martineau 2021).

Research software is already recognised as critical to research outcomes (Barker, Katz, and Gonzalez-Beltran 2020; Hocquet et al. 2024; Strasser et al. 2022). Research software can be a research output in itself, and its many functions range from being a component of research instruments, being the research instrument (where research software generates research data, validates research data, or tests hypotheses, such as models and simulations), analysing and presenting research data, and providing infrastructure or underlying tools (Nieuwpoort and Katz 2024). All machine learning (ML) and AI began as research software. In many cases AI research software has transitioned outside of research and is widely used outside research today.

Consequently, research software is essential in AI-based research, where newly developed methods are research software. Some AI reports and strategies recognise this; for example, OECD’s Artificial Intelligence in Science report (OECD 2023) highlights the need to better support research software as part of utilising AI to accelerate scientific productivity, in a number of ways, including:

• Increase access to software
• Share best practices and applications of research software
• Facilitate more regular funding and more secure positions
• Improve ways to measure research impact and productivity
• Consider academic training changes needed
• Utilise disciplinary consortiums to accelerate discovery and improve reproducibility through sharing of software
• Understand how AI can speed up research software development

The research community personnel who develop and maintain research software are just as important as the research software itself, if not more important, as the software will stop working without ongoing maintenance. This maintenance is needed 1) to respond to bugs so that the software continues to be correct, 2) to add new features so that the software continues to be useful as research progresses, 3) to adapt to change in underlying software and hardware or in related software ecosystems, such as changed libraries and computing systems, so that the software continues to work.

The people who do this work can have many titles, including researchers, research software engineers (RSEs), data scientists, computer scientists, data engineers, bioinformaticians, students, community scientists, and many more (Hettrick et al. 2022; Barker and Buchhorn 2022). The inclusion of improved support for the staff who develop and maintain research software is also emphasised in the UK-focused Review of Digital Research Infrastructure Requirements for AI (Lazauskas et al. 2022). This report’s findings included emphasis that “any investment in any infrastructure for AI would need to be matched by investments in training and support”, with the second highest priority areas identified as funding for RSEs (Lazauskas et al. 2022). The UK’s ExCALIBUR RSE Knowledge Integration Landscape Review also highlights the need for RSEs to acquire new skills relevant to AI, and notes that “These skill sets also begin to deviate from what is demanded by industries, requiring novel AI software and capabilities'' (Parsons et al. 2021).

ReSA’s initial research on recognition of research software in the AI landscape was presented in March 2024 at its Research Software Funders Forum meetings. This forum engages representatives from over 60 funding organisations in online meetings and hybrid venues (ReSA 2024a). Attendees of these March meetings were immediately able to utilise these early findings from the presentation to strengthen their own cases for investment in research software within AI-dominated programs, where the criticality of this building block is not yet understood by many in their organisations. It became clear that further research on best practices would support not only funders, but a variety of stakeholders in the international research software community.

The research in this position paper was undertaken as a partnership between ReSA and the Digital Research Alliance of Canada, one of ReSA’s Founding Members. One of ReSA’s key functions is to ensure that research software is considered in international discussions on how to advance research capabilities. ReSA is a global organisation that unites decision-makers and influencers across the international research software community. The Digital Research Alliance of Canada is a non-profit organisation funded by the Government of Canada. It advances Canada’s position as a leader in the knowledge economy on the international stage by integrating, championing and funding the infrastructure and activities required, with research software as one of the three main areas. Other community leaders were also brought in on this paper in order to present more balanced perspectives on the need for research software in AI, particularly from institutional and open source perspectives.

3. Consequences if research software is not supported

There is a tendency for AI initiatives to minimise or exclude focus on research software. This minimisation creates challenges. To show this, evidence on the issues in supporting research software, including open-source scientific software, is also presented here.

Many AI initiatives tend to focus on computing systems, data, and models, considering issues such as availability of systems that can train models and use them for inference in reasonable time periods; FAIRness of underlying data as well as its suitability for training models, including regularity and labelling; understanding and reducing bias in data and models; and understanding privacy. The fact that AI models are implemented in software, and that software is required for training, is rarely considered. Basically, models are thought of as extensions of data, not of software as well. Consequently, many AI strategies focus on data and models, but exclude the underlying software. An example of an AI strategy that does not necessarily adequately include a research software focus include the US National Institutes of Health (NIH) workshop, Towards an ethical framework for AI in biomedical and behavioural research, which focused on data and models, but not the research software inherent in using the data and models (ODSS 2024).

This challenge is not limited to research software. Discussions of AI capacity often omit open source software, instead focusing on areas such as talent, funding, data, semiconductors, and compute access (Engler 2021). Like research software, open source software can advance science, but also has significant effects in other areas such as development of AI standards (Engler 2021). Further, research software is often either open source itself or depends on open source software. But considerations for software from AI initiatives, if any, do not normally include open source software maintenance as a priority, nor analysis of its dependency on other software (Nahar et al. Forthcoming). Where dependency analysis does happen, it is usually due to concerns about supply chain security or perceived attack vectors (a pathway or method used to access a network or computer to attempt to exploit system vulnerabilities).

The nature of open source software as a complex stack, and the concomitant extra step of funding those open source software dependencies of the research software necessary for AI is largely ignored. This could reflect a misunderstanding of how open source dependencies introduce systemic vulnerabilities that need to be addressed and maintained on an ongoing basis, or concerns regarding funding many independent projects down the dependency tree. When open source software is used without planning for future community or dependency support, the result is a more brittle system in the long run – for research software, open source software, and AI.

There has been some recent work done on software bills of materials (SBOMs) in open source supply chains, where dependencies are charted out, in part so that their vulnerabilities can be understood (National Security Agency 2024; SCAWG 2022) (and it is relevant to note that research software is often used to map these dependencies). Mapping supply chains is increasingly important for understanding how to shore up digital infrastructure, where large economies depend upon open source software that is poorly maintained and that may be internationally broken or hijacked, or may break (as in the XZ Utils, Heartbleed, or the Log4j incidents (Goodin 2024; Buchanan 2021), or the more recent CrowdStrike failure which caused a global Windows outage (Robins-Early 2024)). However, SBOMs are merely an atlas for understanding dependencies – they are policy agnostic and their creation doesn’t mandate funding or supporting important open source or research software.

This paper suggests that infrastructural issues with using and supporting open source software are similar or identical to issues with using research software. Where AI is involved, the same questions of continued support for the research software being used to develop, train, and run the AI systems are raised. Any AI initiative that plans to use global infrastructure, to adapt to new fields and target areas, or which hopes to exist for the long haul must consider the research software that enables those goals.

4. Challenges if research software personnel are not supported

There is also a need to better support the people who develop and maintain research software that is important to AI-driven research. This is recognised in relation to research software personnel in general, particularly in relation to the training, hiring, and funding of both professional research and technical staff able to reuse, develop, and maintain sustainable research software; appropriate reward and recognition measures that enable career progression for all people involved in the creation and maintenance of research software; and citation practices for research software that recognise substantial contributors to all aspects of the software (Van Tuyl 2023; US-RSE Association and IEEE Computer Society 2023; Barker and Katz 2022a; ReSA 2023). For example, Software and skills for research computing in the UK recommendations include facilitating detailed analysis of how to professionalise RSE roles; and collaboration between government, funders, and employers to create national policies aimed at improving standards of employment (Barker et al. 2024).

Whilst many of the issues for research software in general are the same for research software to support AI-driven research, one area of difference is in the specifics of skills and training. Human skills, training, and career paths often feature in AI strategies, and it is important that these focus on relevant research software practices. For example, the Review of Digital Research Infrastructure Requirements for AI specifically emphasises the lack of AI skills and training, and career paths for research personnel, including for RSEs. It highlights that these staff often lack formal training, and ongoing professional development is crucial in the fast-paced world of AI tools and techniques (Lazauskas et al. 2022). With many researchers now using GenAI for coding tasks (Nordling 2023, cited in Hosseini et al. 2024), software development skills need to change to reflect this (Caballar 2024; Dursi 2024).

5. National approaches

AI is now seen as a geopolitical asset, and as international organisations and a range of countries seek to show leadership for AI in science, some countries are operationalising AI strategies that could support the critical element of research software. However, more focus is needed. One example of discussing software within a national AI strategy is the US National Artificial Intelligence Research Resource (NAIRR). NAIRR aims to provide a shared national research infrastructure for responsible discovery and innovation in AI, to address the fact that many researchers lack the necessary access to the computing, data, software and educational resources needed. The NAIRR pilot’s four operational focus areas include one area dedicated to software, to “facilitate and investigate interoperable use of AI software, platforms, tools and services for NAIRR pilot resources” (NSF 2024). However, there are potential challenges with this approach as the focus is on interoperability of existing software, which assumes that sustainable software is already in place.

Many countries also have existing investments in research software (Barker and Katz 2022b; ReSA 2024b) that national initiatives have the potential to build on. The UK’s AI Research Resource (AIRR) is another example of a national initiative, which focuses on increasing computational power to support AI-driven research (UKRI 2023), although its funding is now uncertain (Trueman 2024). Whilst AIRR does not seem to include a focus on research software, AIRR is a key component of UK Research and Innovation (UKRI)’s Digital Research Infrastructure (DRI) which has featured a number of recent investments in research software, including funding of research technical professionals (RTPs), such as RSEs. For example, the Engineering and Physical Sciences Research Council (EPSRC) and UKRI DRI have invested £16 million to support community-driven projects providing training and development for RTPs (UKRI 2024a). UKRI’s Digital RTP Skills NetworkPlus aims to explore key challenges and interventions related to skills and careers that are faced by digital RTP communities across the UK research and innovation landscape (UKRI 2024b).

Canada has been a leader in AI, as the first country in the world to put in place in 2017 a fully-funded AI strategy, the Pan-Canadian AI Strategy (ISED 2022). The strategy included establishment of a national program of research chairs to recruit and retain top researchers at Canadian universities, establishment of three national AI institutes to be global centres of training and research excellence, and the creation of a Pan-Canadian AI Compute Environment (PAICE) platform. The Digital Research Alliance of Canada’s National Research Software Strategy for 2025-2030 (Digital Research Alliance of Canada, National Research Software Strategy Working Group 2023) also reviewed (inter)national research software funding programs in support of AI, deep learning, and ML-facilitated research. Before the Alliance, CANARIE has been a national Research Software funder and service provider since 2007, whose successful research software initiatives had resulted in the development of sophisticated software tools, known as research platforms, that typically support end-to-end research workflow within a specific domain. New platforms re-used software components previously developed through CANARIE funding, and contributed additional components back to the research community, creating a powerful cycle of software development and reuse (CANARIE 2024). The Alliance is now using the Research Software Directory to continue promoting the visibility, impact, and reuse of the Canadian research software (Digital Research Alliance of Canada 2024).

6. Recommendations for research software to support AI

Research software needs to be included in AI strategies. Our recommendations for doing so are listed below, organised in three areas from the Amsterdam Declaration on Funding Research Software Sustainability (ReSA 2023):

1. Research software practice:

   a. AI strategies and funding must recognise that research software is a key part of the (publicly funded) AI pipeline and that AI is dependent on software, and therefore should stimulate the development and maintenance of research software to ensure the success of the AI work.

2. Research software ecosystem:

   a. Because AI-driven research is dependent on the existing research software ecosystem, AI strategies should provide long-term support for its elements, including personnel, communities, and infrastructure, and should add new elements that focus on AI-specific parts as needed.

3. Research software personnel:

   a. Because the existing research software ecosystem that supports AI-driven research is dependent on research software personnel, AI strategies should facilitate appropriate reward and recognition measures that enable career progression for all people involved in the creation and maintenance of research software that supports AI- driven research.

Other ways to help ensure that the focus on technological infrastructure to support AI acceleration includes research software and its personnel as a vital part of that infrastructure include:

• Signing the Amsterdam Declaration on Funding Research Software Sustainability (ADORE.software), which represents a first step to formalise, on a global level, the basic principles and recommendations related to funding the sustainability of research software, including the people needed to achieve this goal.
• Supporting ReSA in its aim to ensure that research software is considered in international discussions on how to advance research capabilities. Become a ReSA Organisational Member, support a task force, or donate.
• For funders, joining the Research Software Funders Forum, a global collaboration of funding organisations committed to supporting research software, and those who develop and maintain it, as fundamental and vital to research.

References

Barker, Michelle, Elena Breitmoser, Philippa Broadbent, Neil Chue Hong, Simon Hettrick, Ioanna Lampaki, Anthony Quinn, and Rebecca Taylor. 2024. ‘Software and Skills for Research Computing in the UK’. Zenodo. https://doi.org/10.5281/ZENODO.10473186.

Barker, Michelle, and Markus Buchhorn. 2022. ‘Research Software Capability in Australia’. Zenodo. https://doi.org/10.5281/ZENODO.6335998.

Barker, Michelle, Neil P. Chue Hong, Daniel S. Katz, Anna-Lena Lamprecht, Carlos Martinez-Ortiz, Fotis Psomopoulos, Jennifer Harrow, et al. 2022. ‘Introducing the FAIR Principles for Research Software’. Scientific Data 9 (1): 622. https://doi.org/10.1038/s41597-022-01710-x.

Barker, Michelle, and Daniel S Katz. 2022a. ‘Encouraging Entry, Retention, Diversity and Inclusion in Research Software Careers’. https://doi.org/10.5281/ZENODO.7117842.

Barker, Michelle, and Daniel S. Katz. 2022b. ‘Overview of Research Software Funding Landscape’, February. https://doi.org/10.5281/ZENODO.6102487.

Barker, Michelle, Daniel S. Katz, and Alejandra Gonzalez-Beltran. 2020. ‘Evidence for the Importance of Research Software’. Zenodo. https://doi.org/10.5281/ZENODO.3884311.

Buchanan, Bill. 2021. ‘Log4j: The Worst Vulnerability In Nearly A Decade?’ Medium (blog). 2021. https://medium.com/asecuritysite-when-bob-met-alice/log4j-the-worst-vulnerability-in-nearly-a-decade-e0cc80cbb49a.

Caballar, Rina Diane. 2024. ‘AI Copilots Are Changing How Coding Is Taught - IEEE Spectrum’. 2 May 2024. https://spectrum.ieee.org/ai-coding.

CANARIE. 2024. ‘Funded Research Software Platforms’. 2024. https://www.canarie.ca/software/platforms/.

Digital Research Alliance of Canada. 2024. ‘Research Software Directory’. 2024. https://research-software-directory.org/organisations/digital-research-alliance-of-canada?tab=software&order=is_featured.

Digital Research Alliance of Canada, National Research Software Strategy Working Group. 2023. ‘National Research Software Strategy 2023’. Zenodo. https://doi.org/10.5281/ZENODO.10214741.

Dursi, Jonathan. 2024. ‘We Need To Talk About AI’. Research Computing Teams 183 (June). https://newsletter.researchcomputingteams.org/archive/rct-183-we-need-to-talk-about-ai-plus-upfront/.

Engler, Alex. 2021. ‘How Open-Source Software Shapes AI Policy’. Brookings. 10 August 2021. https://www.brookings.edu/articles/how-open-source-software-shapes-ai-policy/.

Goodin, Dan. 2024. ‘The XZ Backdoor: Everything You Need to Know’. Wired. 2 April 2024. https://www.wired.com/story/xz-backdoor-everything-you-need-to-know/.

Gruenpeter, Morane, Daniel S. Katz, Anna-Lena Lamprecht, Tom Honeyman, Daniel Garijo, Alexander Struck, Anna Niehues, et al. 2021. ‘Defining Research Software: A Controversial Discussion’. Zenodo. https://doi.org/10.5281/ZENODO.5504016.

Hettrick, Simon, Radovan Bast, Steve Crouch, Claire Bradley, Philippe, Botzki, Alex, Carver, Jeffrey, et al. 2022. ‘International RSE Survey 2022’. https://softwaresaved.github.io/international-survey-2022/. https://doi.org/10.5281/ZENODO.6884882.

Hocquet, Alexandre, Frédéric Wieber, Gabriele Gramelsberger, Konrad Hinsen, Markus Diesmann, Fernando Pasquini Santos, Catharina Landström, et al. 2024. ‘Software in Science Is Ubiquitous yet Overlooked’. Nature Computational Science, July. https://doi.org/10.1038/s43588-024-00651-2.

ISED. 2022. ‘Pan-Canadian Artificial Intelligence Strategy’. Home page; Innovation, Science and Economic Development Canada. 20 July 2022. https://ised-isde.canada.ca/site/ai-strategy/en/pan-canadian-artificial-intelligence-strategy.

Lazauskas, Tomas, Jennifer Ding, Neil Brown, Reda Nausedaite, Felix Dijkstal, Aaron Vinnik, Bruno Raabe, et al. 2022. ‘Review of Digital Research Infrastructure Requirements for AI’. https://doi.org/10.13140/RG.2.2.29376.00009.

Martineau, Kim. 2021. ‘What Is Generative AI?’ IBM Research. 9 February 2021. https://research.ibm.com/blog/what-is-generative-AI.

Nahar, Nadia, Haoran Zhang, Grace Lewis, Shurui Zhou, and Christian Kästner. Forthcoming. ‘The Product Beyond the Model - An Empirical Study of Repositories of Open-Source ML Products’. In . https://www.cs.cmu.edu/~ckaestne/publications.html.

National Security Agency. 2024. ‘Recommendations for Software Bill of Materials (SBOM) Management’. https://media.defense.gov/2023/Dec/14/2003359097/-1/-1/0/CSI-SCRM-SBOM-MANAGEMENT.PDF.

Nordling, Linda. 2023. ‘How ChatGPT Is Transforming the Postdoc Experience’. Nature 622 (7983): 655–57. https://doi.org/10.1038/d41586-023-03235-8.

NSF. 2024. ‘National Artificial Intelligence Research Resource Pilot’. 2024. https://new.nsf.gov/focus-areas/artificial-intelligence/nairr.

ODSS. 2024. ‘Toward an Ethical Framework for AI in Biomedical and Behavioral Research’. 2024. https://www.scgcorp.com/ethicalframework2024/Agenda.

OECD. 2023. Artificial Intelligence in Science: Challenges, Opportunities and the Future of Research. OECD. https://doi.org/10.1787/a8d820bd-en.

Parsons, Mark, Alastair Basden, Richard Bower, Neil P. Chue Hong, Davide Constanzo, Shaun Witt, Luigi Del Debbio, et al. 2021. ‘ExCALIBUR Research Software Engineer Knowledge Integration Landscape Review’. Zenodo. https://doi.org/10.5281/ZENODO.4986062.

ReSA. 2023. ‘Amsterdam Declaration on Funding Research Software Sustainability’, August. https://doi.org/10.5281/ZENODO.8325436.

———. 2024a. ‘Research Software Funders Forum’. 2024. https://researchsoft.org/funders-forum/.

———. 2024b. ‘Research Software Funding Opportunities’. 2024. https://researchsoft.org/funding-opportunities/.

Robins-Early, Nick. 2024. ‘What Is CrowdStrike, and How Did It Cause a Global Windows Outage?’ The Guardian, 19 July 2024. https://www.theguardian.com/technology/article/2024/jul/19/what-is-crowdstrike-microsoft-windows-outage.

SCAWG. 2022. ‘Recommendations to Improve the Resilience of Canada’s Digital Supply Chain’. https://ised-isde.canada.ca/site/spectrum-management-telecommunications/sites/default/files/attachments/2022/CFDIR-June2022-recommendations.pdf.

Strasser, Carly, Kate Hertweck, Josh Greenberg, Dario Taraborelli, and Elizabeth Vu. 2022. ‘10 Simple Rules for Funding Scientific Open Source Software’, June. https://doi.org/10.5281/ZENODO.6611500.

Trueman, Charlotte. 2024. ‘UK Government Shelves £1.3bn of Tech and AI Projects; Scraps Plans for First Exascale Supercomputer in Edinburgh’. 2 August 2024. https://www.datacenterdynamics.com/en/news/uk-government-shelves-13bn-of-tech-and-ai-projects-scraps-plans-for-first-exascale-supercomputer-in-edinburgh/.

UKRI. 2023. ‘£300 Million to Launch First Phase of New AI Research Resource’. 1 November 2023. https://www.ukri.org/news/300-million-to-launch-first-phase-of-new-ai-research-resource/.

———. 2024a. ‘New Funding to Support Research Technical Professionals’. 18 March 2024. https://www.ukri.org/news/new-funding-to-support-research-technical-professionals/.

———. 2024b. ‘UKRI Digital Research Technical Professional Skills NetworkPlus’. 22 April 2024. https://www.ukri.org/opportunity/ukri-digital-research-technical-professional-skills-networkplus/.

US-RSE Association and IEEE Computer Society. 2023. ‘Research Software Engineers: Creating a Career Path—and a Career’. Zenodo. https://doi.org/10.5281/ZENODO.10073232.

Van Tuyl, Steve (Ed.). 2023. ‘Hiring, Managing, and Retaining Data Scientists and Research Software Engineers in Academia: A Career Guidebook from ADSA and US-RSE’. Zenodo. https://doi.org/10.5281/ZENODO.8274378.

The FAIR for Research Software (FAIR4RS) Principles aim to promote and encourage the findability, accessibility, interoperability, and reusability (FAIR) of research software. The FAIR4RS Principles were released in 2022, with a number of organisations already planning adoption at that time. Adoption and implementation of the FAIR4RS Principles can increase the transparency, reproducibility, and reusability of research by providing research software that can be executed, replicated, built-upon, combined, reinterpreted, reimplemented, and/or used in different settings and by third-parties.

This report provides an update on initiatives that are working to implement the principles across five areas of cultural change: policies, incentives, communities, training and infrastructure. It is noted that while many activities are increasing aspects of the FAIRness of research software, more work is still needed to make it easier to embrace the FAIR4RS Principles in their entirety.

A framework for tracking adoption

The release of the FAIR4RS Principles succeeded in raising the profile of research software in FAIR initiatives, which have mostly been focused on FAIR data. Examples can be found across a range of initiatives, using a slightly adapted version of the five elements of Brian Nosek’s strategy for culture change:

• Policy - make it required
• Incentives - make it rewarding
• Communities - make it normative
• Training - make it easy (this replaces user interface/experience in Nosek’s version)
• Infrastructure - make it possible

To achieve culture change, initiatives are needed across all of the five elements. Whilst in the early stages, some elements can progress more quickly than others. For example, it’s difficult to implement policy requirements if infrastructure and training (and preferably incentives) aren’t already in place. Similarly, for tools evaluating FAIRness to be useful, changes in major infrastructures such as code hosting repositories are first required so that researchers can create the metadata files that evaluations may depend upon.

It should be noted that while the many activities listed here support increasing FAIRness of research software, most of them do not address aspects of all four of the FAIRness of research software foundational principles:

Findable (F): Software, and its associated metadata, is easy for both humans and machines to find.

Accessible (A): Software, and its metadata, is retrievable via standardised protocols.

Interoperable (I): Software interoperates with other software by exchanging data and/or metadata, and/or through interaction via application programming interfaces (APIs), described through standards.

Reusable (R): Software is both usable (can be executed) and reusable (can be understood, modified, built upon, or incorporated into other software).

This reflects that the FAIR4RS Principles are aspirational and high-level, and do not contain detailed guidance on how to achieve them. This is because specific technologies and tools are always changing, while the principles are intended to be long-lasting. Consequently, additional work is needed to make it simpler for people wanting to follow the FAIR4RS Principles to know how to practically do so. The following initiatives are assisting in achieving this, with some of these initiatives specifically addressing the range of opportunities for future work identified in 2022 by the FAIR4RS Working Group, which developed the FAIR4RS Principles. These opportunities include “metadata and identifier authority, metadata vocabularies and metadata properties, software identifiers, domain-relevant community standards for software and identification targets”. Whilst some of the initiatives identified in the infrastructure section below are contributing to this, more work still needs to be done.

1. Policies that encourage implementation:

• The Netherlands eScience Center and the Dutch Research Council (NWO) formed a working group that developed national guidelines for software management plans. Research institutions such as the University of Groningen’s Digital Competence Center link to this resource in its guidance for its researchers on research software management plans.
• The Software Management Plan Template of the Netherlands eScience Center has been updated to align and address the FAIR4RS Principles, and emphasises that open science, software quality, and software sustainability are key elements in eScience Center projects. A total of 13 eScience Center projects (11 under the Open eScience call 2023 (OEC2023) and 2 under the Software Sustainability call 2023 (SS2023)) are now using this template.
• Maastricht University in the Netherlands has an Open Science @UM policy, which includes FAIR software as one of its seven areas. Maastricht University proposes to inventory current practices in storage, sharing and reuse of software; and then identify gaps in awareness, knowledge and/or support on FAIR software. Actions likely to be taken include promotion of the creation of software management plans to ensure responsible use of research software.
• Enabling FAIR Workflows - Key actors and actions by the Open Research Funders Group outlines steps for different actors in the research community to take to embed sharing practices, persistent identifiers, and metadata throughout the research lifecycle. This report references the FAIR4RS Principles in its recommendations on depositing research software.
• The German Research Council (DFG) published guidelines for reviewing grant proposals for Collaborative Research Centers (CRC/SFB) and suggested compliance with the FAIR4RS Principles for archiving and reuse.
• The Digital Research Alliance of Canada (the Alliance) published the National Research Software Strategy 2023. This proposes a set of strategic goals for advancing research software capability, community, and coordination in Canada for 2025-2030, with the FAIR4RS Principles providing a cross-cutting theme. The Alliance is developing a Software Management Plan (SMP) template to promote, cultivate and implement best practices, with an emphasis on the FAIR4RS Principles, for Canadian researchers. The goal is to implement the SMP template in future grant applications.

Resources that support inclusion of FAIR into institutional policies are also relevant. The list of research institutional policies that support research software, curated by the Research Software Alliance (ReSA), is part of ongoing work by the joint ReSA and Research Data Alliance (RDA) Policies in Research Organisations for Research Software (PRO4RS) Working Group to create a community of stakeholders involved in promoting and/or implementing policy that supports research software at the research institution level (such as universities, national laboratories). This includes curation of resources on how to influence policy change, such as Health Research Performing Organisations (HRPOs) FAIR Guidelines by Celia Alvarez Romero et al. This provides principles, steps, and resources to support the complex change needed to implement a data policy, which could also be applied to research software policy change.

2. Incentives that motivate change:

• FAIR-IMPACT launched open calls for cascading grants that provide financial support ranging from 4,000-10,000 euros. Focus areas on research software include:
  • Path 1: Assessment and improvement of existing research software using a new extension of F-UJI (a web service to assess FAIRness of research objects).
• Path 2: Implementation of the Research Software MetaData Guidelines for better archiving, referencing, describing, and citing research software artefacts.
• The German Research Council (DFG) issued a Call for Proposals to Increase the Usability of Existing Research Software that refers to the FAIR4RS Principles in terms of availability and reproducibility.
• The German Ministry of Education and Research (BMBF) issued funding guidelines (in German) for developing data custodian models that refer to standards based on the FAIR4RS Principles for reusable and well-documented open source software developed under this program.

3. Communities that are normalising adoption:

• The RDA Software Source Code Interest Group provides a forum to discuss issues on management, sharing, discovery, archival and provenance of software source code. In October 2023 the group session at the RDA Plenary included a focus on the FAIR-IMPACT Metrics for Assessing Research Software FAIRness (recording). In addition, this group is the maintenance home for the FAIR4RS Principles.
• The Research Software Funders Forum included a working group focused on implementation of the FAIR4RS Principles, which review the FAIR4RS Principles per the interests of research software funders, to identify gaps.
• Ten simple rules for starting FAIR discussions in your community presents guidance and recommendations on how to start up discussions around the implementation of the FAIR Principles and creation of standardised ways of working. Whilst not specific to FAIR4RS Principles, these recommendations can assist in providing understanding of the benefits and barriers of standardisation are, and will support a more effective way of engaging the community.

4. Training to develop relevant skills:

• The Netherlands X-omics Initiative has created Ten quick tips for building FAIR workflows, which provides practical recommendations for implementation of the FAIR4RS Principles when creating computational workflows.
• Making Biomedical Research Software FAIR: Actionable Step-by-step Guidelines with a User-support Tool assists biomedical researchers to make their research software compliant with the FAIR4RS Principles. The FAIR Biomedical Research Software (FAIR-BioRS) guidelines are based on the FAIR4RS Principles and a workflow that streamlines the process of implementing these guidelines is also provided.
• The Netherlands eScience Center is developing and piloting the ⁠Research Software Support course which covers the FAIR4RS Principles and provides guidance for research personnel.
• The Digital Competence Centre of the Delft University of Technology (TU Delft) in the Netherlands is offering a 13-week FAIR for Research Software Program, which includes resources from the Code Refinery, a project within the Nordic e-Infrastructure Collaboration (NeIC). TU Delft’s program aims to teach the essential tools for creating scientific software following the FAIR4RS Principles and how to apply them.
• The Data Competence Center at the Leiden University Medical Center (LUMC) aims to help researchers implement the FAIR4RS Principles by increasing software stewardship capacity.
• The European Infrastructure for Translational Medicine (EATRIS) promotes adoption of the FAIR4RS Principles by sharing practical implementation examples from and with the community via its Multi-omics Toolbox (MOTBX).
• The Alliance’s National Research Software Strategy 2023 recommends FAIR4RS as a key element underlying training & support initiatives for research software, which will be included in the forthcoming national training framework and resourced appropriately.

5. Infrastructure that provide supporting tools:

• FAIR-IMPACT’s release of Metrics for automated FAIR software assessment in a disciplinary context defines 17 metrics that can be used to automate the assessment of research software against the FAIR4RS Principles, and provides examples of how these might be implemented in one exemplar disciplinary context of the social sciences. The FAIR-IMPACT project will work to implement the metrics as practical tests by extending existing assessment tools such as F-UJI.
• Horizon Europe’s European Virtual Institute for Research Software Excellence (EVERSE) aims to create a framework for research software and code excellence, that are collaboratively designed and championed by the research communities. EVERSE will also continue the work of FAIR-IMPACT by developing and implementing processes and tools that support the assessment and verification of code quality, based on established best practices and standards across scientific communities.
• FAIRsoft is a practical implementation of the FAIR4RS Principles, and the FAIRsoft evaluator is a tool for developers and users to assess how specific software complies with FAIR for software indicators. It is part of the ELIXIR’s OpenEbench Software Observatory, an instrument for the systematic observation and diagnosis of the quality of research software in the life sciences.
• FAIR-Impact’s Guidelines for recommended metadata standard for research software within EOSC acknowledges the rising need for establishing software metadata guidelines to effectively collect and curate metadata. A comprehensive set of Research Software MetaData (RSMD) Guidelines are provided that offer flexible and adaptable recommendations for end-users that can be used in different disciplines and different software development contexts. The guidelines are directly relevant to end users, including software creators and curators in their quest to improve the FAIRness of their software.
• The ELIXIR Software Best Practices group, NFDI4DataScience and Bioschemas are collaborating together to support machine-actionable SMPs which are aligned to the ELIXIR Software Management Plan for Life Sciences published by ELIXIR; and also aligned to other initiatives but also to others, such as the practical guide to SMPs by the Dutch Research Council (NWO) and the Netherlands eScience Center, and the SMP template by the Max Planck Digital Library.
• A new version of the machine-actionable Software Management Plan Ontology (maSMP Ontology) metadata schema, vr2.1.0, was released in January 2024, together with usage guidance about the properties (profiles, guides on minimum, recommended and optional properties with cardinalities). The metadata schema includes entities involved in software management planning; such as an SMP itself, software source code, software release, documentation, authors and their relations. Integration into Bioschemas is still pending.
• A metadata enrichment cycle aligned to the maSMP metadata schema has been proposed by ELIXIR thanks to the Software Management Wizard, a tool to make completion of SMPs easier. A similar effort is within the scope of NFDI4DataScience and the Research Data Management Organiser (RDMO) SMP. This approach reuses a command-based tool to extract metadata from GitHub repositories, SOMEF, which is currently being extended to cover the maSMP metadata schema case.
• The Research Software APIs and Connectors project within the FAIRCORE4EOSC project is working on developing tools and services for archival, reference, description, and citation of research software artefacts. This implements the key recommendations of the Scholarly Infrastructures of Research Software report to interconnect scholarly repositories, publishers, and aggregators. Interconnections are possible with the Software Heritage universal source code archive, using the CodeMeta standard, and the Software Heritage intrinsic identifiers (SWHID). Instructions on how to archive your software to Software Heritage is one outcome.
• FAIRSECO: An Extensible Framework for Impact Measurement of Research Software aims to enable research software engineers to rapidly find and extract relevant software fragments from the worldwide research software ecosystem.
• Towards a Quality Indicator for Research Data publications and Research Software publications – A vision from the Helmholtz Association develops indicators to be used within the Association. It presents a quality assessment spanning six dimensions of research software quality that augments the four FAIR principles with two additional indicators: Scientific basis and Technical basis, resulting in the FAIR-ST framework.
• A self-assessment tool to promote FAIR research software has been developed by the Netherlands eScience Center and Australian Research Data Commons, to encourage the uptake of the FAIR4RS Principles (and see the 2022 Survey on Adoption Guidelines for the FAIR4RS Principles for more resources).
• openCARP-CI provides Python scripts that allow developers to automatically derive CFF and DataCite files from a CodeMeta file. These pipelines can easily be integrated in continuous integration and deployment environments. They also provide tools for software publication via tagged releases, creation of BagIt and BagPack files, and publication on the research data repository RADAR.
• CodeMeta-3.0: The minimal metadata schema for science software and code, in JSON-LD, provides a possibility to developers and researchers to insert metadata in their code and increase FAIRness.
• FAIR-USE4OS: From open source to Open Source by Raphael Sonabend et al., extends the FAIR4RS Principles to provide criteria for assessing if software is Open Source. By adding ‘USE’ (User-Centred, Sustainable, Equitable), software development can adhere to open source best practice by incorporating user-input early on, ensuring front-end designs are accessible to all possible stakeholders, and planning long-term sustainability alongside software design.
• FAIR4RS has also been discussed within the scope of Open Science and software quality, for instance the EOSC Task Force Infrastructures for Quality Research Software compiled software quality metrics and identified those that can be aligned to the FAIR4RS Principles.

Other impacts

The publication of the FAIR4RS principles and introductory articles created awareness and raised interest in the research community, reflected in over 200 citations from across the disciplinary spectrum. Examples include the results of searches on Google Scholar for “FAIR principles for research software (FAIR4RS principles)” or “ Introducing the FAIR Principles for research software”. In addition to work discussing trans-disciplinary application of the FAIR4RS Principles, there are research policy and software management publications referring to and implementing the FAIR4RS Principles.

The FAIR4RS Principles have also provided value to the broader research ecosystem by providing a base for other communities to adapt the FAIR Principles to different research objects. Examples include FAIR AI Models in High Energy Physics, which provides a practical definition of FAIR principles for machine learning and artificial intelligence models in experimental high energy physics, including a FAIR AI project template; and the Open Modeling Foundation’s work to identify, develop, and promote common standards and best practices for FAIR modelling, by working with model organisations and individuals active within the social, ecological, environmental, and geophysical sciences.

The Research Software Alliance (ReSA) has released their 2023 Community Report, a collection of all highlights and chievements for the past year. The full report can be accessed on Zenodo.

They have also released the annual reports from the ReSA Community Managers for Asia and Africa: Advocating for Research Software Engineering in Asia: 2023 in Review and Navigating the Unseen: Unveiling the Scope of African Research Software and Systems Engineering.

The Research Software Alliance (ReSA) aims to ensure that research software and those who develop and maintain it are recognised and valued as fundamental and vital to research worldwide. ReSA’s mission is to advance the research software ecosystem by collaborating with decision makers and key influencers. A wide range of research software organisations and programs exist internationally to address the varied challenges in software productivity, quality, reproducibility, and sustainability. ReSA aims to coordinate across these efforts to leverage investments and achieve shared goals. For further information or to cite ReSA, please refer to The Research Software Alliance (ReSA).