With growing demand for workshops across the UK, there is a need for more qualified Carpentry instructors. The Institute is therefore working with The Carpentries to run instructor training events, based on a curriculum designed by Greg Wilson, the founder of Software Carpentry. 

This guide is intended to help you understand what cloud computing is, the benefits it may be able to offer you as a researcher and the different options available for gaining access to cloud computing resources. The guide is intended to help you make informed decisions about whether or not use of a cloud computing platform could contribute to your research and associated costs and benefits of its use. It is also hoped that this guide will assist you when applying for funding by helping you to justify any requests for use of cloud resources, or in considering whether to use cloud on projects that you may be currently involved in.

The following guidance on good practice in submitting software outputs to REF 2021 has been developed by the UK Software Sustainability Institute in consultation with REF 2021 Sub-panel 11. It should be read in conjunction with the information already provided in the Guidance on Submissions (REF 2019/01, Annex K) and Panel Criteria and Working Methods (REF 2019/02, Paragraph 254 and Annex B).

The submission should be in the form of a PDF that provides key information on the software (no length limit, within reason). To ensure the sub panel has enough information to assess the submission, it should include the following components

Access Details

DOI/URL pointing to a code repository or deposit in a digital repository containing:

• source code;
• compiled code and/or instructions on how to compile or install the software;
• user documentation;
• test suite and/or quality assurance procedures.

 Where the submitted software is part of a larger code base (e.g. a library contributed to an open source project) it must be clear which part the submission refers to. Ideally, the DOI/URL should reference a specific version of the software.

Publication Details

Evidence to show when and how the submitted software was first brought into the public domain (which must be within the REF 2021 publication period). If the submission was an enhancement of pre-existing software, the date when the enhancement was published should be given, and the remainder of the submission should make a clear distinction between the enhancement and the existing software (which may need to be described to provide context).

Authorship Details

Where the software is the work of multiple authors, they should be listed where possible, whether from the submitting unit or not. In every case the role of the individual to whom the output is attached should be made clear.

Research Process

A summary of the research process of which the software is the output and the insights it embodies, where research is defined for REF purposes as a process of investigation leading to new insights, effectively shared (see Guidance on Submissions, Annex C for an expanded definition).

[NB the Criteria and Working Methods (paragraph 254) allow for the submission of a separate description of the research process and content, ‘where this is not evident within the output’. If this is covered in the software submission itself, as suggested here and in the Guidance on Submissions, Annex K, the separate statement is not necessary].

Description of the Software

A description of the purpose of the software, the context in which it is used, and its functionality (300 words maximum).

Supporting Evidence

Numbered list of other publicly accessible sources of supporting evidence. This might include:

• DOI for a companion software paper;
• DOIs for research outputs resulting from use of the software;
• Usage metrics.

Addressing the REF Assessment Criteria

REF2021 will assess outputs on the basis of their Originality, Significance and Rigour, as defined in Panel Criteria and Working Methods paragraphs 191-193. The following subsections provide further suggestions for how these assessment criteria might be evidenced for software outputs.

Originality

The REF definition of originality is the extent to which an output makes an important and innovative contribution to understanding and knowledge in the field. For software this might include, but is not limited to:

• implementation of a novel algorithm;
• development of innovative research methods;
• a solution to a problem that has not been solved before in practice;
• interpretation of or engagement with novel forms or scales of data;
• improvement in the scale, resolution or accuracy at which research can be performed.

Significance

The REF definition of significance is the extent to which the work has influenced, or has the capacity to influence, knowledge and scholarly thought, or the development and understanding of policy and/or practice. For software this might include, but is not limited to:

• enabling or accelerating significant research advances;
• enabling research or thinking that was not previously possible;
• influencing policy, practice or users/audiences;
• being used by a large number of researchers, or within a diverse range of disciplines.

Rigour

The REF definition of rigour is the extent to which the work demonstrates intellectual coherence and integrity, and adopts robust and appropriate concepts, analyses, sources, theories and/or methodologies. For software this might include, but is not limited to:

• being used to produce peer-reviewed research outputs;
• clear documentation that explains:
  • the software purpose and intended users, including the research context;
  • how the software works, including the research process it enables;
  • architecture and design rationale, to provide confidence in the appropriateness of the approach;
• practices to improve reproducibility of results, e.g. automation, recording of provenance, use of standard data formats;
• a clear strategy for internal software quality assurance, e.g. a unit test suite to provide confidence in research results, and consistent coding standards to ensure readability of code;
• formal verification;
• adoption of community-accepted standards;
• certification;
• code and documentation held in version control;
• regular releases;
• structured end-user evaluation;
• quality-based peer review (e.g. publishing a software paper in a specialist software journal, or receiving an ACM artefact badge).

This page used to host our guide on how to run a Software Carpentry workshop (previously called a boot camp). After a very positive reaction from Greg Wilson, the creator of Software Carpentry, we were asked if we could host our guide on the Software Carpentry web site. In a spirit of sustainability and a desire to contribute to the Software Carpentry community, we agreed and we will continue to develop the guide as part of the Software Carpentry community. Below you can find some quick guidelines and tips for running a workshop. Even though originally developed for Software Carpentry workshops, this guide is applicable to all Carpentry workshops (Data and Library) that came a bit later. 

Building a community around software is an important step for its sustainability. This guide is for researchers, developers and managers of a software product that has started to gain a community. This guide will discuss how to increase your community and encourage contributions from it.

The ultimate goal of community building is to nurture an active supporting community. An active community can become self-supporting: answering queries raised by community members and contributing new functionality and bug fixes back to your project. This gives you more time to develop your software and reduces the resources needed to sustain it. In this way, a community can help you make your software more useful to more people. 

Once it has left the confines of your own machine, there are four things that are needed for the successful development of your software: a website, a mailing list, an issue tracker and a code repository.

Although most of the infrastructure needed by your project can be set up on your own systems, there are many tools and services that can help you to develop, maintain and publish your software. This guide provides an overview of the different options for repositories, and looks at some of the decisions you will need to make before choosing a repository. Other SSI guides take a more detailed look at specific repositories.

We've also written a blog post about one of our staff member's experiences of choosing a code repository. It provides further information about which repository you should choose.

Say you've got a Google spreadsheet with a column for addresses. It could be street addresses or postcodes. You want to map this data and embed the map into a website. Maybe you also want the map to update dynamically as more rows are added to the spreadsheet. What are your options?

This guide goes through the different ways to do this. However, to first map the data you’ll need to find the geocodes; i.e., latitude and longitude coordinates for these addresses. For locations that are more general, such as “UK”, geocoding APIs usually return the coordinates of the centroid—the area’s center point—or the capital.

After an hour of discussion on a research software related topic at a workshop, a discussion group would stand up for 2-3 minutes and present back their findings (e.g. problems, solutions, future work or however they chose to speak about a topic). However without context the notes produced from such a session are not of much use to the wider community after the workshop. So, what's the solution?

It's frustrating when you have an idea for improving your software, but lack the resources needed to realise it. Not all is lost! Your idea might be the perfect basis for a student project. Student projects are generally a win-win situation for you and the student. You get to investigate ideas with relatively little outlay and the student gets highly valuable work experience in a professional environment.