Software and research: the Institute's Blog

Latest version published on 12 January, 2018.

38449949554_d3c3ccd279_z.jpgBy Raniere Silva, Community Officer.

2018 started and my New Year resolutions are: using version control in all projects, commenting on my scripts, writing documentation for my packages, including tests for the code that I develop, taking advantage of continuous integration, providing virtual machine or containers for users, being more productive, and being more sustainable. Are any of those on your list too? If the answer is "yes", then our Collaborations Workshop 2018 (CW18) is a must go event for you this year. You can register now for CW18, which will take place on 26–28 March 2018 at Cardiff University.

Our Collaborations Workshop series is held in a unique ‘unconference’ style format that will definitely help you cross some of those New Year resolutions. The Hackday will be especially useful, as attendees form teams, get to know other, work for 24 hours in an idea using all sort of best practices and compete for generous prizes— in previous years we had Amazon Echo and Raspberry Pi 3 kits as prize.

The first two…

Continue Reading

Latest version published on 11 January, 2018.

By Edward Smith, Imperial College London.

The Research Software Engineer (RSE) 2017 conference started with an excellent keynote talk by Chris Woods, emphasising a number of issues. This set the tone for the conference, outlining the aspiration of developing the RSE community and changing perceptions. There seems to be clear progress in this area and a number of challenges still to face. One issue that came up was the misalignment of aspiration in the university selection processes for RSE fellowships candidates compared to those in the national RSE community. For me, the most interesting point was Chris’ emphasis on letting people use the technologies they feel most comfortable with, when working as an RSE. This was a theme from the conference that continued with no less than four talks by software engineers working at or with the MET office. Weather prediction is unique in that it is scientific software with a direct and constant validation, often by angry people without umbrellas. It is also an area where code and hardware reliability is crucial. They have an entire standby supercomputer in case the main one goes down and an extensive testing framework. In this context, it is therefore interesting that the MET RSEs are working on rewriting the entire HPC…

Continue Reading

Latest version published on 18 December, 2017.

Assigning fellows applicationsBy Raniere Silva, Community Officer

A few people have asked us how we run certain processes at the Institute. This time, we will look at how we assigned Fellowship programme 2018 applications to our reviewers.

Data repository

We used Google Forms to collect applications as, from experience in previous editions, we know that Google Spreadsheet works well for reviewers as they are familiar with the platform and usually have a Google account. Google Drive then allows us to share the data the reviewers need and use "Microsoft Excel programming language" to summarise the result of the reviews.

On the master spreadsheet each reviewer has a sheet with their initials, where they will find all the information needed to assess the candidates, with relevant columns to mark their thoughts. Data validation helps reviewers input correct values.

Sheet generation

To generate each of the reviewers’ sheet we use pandas and PuLP, both Python libraries. Pandas allows us to interact with raw data stored in a tabular form as a Google Spreadsheet and to create local CSV files. PuLP is a linear optimisation

Continue Reading

Latest version published on 14 December, 2017.

Motivate Better Practice in Research SoftwareBy James Grant, University of Bath, Andrew Washbrook, University of Edinburgh, Louise Brown, University of Nottingham, Niels Drost, Netherlands eScience Center, and Andrew Bennett, European Centre for Medium-range Weather Forecasts

What can be termed as "coding" is a subset of wider software engineering practices such as version control, continuous integration and good software design. Coding is prevalent in academia but practices that allow sustainable software to be produced are frequently overlooked.  Motivating the uptake of the approaches, methods and tools, and highlighting the benefit they deliver, by engaging with researchers who develop software is the first step in spreading best practice in our community.

In discussions with researchers, we find that the use of version control is often highlighted as the first methodology that they would like to introduce into their workflow. We would therefore like to 1) identify approaches that can promote the use of version control by reducing barriers from textbook to full integration and 2) highlight the wider benefits of the methods beyond traditional software development.

Software related courses at an undergraduate level tend to focus on code syntax and functionality with limited time spent covering software management practices.  By including the use of version control as part of these training programs we can avoid much…

Continue Reading

Latest version published on 13 December, 2017.

Good software development practiceBy Toby Hodges, EMBL, Roman Klapaukh, UCL, David McKain, University of Edinburgh, and Tobias Schlauch, DLR

WSSSPE members recognise the value of good practice in software development and the benefits of robust and reproducible code in research. The adoption of such practice requires an investment of time, energy, and money, and unfortunately it can be difficult to convince research scientists that this investment is worthwhile. Several initiatives already aim to encourage good practice in research software development, for example, Software Carpentry, the Software Sustainability Institute, the internal Software Engineering Initiative at the German Aerospace Center (DLR), and the Bio-IT project at EMBL. However, some research groups have remained difficult to reach.

Different approaches can be targeted at different levels and career stages. The majority of these share a common theme of support for the local research community.

For those at an early career stage, adoption of good practice can be encouraged by the availability of training, resources, and tools…

Continue Reading