The impact

Completely new types of climate simulation have been made possible by a Software Sustainability Institute project at the University of East Anglia. Higher throughput rates have been achieved, and visualisation tools are helping scientists better interpret their results and more easily explain them to others.

The Institute worked with the University of East Anglia’s Community Integrated Assessment System (CIAS) portal on the new functionality. The team also worked through a careful iterative process to create an installation procedure, allowing CIAS to finally replicate its portal elsewhere, so that researchers outside the University of East Anglia can make use of the code.

The problem

The University of East Anglia’s CIAS portal allows researchers to bring together data from different environmental areas – land, ocean and atmospheric data, for example – and combine them to create complex and in-depth simulations. While useful, the system was limited in its throughput capabilities – it was only able to run one simulation at a time and the system needed constant monitoring while running.

Searching for previous research was also difficult, with limited metadata on work that had been completed and poor keyword search capabilities. 

Replicating the portal was a further problem – the UEA portal had been created over time by researchers and there was little or no documentation available on how to create a copy.

The solution

The Institute began by creating a development copy of the portal in order to insert the batch queueing system. This involved a painstaking step-by-step process of asking UEA researchers to write each procedure, which was then carefully followed by an Institute developer. If any problem arose in following a set of instructions then new ones were written. This slow, iterative process eventually created a process that can be followed by any competent systems administrator to create a duplicate copy of the CIAS portal.

With a development copy in place, the Institute went on to add the batch queue – a challenging process due to the complex internal code that had been written and amended by a succession of UEA researchers over the years. This allowed many climate simulations to be queued, which freed researchers from the laborious task of having to wait for one simulation to complete before another could be run.

The internal software modelling framework, Bespoke Framework Generator (BFG), had to be upgraded. This challenge was tackled in partnership with the framework’s author, Rupert Ford, of the University of Manchester. The result was a successful upgrade allowing CIAS to carry out models with feedback loops, making possible a whole new class of climate simulations.

Utilities were added to visualise netCDF data and generate scatter plots, for easy visual display of data and results.

Finally, the review of the internal code also allowed redundancy to be identified and security holes plugged (something of great importance in this occassionally contentious research field). The result has been a vastly improved and more accessible service, offering much broader climate modelling possibilities.