The impact

Data being generated by today’s large-scale scientific research projects will remain available, searchable and useable in future, thanks to the Software Sustainability Institute’s work with the ICAT project.

The Institute’s advice on the need for defined governance and careful roadmapping have transformed the ICAT service into one that is making a real difference to the work of researchers in data-intensive spheres.

ICAT is now used to catalogue millions of files every day. Current stakeholders include the ISIS pulsed neutron and muon source, the Diamond Light Source and Central Laser Facility (CLF) at Rutherford Appleton Laboratory (RAL) in the UK, the Spallation Nuetron Source in the USA, the Elletra synchrotron in Italy and, currently coming on stream, the 11 European facilities in the European PaNdataODI Open Data Infrastructure project that represents 30,000 scientists.

The problem

The ICAT project was set up 10 years ago to look at how to store, catalogue and manage the enormous amounts of data being produced by the Scientific Technology Facilities Council (STFC) at RAL in Oxfordshire.

Since being established, ICAT had developed organically and were looking to improve their governance. The project was beginning to attract many potential new partners, but did not have the mechanisms in place to cope with the influx. There was also a need to improve information on how to contribute code, and how to gain support.

Similarly, planning and roadmapping needed to include provisions for the many new users. There were lots of potential users out there, and work was needed to welcome and help these new people get involved.

The solution

The Institute took a close look at the set up of ICAT and its current operations, and produced a comprehensive report giving over 30 recommendations in a range of areas including governance, community and outreach, process and policies, support and infrastructure. Within a year, the ICAT team have carried out almost all of the recommendations in the Institute's report.

The report included a number of observations on the ICAT software and improvements that could potentially be made. The ICAT team were able to take these suggestions on board and develop a roadmap for improvement, creating a solid foundation for the huge increase in uptake expected over the next couple of years.

Governance of the collaboration is now clearly defined. The requirements-ingest process has been improved and streamlined, with policies in place for deprecation, code contribution and software release. The ICAT team now engages actively in facility roll out plans. A well-defined induction and mentoring process is also now in place for new users.