By M. H. Beals, Catherine Jones, Geraint Palmer, Mike Jackson, Henry Wilde, John Hammersley, Daniel Grose, Robin Long, Adrian-Tudor Panescu, Kirstie Whitaker
What does reproducible mean? Who do we want to help and support by making our research reproducible? At what point does non-reproducible research become good enough (and carries on to the highest standards of reproducibility?)
In our discussions during the first speed blogging session at the Software Sustainability Institute’s Collaborations Workshop in Cardiff in March 2018, we brainstormed criteria for judging the quality of reproducible research. What emerged were two clear messages: 1) We all have our own overlapping definitions of the desirable features of reproducible research, and 2) there is no great benefit in rehashing old discussions.
In this blog post we outline 9 criteria that can be met by reproducible research. We believe that meeting as many of these as possible is moving in the right direction. Source code and data availability are often seen as important requirements, but documenting what code is trying to achieve, which other software libraries are required to run the code, the greater research ecosystem, what lessons were learned in the development of the…Continue Reading