Reproducibility Hackathon Netherlands – the aftermath

Posted by on 15 January 2020 - 12:30pm
Reprohack NL logo
Image by ReproHack NL

By Linda Nab, Daniela Gawehns, Paloma Rojas Saunero, Ricarda Proppert, and Software Sustainability Institute Fellow Anna Krystalli

On Saturday 30 November, 44 excited researchers gathered at the Leiden University Library to join the very first ReproHack – short for Reproducibility Hackathon – in the Netherlands. In a nutshell, a ReproHack is aimed at assessing the reproducibility of scientific research that publicly shared code and data. During a ReproHack, participants get to work with other people’s material in a low pressure environment and record their experiences on a number of key aspects, including reproducibility, transparency and reusability of materials.

The first few ReproHacks were run by Anna Krystalli as part of Opencon satellite events. These events were picked up on by Daniela and her call on Twitter led to the formation of the so-called “Leiden ReproHack Ladies”. The rest is history: hard work in the leading months to the event led to the first Dutch ReproHack.


The event in summary

Anna Krystalli took the honour of kicking off the event and shared her perspective to research reproducibility and described the concept of “research compendia”, which provides a standard and easily recognisable way for organising the digital materials of a project to enable others to inspect, reproduce, and extend the research. After Anna’s talk, participants formed teams and began the challenge of reproducing their chosen papers.


Lunch break provided a great opportunity for networking and discussing hacking progress, difficulties and experiences so far. After lunch, the day continued with a talk by John Boy, who shared his vision for open science beyond the reproducibility crisis and generated an enriching discussion among the participants. 

By the end of the afternoon, Mateusz Kuzak from the Netherlands eScience center improvised a hands-on tutorial on making software citable and introduced FAIR software for promoting good research software development. The day ended with a group discussion about the different hacking experiences and overall feedback of the event, followed by refreshments and networking. And last but not least, future ReproHack plannings! 

The 44 participants formed a diverse group of researchers from the fields of science and engineering, all eager to build towards more open and reproducible research practices. Of 19 papers with code and data, 24 feedback forms were filled, five papers were successfully reproduced and six were nearly reproduced. The filled feedback forms were sent to the authors of the papers after the event, who appreciated the received feedback. 

Brief evaluation of the event

The organising team attempted to reproduce the ReproHack by using the material available on github. To make organising future ReproHacks easier, experiences will be written down in an organiser’s guideline. Follow ReproHack on Twitter to stay tuned for these guidelines. The excitement of ReproHackNL’s participants made the event a success.

Read how Mateusz Kuzak experienced the ReproHack in his blog. Improvements include a better internet connection and guidelines for participants on how to review the reproducibility of a paper. 

Big thank you

ReproHackNL would not have existed without sponsors: Leiden institute of advanced computer science, the data science research programmedata archiving and network servicesyoung academy Leiden and Erasmus medical center. Also, a big shout out for Leiden university library for opening their doors. The Software Sustainability Institute kindly sponsored Anna’s trip to the Netherlands.

The ReproHack community is slowly growing. More and more people volunteer to organise ReproHacks at their own universities and research institutes. Are you excited to become part of the ReproHack community? Check here (website under construction!) if there is an event nearby or commit a paper for a future ReproHack and follow us on Twitter.

Want to discuss this post with us? Send us an email or contact us on Twitter @SoftwareSaved. 

Share this page