The Institute for Replication (I4R) and the University of Sheffield will be organising the AI Replication Game at the University of Sheffield on Monday 17 June 2024 from 9.14 AM to 4 PM. This one-day event aims to bring researchers together to collaborate on reproducing quantitative results published in high-ranking social science journals. By taking part in the Replication Game, participants will not only contribute to the integrity of research in their field but also have the opportunity to network with fellow researchers and develop their coding and AI skills.

Researchers participating in the AI Replication Game will be randomly assigned to one of three teams: Machine, Cyborg or Human. Machine and Cyborg teams will have access to (commercially available) LLM models to conduct their work; Human teams will rely only on unaugmented human skills. Teams will be asked to check for coding errors and conduct a robustness reproduction, which is the ability to duplicate the results of a prior study using the same data but different procedures. During the event, participants are expected to read the paper and familiarise themselves with the code and data shared alongside the paper. Teams will then work together to check for coding errors and conduct sensitivity analysis. No work is conducted before/after the event other than answering a short survey.

All participants will get coauthorship on a metaresearch journal paper which combines the work of all teams.

The Replication Game is open to all researchers: academics, post-docs, and graduate students. Knowledge of Python or R is essential.

For more information and to register for the AI Replication Game, please follow the link below. The deadline for registration is Saturday 1 June.