EEGManyPipelines: A large-scale, grass-root multi-analyst study of EEG analysis

The ongoing reproducibility crisis in psychology and cognitive neuroscience has sparked increasing calls to re-evaluate and reshape scientific culture and practices. Heeding those calls, we have recently launched the EEGManyPipelines project as a means to assess the robustness of electroencephalography (EEG) research in naturalistic conditions and experiment with an alternative model of conducting scientific research. 168 analyst teams, encompassing 396 individual researchers from 37 countries, independently analyzed the same unpublished, representative EEG dataset to test the same set of pre-defined hypotheses, and then provided their analysis pipelines and reported outcomes. Here, we lay out how large-scale scientific projects can be set up in a grass-roots, community-driven manner without a central organizing laboratory. We explain our recruitment strategy, our guidance for analysts, the eventual outputs of this project and how it might have a lasting impact on the field.