Neuroscience: Sample size matters in studies linking brain scans to behaviour

Studies assessing the association between brain structure and/or function with complex behaviours require thousands of people to make the results reliable, an analysis of data from around 50,000 individuals in Nature reveals.

Brain-wide association studies (BWAS) aim to use data from brain scans, such as magnetic resonance imaging (MRI), to look for links between variation in brain structure and/or function and characteristics related to cognition and mental health. Such associations may help us to predict or prevent psychiatric diseases and improve our understanding of human cognitive abilities. However, the high costs associated with obtaining MRI data (around US $1,000 per hour) have limited the sample sizes of BWAS (often to around 25 participants), which makes it hard to obtain reproducible results.

Scott Marek and colleagues evaluate the effect of sample sizes on brain-wide association reproducibility. They analyse data from three of the largest neuroimaging studies to date: the Adolescent Brain Cognitive Development study (11,874 participants), the Human Connectome Project (1,200 participants) and the UK Biobank (35,735 participants), allowing them to investigate the reliability of BWAS with samples ranging from 25 to more than 30,000. Their analyses reveal that small BWAS sample sizes result in inflated effect sizes and irreproducible associations. As sample sizes reach the thousands, replication rates begin to improve and effect-size inflation decreases. Variability across population subsamples and small association effects can explain widespread BWAS replication failures, the authors note, and suggest that the results from any BWAS with smaller sample sizes should be interpreted with a degree of caution.

Future efforts to improve the outcomes of BWAS could include mandatory sharing policies to aggregate data and focussing on the most robust associations between brain structure, function and cognitive or mental health traits, the authors conclude.