The dangers of double dipping
Nature Neuroscience
April 27, 2009
Analyses of data from functional magnetic resonance imaging (fMRI) and studies tracking electrical activity in the brain can generate misleading results, suggests an article online this week in Nature Neuroscience.
Techniques such as fMRI, which provides images of the brain, and electrophysiology, which record electrical activity from many neurons at once, generate a large amount of data. Some of this data is intrinsically 'noisy', being accompanied by other background activity, like listening to a radio station with poor signal quality. In order to enhance the signal of interest, the raw data is substantially analyzed, often using complex transformations.
Nikolaus Kriegeskorte and Chris Baker analysed some artificially generated 'noise' data, unrelated to experimental variables, but still obtained results This shows that in many cases, analysis can generate spurious results entirely unconnected with real experimental results.
These spurious results arise from a practice the authors call 'double dipping'. For example, scientists may hypothesize that a brain region responds more strongly to one particular stimulus compared to another. An incorrect, 'double dipping' analysis would look for areas that were more active with the original stimulus and selectively analyze only these areas to test the hypothesis. However, some areas may activate more to one stimulus than another purely by chance, and restricting further analysis only to these areas would yield a misleading result.
doi: 10.1038/nn.2303
Research highlights
-
Jun 30
Genomics: Gray wolf genome hints at dual ancestry of dogsNature
-
Jun 30
Evolution: Hawks learn on the fly to swoop up before perchingNature
-
Jun 30
Microbiology: Transmission of gastrointestinal viruses in salivaNature
-
Jun 29
COVID-19: Assessing instances of long COVID in UK health dataNature Communications
-
Jun 24
Palaeontology: It sucked to be the prey of ancient cephalopodsScientific Reports
-
Jun 24
Sport science: New wearable sensor to measure neck strain may detect potential concussionScientific Reports