The Dangers of Double Dipping

Techniques such as fMRI, which provides images of the brain, and electrophysiology, which record electrical activity from many neurons at once, generate a large amount of data. Some of this data is intrinsically 'noisy', being accompanied by other background activity, like listening to a radio station with poor signal quality. In order to enhance the signal of interest, the raw data is substantially analyzed, often using complex transformations.

Nikolaus Kriegeskorte and Chris Baker analysed some artificially generated 'noise' data, unrelated to experimental variables, but still obtained results This shows that in many cases, analysis can generate spurious results entirely unconnected with real experimental results.

These spurious results arise from a practice the authors call 'double dipping'. For example, scientists may hypothesize that a brain region responds more strongly to one particular stimulus compared to another. An incorrect, 'double dipping' analysis would look for areas that were more active with the original stimulus and selectively analyze only these areas to test the hypothesis. However, some areas may activate more to one stimulus than another purely by chance, and restricting further analysis only to these areas would yield a misleading result.

Author contact:

Nikolaus Kriegeskorte (National Institutes of Health, Bethesda, MD, USA)
E-mail: nikokriegeskorte@gmail.com

Abstract available online.

(C) Nature Neuroscience press release.

Message posted by: Trevor M. D'Souza