-
Go Figure: Transparency in neuroscience images preserves context and clarifies interpretation
Authors:
Paul A. Taylor,
Himanshu Aggarwal,
Peter Bandettini,
Marco Barilari,
Molly Bright,
Cesar Caballero-Gaudes,
Vince Calhoun,
Mallar Chakravarty,
Gabriel Devenyi,
Jennifer Evans,
Eduardo Garza-Villarreal,
Jalil Rasgado-Toledo,
Remi Gau,
Daniel Glen,
Rainer Goebel,
Javier Gonzalez-Castillo,
Omer Faruk Gulban,
Yaroslav Halchenko,
Daniel Handwerker,
Taylor Hanayik,
Peter Lauren,
David Leopold,
Jason Lerch,
Christian Mathys,
Paul McCarthy
, et al. (17 additional authors not shown)
Abstract:
Visualizations are vital for communicating scientific results. Historically, neuroimaging figures have only depicted regions that surpass a given statistical threshold. This practice substantially biases interpretation of the results and subsequent meta-analyses, particularly towards non-reproducibility. Here we advocate for a "transparent thresholding" approach that not only highlights statistica…
▽ More
Visualizations are vital for communicating scientific results. Historically, neuroimaging figures have only depicted regions that surpass a given statistical threshold. This practice substantially biases interpretation of the results and subsequent meta-analyses, particularly towards non-reproducibility. Here we advocate for a "transparent thresholding" approach that not only highlights statistically significant regions but also includes subthreshold locations, which provide key experimental context. This balances the dual needs of distilling modeling results and enabling informed interpretations for modern neuroimaging. We present four examples that demonstrate the many benefits of transparent thresholding, including: removing ambiguity, decreasing hypersensitivity to non-physiological features, catching potential artifacts, improving cross-study comparisons, reducing non-reproducibility biases, and clarifying interpretations. We also demonstrate the many software packages that implement transparent thresholding, several of which were added or streamlined recently as part of this work. A point-counterpoint discussion addresses issues with thresholding raised in real conversations with researchers in the field. We hope that by showing how transparent thresholding can drastically improve the interpretation (and reproducibility) of neuroimaging findings, more researchers will adopt this method.
△ Less
Submitted 10 April, 2025;
originally announced April 2025.
-
Processing, evaluating and understanding FMRI data with afni_proc.py
Authors:
Richard C. Reynolds,
Daniel R. Glen,
Gang Chen,
Ziad S. Saad,
Robert W. Cox,
Paul A. Taylor
Abstract:
FMRI data are noisy, complicated to acquire, and typically go through many steps of processing before they are used in a study or clinical practice. Being able to visualize and understand the data from the start through the completion of processing, while being confident that each intermediate step was successful, is challenging. AFNI's afni_proc$.$py is a tool to create and run a processing pipel…
▽ More
FMRI data are noisy, complicated to acquire, and typically go through many steps of processing before they are used in a study or clinical practice. Being able to visualize and understand the data from the start through the completion of processing, while being confident that each intermediate step was successful, is challenging. AFNI's afni_proc$.$py is a tool to create and run a processing pipeline for FMRI data. With its flexible features, afni_proc$.$py allows users to both control and evaluate their processing at a detailed level. It has been designed to keep users informed about all processing steps: it does not just process the data, but first outputs a fully commented processing script that the users can read, query, interpret and refer back to. Having this full provenance is important for being able to understand each step of processing; it also promotes transparency and reproducibility by keeping the record of individual-level processing and modeling specifics in a single, shareable place. Additionally, afni_proc$.$py creates pipelines that contain several automatic self-checks for potential problems during runtime. The output directory contains a dictionary of relevant quantities that can be programmatically queried for potential issues and a systematic, interactive quality control (QC) HTML. All of these features help users evaluate and understand their data and processing in detail. We describe these and other aspects of afni_proc$.$py here using a set of task-based and resting state FMRI example commands.
△ Less
Submitted 22 August, 2024; v1 submitted 7 June, 2024;
originally announced June 2024.
-
FMRI Clustering and False Positive Rates
Authors:
Robert W. Cox,
Gang Chen,
Daniel R. Glen,
Richard C. Reynolds,
Paul A. Taylor
Abstract:
Recently, Eklund et al. (2016) analyzed clustering methods in standard FMRI packages: AFNI (which we maintain), FSL, and SPM [1]. They claimed: 1) false positive rates (FPRs) in traditional approaches are greatly inflated, questioning the validity of "countless published fMRI studies"; 2) nonparametric methods produce valid, but slightly conservative, FPRs; 3) a common flawed assumption is that th…
▽ More
Recently, Eklund et al. (2016) analyzed clustering methods in standard FMRI packages: AFNI (which we maintain), FSL, and SPM [1]. They claimed: 1) false positive rates (FPRs) in traditional approaches are greatly inflated, questioning the validity of "countless published fMRI studies"; 2) nonparametric methods produce valid, but slightly conservative, FPRs; 3) a common flawed assumption is that the spatial autocorrelation function (ACF) of FMRI noise is Gaussian-shaped; and 4) a 15-year-old bug in AFNI's 3dClustSim significantly contributed to producing "particularly high" FPRs compared to other software. We repeated simulations from [1] (Beijing-Zang data [2], see [3]), and comment on each point briefly.
△ Less
Submitted 15 February, 2017;
originally announced February 2017.
-
FMRI Clustering in AFNI: False Positive Rates Redux
Authors:
Robert W. Cox,
Gang Chen,
Daniel R. Glen,
Richard C. Reynolds,
Paul A. Taylor
Abstract:
Recent reports of inflated false positive rates (FPRs) in FMRI group analysis tools by Eklund et al. (2016) have become a large topic within (and outside) neuroimaging. They concluded that: existing parametric methods for determining statistically significant clusters had greatly inflated FPRs ("up to 70%," mainly due to the faulty assumption that the noise spatial autocorrelation function is Gaus…
▽ More
Recent reports of inflated false positive rates (FPRs) in FMRI group analysis tools by Eklund et al. (2016) have become a large topic within (and outside) neuroimaging. They concluded that: existing parametric methods for determining statistically significant clusters had greatly inflated FPRs ("up to 70%," mainly due to the faulty assumption that the noise spatial autocorrelation function is Gaussian- shaped and stationary), calling into question potentially "countless" previous results; in contrast, nonparametric methods, such as their approach, accurately reflected nominal 5% FPRs. They also stated that AFNI showed "particularly high" FPRs compared to other software, largely due to a bug in 3dClustSim. We comment on these points using their own results and figures and by repeating some of their simulations. Briefly, while parametric methods show some FPR inflation in those tests (and assumptions of Gaussian-shaped spatial smoothness also appear to be generally incorrect), their emphasis on reporting the single worst result from thousands of simulation cases greatly exaggerated the scale of the problem. Importantly, FPR statistics depend on "task" paradigm and voxelwise p-value threshold; as such, we show how results of their study provide useful suggestions for FMRI study design and analysis, rather than simply a catastrophic downgrading of the field's earlier results. Regarding AFNI (which we maintain), 3dClustSim's bug-effect was greatly overstated - their own results show that AFNI results were not "particularly" worse than others. We describe further updates in AFNI for characterizing spatial smoothness more appropriately (greatly reducing FPRs, though some remain >5%); additionally, we outline two newly implemented permutation/randomization-based approaches producing FPRs clustered much more tightly about 5% for voxelwise p<=0.01.
△ Less
Submitted 15 February, 2017;
originally announced February 2017.
