Valid Double-Dipping via Permutation-Based Closed Testing

Functional Magnetic Resonance Imaging (fMRI) cluster analysis is widely popular for finding neural activation associated with some stimulus. However, it suffers from the spatial specificity paradox, and making follow-up inference inside clusters is not allowed. Valid double-dipping can be performed by closed testing, which determines lower confidence bounds for the number of active voxels, simultaneously over all regions. Moreover, a permutation framework adapts to the unknown joint distribution of the data. In the fMRI context, we evaluate two methods that rely on closed testing and permutations: permutation-based true discovery guarantee by sum tests, and permutation-based All-Resolutions Inference.