The piranha problem: Large effects swimming in a small pond
Authors:
Christopher Tosh,
Philip Greengard,
Ben Goodrich,
Andrew Gelman,
Aki Vehtari,
Daniel Hsu
Abstract:
In some scientific fields, it is common to have certain variables of interest that are of particular importance and for which there are many studies indicating a relationship with different explanatory variables. In such cases, particularly those where no relationships are known among the explanatory variables, it is worth asking under what conditions it is possible for all such claimed effects to…
▽ More
In some scientific fields, it is common to have certain variables of interest that are of particular importance and for which there are many studies indicating a relationship with different explanatory variables. In such cases, particularly those where no relationships are known among the explanatory variables, it is worth asking under what conditions it is possible for all such claimed effects to exist simultaneously. This paper addresses this question by reviewing some theorems from multivariate analysis showing that, unless the explanatory variables also have sizable dependencies with each other, it is impossible to have many such large effects. We discuss implications for the replication crisis in social science.
△ Less
Submitted 23 July, 2024; v1 submitted 27 May, 2021;
originally announced May 2021.
