Title:A simple way to assess inference methods

Abstract:We propose a simple way to assess whether inference methods are reliable. The assessment can detect problems when the asymptotic theory that justifies the inference method is invalid and/or provides a poor approximation given the design of the empirical application. It can be easily applied to a wide range of applications. We show that, despite being a simple idea and despite its limitations, this assessment has the potential of substantially reducing the number of papers that are published based on misleading inference, if it becomes widely used by applied researchers. We analyze in detail the cases of differences-in-differences with few treated clusters, stratified experiments, shift-share designs, weighted OLS, and matching estimators.