Showing 1–1 of 1 results for author: Brinda, W D

Finite-sample risk bounds for maximum likelihood estimation with arbitrary penalties

Authors: W. D. Brinda, Jason M. Klusowski

Abstract: The MDL two-part coding $ \textit{index of resolvability} $ provides a finite-sample upper bound on the statistical risk of penalized likelihood estimators over countable models. However, the bound does not apply to unpenalized maximum likelihood estimation or procedures with exceedingly small penalties. In this paper, we point out a more general inequality that holds for arbitrary penalties. In a… ▽ More The MDL two-part coding $ \textit{index of resolvability} $ provides a finite-sample upper bound on the statistical risk of penalized likelihood estimators over countable models. However, the bound does not apply to unpenalized maximum likelihood estimation or procedures with exceedingly small penalties. In this paper, we point out a more general inequality that holds for arbitrary penalties. In addition, this approach makes it possible to derive exact risk bounds of order $1/n$ for iid parametric models, which improves on the order $(\log n)/n$ resolvability bounds. We conclude by discussing implications for adaptive estimation. △ Less

Submitted 28 December, 2017; originally announced December 2017.

Comments: To appear in IEEE Transactions on Information Theory, 2018

MSC Class: 62B10 94A20 94A15 62E17