Showing 1–6 of 6 results for author: Pérez-Ortiz, M F

Anytime-Valid Tests for Sparse Anomalies

Authors: Muriel F. Pérez-Ortiz, Rui M. Castro

Abstract: We consider the problem of detection of sparse anomalies when monitoring a large number of data streams continuously in time. This problem is addressed using anytime-valid tests. In the context of a normal-means model and for a fixed sample, this problem is known to exhibit a nontrivial phase transition that characterizes when anomalies can and cannot be detected. We show, for the anytime-valid ve… ▽ More We consider the problem of detection of sparse anomalies when monitoring a large number of data streams continuously in time. This problem is addressed using anytime-valid tests. In the context of a normal-means model and for a fixed sample, this problem is known to exhibit a nontrivial phase transition that characterizes when anomalies can and cannot be detected. We show, for the anytime-valid version of the problem, testing procedures that can detect the presence of anomalies quickly. Given that the goal is quick detection, existing approaches to anytime-valid testing that study how evidence accumulates for large times through log-optimality criteria is insufficient. This issue is addressed in this context by studying log-optimal procedures for a fixed moment in time, but as the number of streams grows larger. The resulting characterization is related to, but not implied by the existing results for fixed-sample tests. In addition, we also construct and analyze tests that are parameter-adaptive and exhibit optimal performance (in a well defined sense) even when the hypothesized model parameters are unknown. Numerical results illustrate the behavior of the proposed tests in comparison with oracle tests and suitable benchmarks. △ Less

Submitted 27 June, 2025; originally announced June 2025.

A Generalisation of Ville's Inequality to Monotonic Lower Bounds and Thresholds

Authors: Wouter M. Koolen, Muriel Felipe Pérez-Ortiz, Tyron Lardy

Abstract: Essentially all anytime-valid methods hinge on Ville's inequality to gain validity across time without incurring a union bound. Ville's inequality is a proper generalisation of Markov's inequality. It states that a non-negative supermartingale will only ever reach a multiple of its initial value with small probability. In the classic rendering both the lower bound (of zero) and the threshold are c… ▽ More Essentially all anytime-valid methods hinge on Ville's inequality to gain validity across time without incurring a union bound. Ville's inequality is a proper generalisation of Markov's inequality. It states that a non-negative supermartingale will only ever reach a multiple of its initial value with small probability. In the classic rendering both the lower bound (of zero) and the threshold are constant in time. We generalise both to monotonic curves. That is, we bound the probability that a supermartingale which remains above a given decreasing curve exceeds a given increasing threshold curve. We show our bound is tight by exhibiting a supermartingale for which the bound is an equality. Using our generalisation, we derive a clean finite-time version of the law of the iterated logarithm. △ Less

Submitted 21 February, 2025; originally announced February 2025.

Anytime-Valid Tests of Group Invariance through Conformal Prediction

Authors: Tyron Lardy, Muriel Felipe Pérez-Ortiz

Abstract: We develop anytime-valid tests of invariance under the action of compact groups. The resulting test statistics are optimal in a logarithmic-growth sense. We apply our method to extend recent anytime-valid tests of independence and to construct tests of normality. We develop anytime-valid tests of invariance under the action of compact groups. The resulting test statistics are optimal in a logarithmic-growth sense. We apply our method to extend recent anytime-valid tests of independence and to construct tests of normality. △ Less

Submitted 23 May, 2024; v1 submitted 27 January, 2024; originally announced January 2024.

Exponential Stochastic Inequality

Authors: Peter D. Grünwald, Muriel F. Pérez-Ortiz, Zakaria Mhammedi

Abstract: We develop the concept of exponential stochastic inequality (ESI), a novel notation that simultaneously captures high-probability and in-expectation statements. It is especially well suited to succinctly state, prove, and reason about excess-risk and generalization bounds in statistical learning, specifically, but not restricted to, the PAC-Bayesian type. We show that the ESI satisfies transitivit… ▽ More We develop the concept of exponential stochastic inequality (ESI), a novel notation that simultaneously captures high-probability and in-expectation statements. It is especially well suited to succinctly state, prove, and reason about excess-risk and generalization bounds in statistical learning, specifically, but not restricted to, the PAC-Bayesian type. We show that the ESI satisfies transitivity and other properties which allow us to use it like standard, nonstochastic inequalities. We substantially extend the original definition from Koolen et al. (2016) and show that general ESIs satisfy a host of useful additional properties, including a novel Markov-like inequality. We show how ESIs relate to, and clarify, PAC-Bayesian bounds, subcentered subgamma random variables and *fast-rate conditions* such as the central and Bernstein conditions. We also show how the ideas can be extended to random scaling factors (learning rates). △ Less

Submitted 27 April, 2023; originally announced April 2023.

E-Statistics, Group Invariance and Anytime Valid Testing

Authors: Muriel Felipe Pérez-Ortiz, Tyron Lardy, Rianne de Heide, Peter Grünwald

Abstract: We study worst-case-growth-rate-optimal (GROW) e-statistics for hypothesis testing between two group models. It is known that under a mild condition on the action of the underlying group G on the data, there exists a maximally invariant statistic. We show that among all e-statistics, invariant or not, the likelihood ratio of the maximally invariant statistic is GROW, both in the absolute and in th… ▽ More We study worst-case-growth-rate-optimal (GROW) e-statistics for hypothesis testing between two group models. It is known that under a mild condition on the action of the underlying group G on the data, there exists a maximally invariant statistic. We show that among all e-statistics, invariant or not, the likelihood ratio of the maximally invariant statistic is GROW, both in the absolute and in the relative sense, and that an anytime-valid test can be based on it. The GROW e-statistic is equal to a Bayes factor with a right Haar prior on G. Our treatment avoids nonuniqueness issues that sometimes arise for such priors in Bayesian contexts. A crucial assumption on the group G is its amenability, a well-known group-theoretical condition, which holds, for instance, in scale-location families. Our results also apply to finite-dimensional linear regression. △ Less

Submitted 17 October, 2023; v1 submitted 16 August, 2022; originally announced August 2022.

Comments: 30 pages. Major rewrite of previous version. Submitted to the Annals of Statistics

The Anytime-Valid Logrank Test: Error Control Under Continuous Monitoring with Unlimited Horizon

Authors: J. ter Schure, M. F. Perez-Ortiz, A. Ly, P. Grunwald

Abstract: We introduce the anytime-valid (AV) logrank test, a version of the logrank test that provides type-I error guarantees under optional stopping and optional continuation. The test is sequential without the need to specify a maximum sample size or stopping rule, and allows for cumulative meta-analysis with type-I error control. The method can be extended to define anytime-valid confidence intervals.… ▽ More We introduce the anytime-valid (AV) logrank test, a version of the logrank test that provides type-I error guarantees under optional stopping and optional continuation. The test is sequential without the need to specify a maximum sample size or stopping rule, and allows for cumulative meta-analysis with type-I error control. The method can be extended to define anytime-valid confidence intervals. The logrank test is an instance of the martingale tests based on E-variables that have been recently developed. We demonstrate type-I error guarantees for the test in a semiparametric setting of proportional hazards and show how to extend it to ties, Cox' regression and confidence sequences. Using a Gaussian approximation on the logrank statistic, we show that the AV logrank test (which itself is always exact) has a similar rejection region to O'Brien-Fleming alpha-spending but with the potential to achieve 100% power by optional continuation. Although our approach to study design requires a larger sample size, the *expected* sample size is competitive by optional stopping. △ Less

Submitted 1 May, 2023; v1 submitted 13 November, 2020; originally announced November 2020.