Skip to main content

Showing 1–6 of 6 results for author: Podkopaev, A

Searching in archive stat. Search in all archives.
.
  1. arXiv:2412.19318  [pdf, other

    stat.ML cs.LG

    Adaptive Conformal Inference by Betting

    Authors: Aleksandr Podkopaev, Darren Xu, Kuang-Chih Lee

    Abstract: Conformal prediction is a valuable tool for quantifying predictive uncertainty of machine learning models. However, its applicability relies on the assumption of data exchangeability, a condition which is often not met in real-world scenarios. In this paper, we consider the problem of adaptive conformal inference without any assumptions about the data generating process. Existing approaches for ad… ▽ More

    Submitted 26 December, 2024; originally announced December 2024.

  2. arXiv:2305.00143  [pdf, other

    stat.ML cs.LG math.ST stat.ME

    Sequential Predictive Two-Sample and Independence Testing

    Authors: Aleksandr Podkopaev, Aaditya Ramdas

    Abstract: We study the problems of sequential nonparametric two-sample and independence testing. Sequential tests process data online and allow using observed data to decide whether to stop and reject the null hypothesis or to collect more data, while maintaining type I error control. We build upon the principle of (nonparametric) testing by betting, where a gambler places bets on future observations and th… ▽ More

    Submitted 19 July, 2023; v1 submitted 28 April, 2023; originally announced May 2023.

  3. arXiv:2212.07383  [pdf, other

    stat.ML cs.LG math.ST stat.ME

    Sequential Kernelized Independence Testing

    Authors: Aleksandr Podkopaev, Patrick Blöbaum, Shiva Prasad Kasiviswanathan, Aaditya Ramdas

    Abstract: Independence testing is a classical statistical problem that has been extensively studied in the batch setting when one fixes the sample size before collecting data. However, practitioners often prefer procedures that adapt to the complexity of a problem at hand instead of setting sample size in advance. Ideally, such procedures should (a) stop earlier on easy tasks (and later on harder tasks), he… ▽ More

    Submitted 20 May, 2025; v1 submitted 14 December, 2022; originally announced December 2022.

    Comments: ICML 2023

  4. arXiv:2110.06177  [pdf, other

    stat.ML cs.LG

    Tracking the risk of a deployed model and detecting harmful distribution shifts

    Authors: Aleksandr Podkopaev, Aaditya Ramdas

    Abstract: When deployed in the real world, machine learning models inevitably encounter changes in the data distribution, and certain -- but not all -- distribution shifts could result in significant performance degradation. In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially, making interventions by a human expert (or model… ▽ More

    Submitted 5 May, 2022; v1 submitted 12 October, 2021; originally announced October 2021.

    Comments: Accepted as a conference paper at ICLR 2022

  5. arXiv:2103.03323  [pdf, other

    stat.ML cs.LG

    Distribution-free uncertainty quantification for classification under label shift

    Authors: Aleksandr Podkopaev, Aaditya Ramdas

    Abstract: Trustworthy deployment of ML models requires a proper measure of uncertainty, especially in safety-critical applications. We focus on uncertainty quantification (UQ) for classification problems via two avenues -- prediction sets using conformal prediction and calibration of probabilistic predictors by post-hoc binning -- since these possess distribution-free guarantees for i.i.d. data. Two common… ▽ More

    Submitted 7 July, 2021; v1 submitted 4 March, 2021; originally announced March 2021.

  6. arXiv:2006.10564  [pdf, other

    stat.ML cs.AI cs.LG math.ST stat.ME

    Distribution-free binary classification: prediction sets, confidence intervals and calibration

    Authors: Chirag Gupta, Aleksandr Podkopaev, Aaditya Ramdas

    Abstract: We study three notions of uncertainty quantification -- calibration, confidence intervals and prediction sets -- for binary classification in the distribution-free setting, that is without making any distributional assumptions on the data. With a focus towards calibration, we establish a 'tripod' of theorems that connect these three notions for score-based classifiers. A direct implication is that… ▽ More

    Submitted 16 February, 2022; v1 submitted 18 June, 2020; originally announced June 2020.

    Comments: 34 pages; significant updates from previous version (unambiguous notation, better exposition, and cleaner results); originally appeared as a spotlight at Neural Information Processing Systems (NeurIPS) '20