Showing 1–50 of 427 results for author: Wu, Y

Small Area Estimation of Fertility in Low- and Middle-Income Countries

Authors: Yunhan Wu, Jon Wakefield

Abstract: Accurate fertility estimates at fine spatial resolution are essential for localized public health planning, particularly in low- and middle-income countries (LMICs). While national-level indicators such as age-specific fertility rates (ASFR) and total fertility rate (TFR) are often reported through official statistics, they lack the spatial granularity needed to guide targeted interventions. To ad… ▽ More Accurate fertility estimates at fine spatial resolution are essential for localized public health planning, particularly in low- and middle-income countries (LMICs). While national-level indicators such as age-specific fertility rates (ASFR) and total fertility rate (TFR) are often reported through official statistics, they lack the spatial granularity needed to guide targeted interventions. To address this, we develop a framework for subnational fertility estimation using small-area estimation (SAE) techniques applied to birth history data from household surveys, in particular Demographic and Health Surveys (DHS). Disaggregation by geographic area, time period, and maternal age group leads to significant data sparsity, limiting the reliability of direct estimates at fine scales. To overcome this, we propose a suite of methods, including direct estimators, area-level and unit-level Bayesian hierarchical models, to produce accurate estimates across varying spatial resolutions. The model-based approaches incorporate spatiotemporal smoothing and integrate covariates such as maternal education, contraceptive use and urbanicity. Using data from the 2021 Madagascar DHS, we generate district-level ASFR and TFR estimates and evaluate model performance through cross-validation. △ Less

Submitted 4 July, 2025; originally announced July 2025.

Comments: 35 pages and 13 figures

Simulated Intervention on Cross-Sectional Nested Data: Development of a Multilevel NIRA Approach

Authors: Yiming Wu, Fei Wang

Abstract: With the rise of the network perspective, researchers have made numerous important discoveries over the past decade by constructing psychological networks. Unfortunately, most of these networks are based on cross-sectional data, which can only reveal associations between variables but not their directional or causal relationships. Recently, the development of the nodeIdentifyR algorithm (NIRA) tec… ▽ More With the rise of the network perspective, researchers have made numerous important discoveries over the past decade by constructing psychological networks. Unfortunately, most of these networks are based on cross-sectional data, which can only reveal associations between variables but not their directional or causal relationships. Recently, the development of the nodeIdentifyR algorithm (NIRA) technique has provided a promising method for simulating causal processes based on cross-sectional network structures. However, this algorithm is not capable of handling cross-sectional nested data, which greatly limits its applicability. In response to this limitation, the present study proposes a multilevel extension of the NIRA algorithm, referred to as multilevel NIRA. We provide a detailed explanation of the algorithm's core principles and modeling procedures. Finally, we discuss the potential applications and practical implications of this approach, as well as its limitations and directions for future research. △ Less

Submitted 27 June, 2025; originally announced June 2025.

POLAR: A Pessimistic Model-based Policy Learning Algorithm for Dynamic Treatment Regimes

Authors: Ruijia Zhang, Zhengling Qi, Yue Wu, Xiangyu Zhang, Yanxun Xu

Abstract: Dynamic treatment regimes (DTRs) provide a principled framework for optimizing sequential decision-making in domains where decisions must adapt over time in response to individual trajectories, such as healthcare, education, and digital interventions. However, existing statistical methods often rely on strong positivity assumptions and lack robustness under partial data coverage, while offline rei… ▽ More Dynamic treatment regimes (DTRs) provide a principled framework for optimizing sequential decision-making in domains where decisions must adapt over time in response to individual trajectories, such as healthcare, education, and digital interventions. However, existing statistical methods often rely on strong positivity assumptions and lack robustness under partial data coverage, while offline reinforcement learning approaches typically focus on average training performance, lack statistical guarantees, and require solving complex optimization problems. To address these challenges, we propose POLAR, a novel pessimistic model-based policy learning algorithm for offline DTR optimization. POLAR estimates the transition dynamics from offline data and quantifies uncertainty for each history-action pair. A pessimistic penalty is then incorporated into the reward function to discourage actions with high uncertainty. Unlike many existing methods that focus on average training performance, POLAR directly targets the suboptimality of the final learned policy and offers theoretical guarantees, without relying on computationally intensive minimax or constrained optimization procedures. To the best of our knowledge, POLAR is the first model-based DTR method to provide both statistical and computational guarantees, including finite-sample bounds on policy suboptimality. Empirical results on both synthetic data and the MIMIC-III dataset demonstrate that POLAR outperforms state-of-the-art methods and yields near-optimal, history-aware treatment strategies. △ Less

Submitted 25 June, 2025; originally announced June 2025.

Learning to Lead: Incentivizing Strategic Agents in the Dark

Authors: Yuchen Wu, Xinyi Zhong, Zhuoran Yang

Abstract: We study an online learning version of the generalized principal-agent model, where a principal interacts repeatedly with a strategic agent possessing private types, private rewards, and taking unobservable actions. The agent is non-myopic, optimizing a discounted sum of future rewards and may strategically misreport types to manipulate the principal's learning. The principal, observing only her o… ▽ More We study an online learning version of the generalized principal-agent model, where a principal interacts repeatedly with a strategic agent possessing private types, private rewards, and taking unobservable actions. The agent is non-myopic, optimizing a discounted sum of future rewards and may strategically misreport types to manipulate the principal's learning. The principal, observing only her own realized rewards and the agent's reported types, aims to learn an optimal coordination mechanism that minimizes strategic regret. We develop the first provably sample-efficient algorithm for this challenging setting. Our approach features a novel pipeline that combines (i) a delaying mechanism to incentivize approximately myopic agent behavior, (ii) an innovative reward angle estimation framework that uses sector tests and a matching procedure to recover type-dependent reward functions, and (iii) a pessimistic-optimistic LinUCB algorithm that enables the principal to explore efficiently while respecting the agent's incentive constraints. We establish a near optimal $\tilde{O}(\sqrt{T}) $ regret bound for learning the principal's optimal policy, where $\tilde{O}(\cdot) $ omits logarithmic factors. Our results open up new avenues for designing robust online learning algorithms for a wide range of game-theoretic settings involving private types and strategic agents. △ Less

Submitted 10 June, 2025; originally announced June 2025.

Comments: 81 pages, 7 figures

Variable Selection in Functional Linear Cox Model

Authors: Yuanzhen Yue, Stella Self, Yichao Wu, Jiajia Zhang, Rahul Ghosal

Abstract: Modern biomedical studies frequently collect complex, high-dimensional physiological signals using wearables and sensors along with time-to-event outcomes, making efficient variable selection methods crucial for interpretation and improving the accuracy of survival models. We propose a novel variable selection method for a functional linear Cox model with multiple functional and scalar covariates… ▽ More Modern biomedical studies frequently collect complex, high-dimensional physiological signals using wearables and sensors along with time-to-event outcomes, making efficient variable selection methods crucial for interpretation and improving the accuracy of survival models. We propose a novel variable selection method for a functional linear Cox model with multiple functional and scalar covariates measured at baseline. We utilize a spline-based semiparametric estimation approach for the functional coefficients and a group minimax concave type penalty (MCP), which effectively integrates smoothness and sparsity into the estimation of functional coefficients. An efficient group descent algorithm is used for optimization, and an automated procedure is provided to select optimal values of the smoothing and sparsity parameters. Through simulation studies, we demonstrate the method's ability to perform accurate variable selection and estimation. The method is applied to 2003-06 cohort of the National Health and Nutrition Examination Survey (NHANES) data, identifying the key temporally varying distributional patterns of physical activity and demographic predictors related to all-cause mortality. Our analysis sheds light on the intricate association between daily distributional patterns of physical activity and all-cause mortality among older US adults. △ Less

Submitted 3 June, 2025; originally announced June 2025.

Towards Robust Influence Functions with Flat Validation Minima

Authors: Xichen Ye, Yifan Wu, Weizhong Zhang, Cheng Jin, Yifan Chen

Abstract: The Influence Function (IF) is a widely used technique for assessing the impact of individual training samples on model predictions. However, existing IF methods often fail to provide reliable influence estimates in deep neural networks, particularly when applied to noisy training data. This issue does not stem from inaccuracies in parameter change estimation, which has been the primary focus of p… ▽ More The Influence Function (IF) is a widely used technique for assessing the impact of individual training samples on model predictions. However, existing IF methods often fail to provide reliable influence estimates in deep neural networks, particularly when applied to noisy training data. This issue does not stem from inaccuracies in parameter change estimation, which has been the primary focus of prior research, but rather from deficiencies in loss change estimation, specifically due to the sharpness of validation risk. In this work, we establish a theoretical connection between influence estimation error, validation set risk, and its sharpness, underscoring the importance of flat validation minima for accurate influence estimation. Furthermore, we introduce a novel estimation form of Influence Function specifically designed for flat validation minima. Experimental results across various tasks validate the superiority of our approach. △ Less

Submitted 25 May, 2025; originally announced May 2025.

Comments: Accepted by ICML 2025. arXiv admin note: text overlap with arXiv:2310.00902 by other authors

Learning to Rank Chain-of-Thought: An Energy-Based Approach with Outcome Supervision

Authors: Eric Hanchen Jiang, Haozheng Luo, Shengyuan Pang, Xiaomin Li, Zhenting Qi, Hengli Li, Cheng-Fu Yang, Zongyu Lin, Xinfeng Li, Hao Xu, Kai-Wei Chang, Ying Nian Wu

Abstract: Mathematical reasoning presents a significant challenge for Large Language Models (LLMs), often requiring robust multi step logical consistency. While Chain of Thought (CoT) prompting elicits reasoning steps, it doesn't guarantee correctness, and improving reliability via extensive sampling is computationally costly. This paper introduces the Energy Outcome Reward Model (EORM), an effective, light… ▽ More Mathematical reasoning presents a significant challenge for Large Language Models (LLMs), often requiring robust multi step logical consistency. While Chain of Thought (CoT) prompting elicits reasoning steps, it doesn't guarantee correctness, and improving reliability via extensive sampling is computationally costly. This paper introduces the Energy Outcome Reward Model (EORM), an effective, lightweight, post hoc verifier. EORM leverages Energy Based Models (EBMs) to simplify the training of reward models by learning to assign a scalar energy score to CoT solutions using only outcome labels, thereby avoiding detailed annotations. It achieves this by interpreting discriminator output logits as negative energies, effectively ranking candidates where lower energy is assigned to solutions leading to correct final outcomes implicitly favoring coherent reasoning. On mathematical benchmarks (GSM8k, MATH), EORM significantly improves final answer accuracy (e.g., with Llama 3 8B, achieving 90.7% on GSM8k and 63.7% on MATH). EORM effectively leverages a given pool of candidate solutions to match or exceed the performance of brute force sampling, thereby enhancing LLM reasoning outcome reliability through its streamlined post hoc verification process. △ Less

Submitted 14 June, 2025; v1 submitted 20 May, 2025; originally announced May 2025.

Place Cells as Proximity-Preserving Embeddings: From Multi-Scale Random Walk to Straight-Forward Path Planning

Authors: Minglu Zhao, Dehong Xu, Deqian Kong, Wen-Hao Zhang, Ying Nian Wu

Abstract: The hippocampus enables spatial navigation through place cell populations forming cognitive maps. We propose proximity-preserving neural embeddings to encode multi-scale random walk transitions, where the inner product $\langle h(x, t), h(y, t) \rangle = q(y|x, t)$ represents normalized transition probabilities, with $h(x, t)$ as the embedding at location $x$ and $q(y|x, t)$ as the transition prob… ▽ More The hippocampus enables spatial navigation through place cell populations forming cognitive maps. We propose proximity-preserving neural embeddings to encode multi-scale random walk transitions, where the inner product $\langle h(x, t), h(y, t) \rangle = q(y|x, t)$ represents normalized transition probabilities, with $h(x, t)$ as the embedding at location $x$ and $q(y|x, t)$ as the transition probability at scale $\sqrt{t}$. This scale hierarchy mirrors hippocampal dorsoventral organization. The embeddings $h(x, t)$ reduce pairwise spatial proximity into an environmental map, with Euclidean distances preserving proximity information. We use gradient ascent on $q(y|x, t)$ for straight-forward path planning, employing adaptive scale selection for trap-free, smooth trajectories, equivalent to minimizing embedding space distances. Matrix squaring ($P_{2t} = P_t^2$) efficiently builds global transitions from local ones ($P_1$), enabling preplay-like shortcut prediction. Experiments demonstrate localized place fields, multi-scale tuning, adaptability, and remapping, achieving robust navigation in complex environments. Our biologically plausible framework, extensible to theta-phase precession, unifies spatial and temporal coding for scalable navigation. △ Less

Submitted 2 June, 2025; v1 submitted 20 May, 2025; originally announced May 2025.

A Hybrid Prior Bayesian Method for Combining Domestic Real-World Data and Overseas Data in Global Drug Development

Authors: Keer Chen, Zengyue Zheng, Pengfei Zhu, Shuping Jiang, Nan Li, Jumin Deng, Pingyan Chen, Zhenyu Wu, Ying Wu

Abstract: Background Hybrid clinical trial design integrates randomized controlled trials (RCTs) with real-world data (RWD) to enhance efficiency through dynamic incorporation of external data. Existing methods like the Meta-Analytic Predictive Prior (MAP) inadequately control data heterogeneity, adjust baseline discrepancies, or optimize dynamic borrowing proportions, introducing bias and limiting applicat… ▽ More Background Hybrid clinical trial design integrates randomized controlled trials (RCTs) with real-world data (RWD) to enhance efficiency through dynamic incorporation of external data. Existing methods like the Meta-Analytic Predictive Prior (MAP) inadequately control data heterogeneity, adjust baseline discrepancies, or optimize dynamic borrowing proportions, introducing bias and limiting applications in bridging trials and multi-regional clinical trials (MRCTs). Objective This study proposes a novel hybrid Bayesian framework (EQPS-rMAP) to address heterogeneity and bias in multi-source data integration, validated through simulations and retrospective case analyses of risankizumab's efficacy in moderate-to-severe plaque psoriasis. Design and Methods EQPS-rMAP eliminates baseline covariate discrepancies via propensity score stratification, constructs stratum-specific MAP priors to dynamically adjust external data weights, and introduces equivalence probability weights to quantify data conflict risks. Performance was evaluated across six simulated scenarios (heterogeneity differences, baseline shifts) and real-world case analyses, comparing it with traditional methods (MAP, PSMAP, EBMAP) on estimation bias, type I error control, and sample size requirements. Results Simulations show EQPS-rMAP maintains estimation robustness under significant heterogeneity while reducing sample size demands and enhancing trial efficiency. Case analyses confirm superior external bias control and accuracy compared to conventional approaches. Conclusion and Significance EQPS-rMAP provides empirical evidence for hybrid clinical designs. By resolving baseline-heterogeneity conflicts through adaptive mechanisms, it enables reliable integration of external and real-world data in bridging trials, MRCTs, and post-marketing studies, broadening applicability without compromising rigor. △ Less

Submitted 18 May, 2025; originally announced May 2025.

Comments: 10 figures

Framing Causal Questions in Sports Analytics: A Case Study of Crossing in Soccer

Authors: Shomoita Alam, Erica E. M. Moodie, Lucas Y. Wu, Tim B. Swartz

Abstract: Causal inference has become an accepted analytic framework in settings where experimentation is impossible, which is frequently the case in sports analytics, particularly for studying in-game tactics. However, subtle differences in implementation can lead to important differences in interpretation. In this work, we provide a case study to demonstrate the utility and the nuance of these approaches.… ▽ More Causal inference has become an accepted analytic framework in settings where experimentation is impossible, which is frequently the case in sports analytics, particularly for studying in-game tactics. However, subtle differences in implementation can lead to important differences in interpretation. In this work, we provide a case study to demonstrate the utility and the nuance of these approaches. Motivated by a case study of crossing in soccer, two causal questions are considered: the overall impact of crossing on shot creation (Average Treatment Effect, ATE) and its impact in plays where crossing was actually attempted (Average Treatment Effect on the Treated, ATT). Using data from Shandong Taishan Luneng Football Club's 2017 season, we demonstrate how distinct matching strategies are used for different estimation targets - the ATE and ATT - though both aim to eliminate any spurious relationship between crossing and shot creation. Results suggest crossing yields a 1.6% additive increase in shot probability overall compared to not crossing (ATE), whereas the ATT is 5.0%. We discuss what insights can be gained from each estimand, and provide examples where one may be preferred over the alternative. Understanding and clearly framing analytics questions through a causal lens ensure rigorous analyses of complex questions. △ Less

Submitted 17 May, 2025; originally announced May 2025.

Comments: 22 pages, 2 figures

Optimal Regret of Bernoulli Bandits under Global Differential Privacy

Authors: Achraf Azize, Yulian Wu, Junya Honda, Francesco Orabona, Shinji Ito, Debabrota Basu

Abstract: As sequential learning algorithms are increasingly applied to real life, ensuring data privacy while maintaining their utilities emerges as a timely question. In this context, regret minimisation in stochastic bandits under $ε$-global Differential Privacy (DP) has been widely studied. Unlike bandits without DP, there is a significant gap between the best-known regret lower and upper bound in this… ▽ More As sequential learning algorithms are increasingly applied to real life, ensuring data privacy while maintaining their utilities emerges as a timely question. In this context, regret minimisation in stochastic bandits under $ε$-global Differential Privacy (DP) has been widely studied. Unlike bandits without DP, there is a significant gap between the best-known regret lower and upper bound in this setting, though they "match" in order. Thus, we revisit the regret lower and upper bounds of $ε$-global DP algorithms for Bernoulli bandits and improve both. First, we prove a tighter regret lower bound involving a novel information-theoretic quantity characterising the hardness of $ε$-global DP in stochastic bandits. Our lower bound strictly improves on the existing ones across all $ε$ values. Then, we choose two asymptotically optimal bandit algorithms, i.e. DP-KLUCB and DP-IMED, and propose their DP versions using a unified blueprint, i.e., (a) running in arm-dependent phases, and (b) adding Laplace noise to achieve privacy. For Bernoulli bandits, we analyse the regrets of these algorithms and show that their regrets asymptotically match our lower bound up to a constant arbitrary close to 1. This refutes the conjecture that forgetting past rewards is necessary to design optimal bandit algorithms under global DP. At the core of our algorithms lies a new concentration inequality for sums of Bernoulli variables under Laplace mechanism, which is a new DP version of the Chernoff bound. This result is universally useful as the DP literature commonly treats the concentrations of Laplace noise and random variables separately, while we couple them to yield a tighter bound. △ Less

Submitted 8 May, 2025; originally announced May 2025.

Statistical method for pooling categorical biomarkers from multi-center matched/nested case-control studies

Authors: Yujie Wu, Xiao Wu, Mitchell H. Gail, Regina G. Ziegler, Stephanie A. Smith-Warner, Molin Wang

Abstract: Pooled analyses that aggregate data from multiple studies are becoming increasingly common in collaborative epidemiologic research in order to increase the size and diversity of the study population. However, biomarker measurements from different studies are subject to systematic measurement errors and directly pooling them for analyses may lead to biased estimates of the regression parameters. Th… ▽ More Pooled analyses that aggregate data from multiple studies are becoming increasingly common in collaborative epidemiologic research in order to increase the size and diversity of the study population. However, biomarker measurements from different studies are subject to systematic measurement errors and directly pooling them for analyses may lead to biased estimates of the regression parameters. Therefore, study-specific calibration processes must be incorporated in the statistical analyses to address between-study/assay/laboratory variability in the biomarker measurements. We propose a likelihood-based method to evaluate biomarker-disease relationships for categorical biomarkers in matched/nested case-control studies. To account for the additional uncertainties from the calibration processes, we propose a sandwich variance estimator to obtain valid asymptotic variances of the estimated regression parameters. Extensive simulation studies with varying sample sizes and biomarker-disease associations are used to evaluate the finite sample performance of our proposed methods. As an illustration, we apply the methods to a vitamin D pooling project of colorectal cancer to evaluate the effect of categorical vitamin D levels on colorectal cancer risks. △ Less

Submitted 4 May, 2025; originally announced May 2025.

Statistical methods for clustered competing risk data when the event types are only available in a training dataset

Authors: Yujie Wu, Molin Wang

Abstract: We develop methods to analyze clustered competing risks data when the event types are only available in a training dataset and are missing in the main study. We propose to estimate the exposure effects through the cause-specific proportional hazards frailty model where random effects are introduced into the model to account for the within-cluster correlation. We propose a weighted penalized partia… ▽ More We develop methods to analyze clustered competing risks data when the event types are only available in a training dataset and are missing in the main study. We propose to estimate the exposure effects through the cause-specific proportional hazards frailty model where random effects are introduced into the model to account for the within-cluster correlation. We propose a weighted penalized partial likelihood method where the weights represent the probabilities of the occurrence of events, and the weights can be obtained by fitting a classification model for the event types on the training dataset. Alternatively, we propose an imputation approach where the missing event types are imputed based on the predictions from the classification model. We derive the analytical variances, and evaluate the finite sample properties of our methods in an extensive simulation study. As an illustrative example, we apply our methods to estimate the associations between tinnitus and metabolic, sensory and metabolic+sensory hearing loss in the Conservation of Hearing Study Audiology Assessment Arm. △ Less

Submitted 4 May, 2025; originally announced May 2025.

sae4health: An R Shiny Application for Small Area Estimation in Low- and Middle-Income Countries

Authors: Yunhan Wu, Qianyu Dong, Jieyi Xu, Zehang Richard Li, Jon Wakefield

Abstract: Accurate subnational estimation of health indicators is critical for public health planning, especially in low- and middle-income countries (LMICs), where data and tools are often limited. The sae4health R shiny app, built on the surveyPrev package, provides a user-friendly tool for prevalence mapping using small area estimation (SAE) methods. Both area- and unit-level models with spatial random e… ▽ More Accurate subnational estimation of health indicators is critical for public health planning, especially in low- and middle-income countries (LMICs), where data and tools are often limited. The sae4health R shiny app, built on the surveyPrev package, provides a user-friendly tool for prevalence mapping using small area estimation (SAE) methods. Both area- and unit-level models with spatial random effects are available, with fast Bayesian inference performed using Integrated Nested Laplace Approximation (INLA). Currently, the app supports analysis of over 150 indicators from Demographic and Health Surveys (DHS) across multiple administrative levels. sae4health simplifies the use of complex prevalence mapping models to support data-driven decision-making. The app provides interactive visualization, summary, and report generation functionalities for a wide range of use cases. This paper outlines the app's statistical framework and demonstrates the workflow through a case study of child stunting in Nigeria. Additional documentation is available on the supporting website (https://sae4health.stat.uw.edu). △ Less

Submitted 1 May, 2025; originally announced May 2025.

Toward a Principled Workflow for Prevalence Mapping Using Household Survey Data

Authors: Qianyu Dong, Yunhan Wu, Zehang Richard Li, Jon Wakefield

Abstract: Understanding the prevalence of key demographic and health indicators in small geographic areas and domains is of global interest, especially in low- and middle-income countries (LMICs), where vital registration data is sparse and household surveys are the primary source of information. Recent advances in computation and the increasing availability of spatially detailed datasets have led to much p… ▽ More Understanding the prevalence of key demographic and health indicators in small geographic areas and domains is of global interest, especially in low- and middle-income countries (LMICs), where vital registration data is sparse and household surveys are the primary source of information. Recent advances in computation and the increasing availability of spatially detailed datasets have led to much progress in sophisticated statistical modeling of prevalence. As a result, high-resolution prevalence maps for many indicators are routinely produced in the literature. However, statistical and practical guidance for producing prevalence maps in LMICs has been largely lacking. In particular, advice in choosing and evaluating models and interpreting results is needed, especially when data is limited. Software and analysis tools are also usually inaccessible to researchers in low-resource settings to conduct their own analysis or reproduce findings in the literature. In this paper, we propose a general workflow for prevalence mapping using household survey data. We consider all stages of the analysis pipeline, with particular emphasis on model choice and interpretation. We illustrate the proposed workflow using a case study mapping the proportion of pregnant women who had at least four antenatal care visits in Kenya. Reproducible code is provided in the Supplementary Materials and can be readily extended to a broad collection of indicators. △ Less

Submitted 23 April, 2025; originally announced April 2025.

Comments: 34 pages, 14 figures

Smooth Calibration and Decision Making

Authors: Jason Hartline, Yifan Wu, Yunran Yang

Abstract: Calibration requires predictor outputs to be consistent with their Bayesian posteriors. For machine learning predictors that do not distinguish between small perturbations, calibration errors are continuous in predictions, e.g., smooth calibration error (Foster and Hart, 2018), Distance to Calibration (Blasiok et al., 2023a). On the contrary, decision-makers who use predictions make optimal decisi… ▽ More Calibration requires predictor outputs to be consistent with their Bayesian posteriors. For machine learning predictors that do not distinguish between small perturbations, calibration errors are continuous in predictions, e.g., smooth calibration error (Foster and Hart, 2018), Distance to Calibration (Blasiok et al., 2023a). On the contrary, decision-makers who use predictions make optimal decisions discontinuously in probabilistic space, experiencing loss from miscalibration discontinuously. Calibration errors for decision-making are thus discontinuous, e.g., Expected Calibration Error (Foster and Vohra, 1997), and Calibration Decision Loss (Hu and Wu, 2024). Thus, predictors with a low calibration error for machine learning may suffer a high calibration error for decision-making, i.e., they may not be trustworthy for decision-makers optimizing assuming their predictions are correct. It is natural to ask if post-processing a predictor with a low calibration error for machine learning is without loss to achieve a low calibration error for decision-making. In our paper, we show that post-processing an online predictor with $ε$ distance to calibration achieves $O(\sqrtε)$ ECE and CDL, which is asymptotically optimal. The post-processing algorithm adds noise to make predictions differentially private. The optimal bound from low distance to calibration predictors from post-processing is non-optimal compared with existing online calibration algorithms that directly optimize for ECE and CDL. △ Less

Submitted 22 April, 2025; originally announced April 2025.

Comments: In FORC 2025

Bayesian Rao test for distributed target detection in interference and noise with limited training data

Authors: Daipeng Xiao, Weijian Liu, Jun Liu, Yuntao Wu, Qinglei Du, Xiaoqiang Hua

Abstract: This paper has studied the problem of detecting a range-spread target in interference and noise when the number of training data is limited. The interference is located within a certain subspace with an unknown coordinate, while the noise follows a Gaussian distribution with an unknown covariance matrix. We concentrate on the scenarios where the training data are limited and employ a Bayesian fram… ▽ More This paper has studied the problem of detecting a range-spread target in interference and noise when the number of training data is limited. The interference is located within a certain subspace with an unknown coordinate, while the noise follows a Gaussian distribution with an unknown covariance matrix. We concentrate on the scenarios where the training data are limited and employ a Bayesian framework to ffnd a solution. Speciffcally, the covariance matrix is assumed to follow an inverse Wishart distribution. Then, we introduce the Bayesian detector according to the Rao test, which, demonstrated by both simulation experiment and real data, has superior detection performance to the existing detectors in certain situations. △ Less

Submitted 5 May, 2025; v1 submitted 17 April, 2025; originally announced April 2025.

Comments: 14 pages,18 figures. This manuscript has been accepted by SCIENCE CHINA Information Sciences

Testing Stochastic Block Models Based on Maximum Sampling Entry-Wise Deviations

Authors: Yujia Wu, Wei Lan, Long Feng, Chih-Ling Tsai

Abstract: The stochastic block model (SBM) has been widely used to analyze network data. Various goodness-of-fit tests have been proposed to assess the adequacy of model structures. To the best of our knowledge, however, none of the existing approaches are applicable for sparse networks in which the connection probability of any two communities is of order log n/n, and the number of communities is divergent… ▽ More The stochastic block model (SBM) has been widely used to analyze network data. Various goodness-of-fit tests have been proposed to assess the adequacy of model structures. To the best of our knowledge, however, none of the existing approaches are applicable for sparse networks in which the connection probability of any two communities is of order log n/n, and the number of communities is divergent. To fill this gap, we propose a novel goodness-of-fit test for the stochastic block model. The key idea is to construct statistics by sampling the maximum entry-deviations of the adjacency matrix that the negative impacts of network sparsity are alleviated by the sampling process. We demonstrate theoretically that the proposed test statistic converges to the Type-I extreme value distribution under the null hypothesis regardless of the network structure. Accordingly, it can be applied to both dense and sparse networks. In addition, we obtain the asymptotic power against alternatives. Moreover, we introduce a bootstrap-corrected test statistic to improve the finite sample performance, recommend an augmented test statistic to increase the power, and extend the proposed test to the degree-corrected SBM. Simulation studies and two empirical examples with both dense and sparse networks indicate that the proposed method performs well. △ Less

Submitted 15 March, 2025; originally announced March 2025.

Conformal Prediction and Human Decision Making

Authors: Jessica Hullman, Yifan Wu, Dawei Xie, Ziyang Guo, Andrew Gelman

Abstract: Methods to quantify uncertainty in predictions from arbitrary models are in demand in high-stakes domains like medicine and finance. Conformal prediction has emerged as a popular method for producing a set of predictions with specified average coverage, in place of a single prediction and confidence value. However, the value of conformal prediction sets to assist human decisions remains elusive du… ▽ More Methods to quantify uncertainty in predictions from arbitrary models are in demand in high-stakes domains like medicine and finance. Conformal prediction has emerged as a popular method for producing a set of predictions with specified average coverage, in place of a single prediction and confidence value. However, the value of conformal prediction sets to assist human decisions remains elusive due to the murky relationship between coverage guarantees and decision makers' goals and strategies. How should we think about conformal prediction sets as a form of decision support? We outline a decision theoretic framework for evaluating predictive uncertainty as informative signals, then contrast what can be said within this framework about idealized use of calibrated probabilities versus conformal prediction sets. Informed by prior empirical results and theories of human decisions under uncertainty, we formalize a set of possible strategies by which a decision maker might use a prediction set. We identify ways in which conformal prediction sets and posthoc predictive uncertainty quantification more broadly are in tension with common goals and needs in human-AI decision making. We give recommendations for future research in predictive uncertainty quantification to support human decision makers. △ Less

Submitted 18 March, 2025; v1 submitted 12 March, 2025; originally announced March 2025.

Uncertainty Quantification for LLM-Based Survey Simulations

Authors: Chengpiao Huang, Yuhang Wu, Kaizheng Wang

Abstract: We investigate the use of large language models (LLMs) to simulate human responses to survey questions, and perform uncertainty quantification to gain reliable insights. Our approach converts imperfect LLM-simulated responses into confidence sets for population parameters of human responses, addressing the distribution shift between the simulated and real populations. A key innovation lies in dete… ▽ More We investigate the use of large language models (LLMs) to simulate human responses to survey questions, and perform uncertainty quantification to gain reliable insights. Our approach converts imperfect LLM-simulated responses into confidence sets for population parameters of human responses, addressing the distribution shift between the simulated and real populations. A key innovation lies in determining the optimal number of simulated responses: too many produce overly narrow confidence sets with poor coverage, while too few yield excessively loose estimates. To resolve this, our method adaptively selects the simulation sample size, ensuring valid average-case coverage guarantees. It is broadly applicable to any LLM, irrespective of its fidelity, and any procedure for constructing confidence sets. Additionally, the selected sample size quantifies the degree of misalignment between the LLM and the target human population. We illustrate our method on real datasets and LLMs. △ Less

Submitted 26 May, 2025; v1 submitted 24 February, 2025; originally announced February 2025.

Comments: 33 pages, 7 figures, 10 tables

Local Information for Global Network Estimation in Latent Space Models

Authors: Lijia Wang, Xiao Han, Yanhui Wu, Y. X. Rachel Wang

Abstract: In social networks, neighborhood is crucial for understanding individual behavior in response to environments, and thus it is essential to analyze an individual's local perspective within the global network. This paper studies how to utilize a partial information network centered around a given individual for global network estimation by fitting a general latent space model. Compared to the entire… ▽ More In social networks, neighborhood is crucial for understanding individual behavior in response to environments, and thus it is essential to analyze an individual's local perspective within the global network. This paper studies how to utilize a partial information network centered around a given individual for global network estimation by fitting a general latent space model. Compared to the entire network, the partial information network contains a significant proportion of missing edges with its structure depending on a random, potentially sparse neighborhood, posing significant challenges for estimation. We address the challenges by proposing a projected gradient descent algorithm for maximizing the likelihood of the observed data and develop theoretical guarantees for its convergence under different neighborhood structures. Our convergence rates and estimation error bounds highlight the impact of bias in an individual's local view of the global network, and we further show that the bias can be quantified with an imbalance measure. Using simulated and real networks, we demonstrate the performance of our estimation method and how our approach enables researchers to gain additional insights into the structure of social networks, such as the tradeoff between degrees and imbalance. △ Less

Submitted 23 February, 2025; originally announced February 2025.

A Fenchel-Young Loss Approach to Data-Driven Inverse Optimization

Authors: Zhehao Li, Yanchen Wu, Xiaojie Mao

Abstract: Data-driven inverse optimization seeks to estimate unknown parameters in an optimization model from observations of optimization solutions. Many existing methods are ineffective in handling noisy and suboptimal solution observations and also suffer from computational challenges. In this paper, we build a connection between inverse optimization and the Fenchel-Young (FY) loss originally designed fo… ▽ More Data-driven inverse optimization seeks to estimate unknown parameters in an optimization model from observations of optimization solutions. Many existing methods are ineffective in handling noisy and suboptimal solution observations and also suffer from computational challenges. In this paper, we build a connection between inverse optimization and the Fenchel-Young (FY) loss originally designed for structured prediction, proposing a FY loss approach to data-driven inverse optimization. This new approach is amenable to efficient gradient-based optimization, hence much more efficient than existing methods. We provide theoretical guarantees for the proposed method and use extensive simulation and real-data experiments to demonstrate its significant advantage in parameter estimation accuracy, decision error and computational speed. △ Less

Submitted 2 April, 2025; v1 submitted 22 February, 2025; originally announced February 2025.

Small Area Estimation of Education Levels in Low- and Middle-Income Countries

Authors: Yunhan Wu, Ameer Dharamshi, Jon Wakefield

Abstract: Education is a key driver of social and economic mobility, yet disparities in attainment persist, particularly in low- and middle-income countries (LMICs). Existing indicators, such as mean years of schooling for adults aged 25 and older (MYS25) and expected years of schooling (EYS), offer a snapshot of an educational system, but lack either cohort-specific or temporal granularity. To address thes… ▽ More Education is a key driver of social and economic mobility, yet disparities in attainment persist, particularly in low- and middle-income countries (LMICs). Existing indicators, such as mean years of schooling for adults aged 25 and older (MYS25) and expected years of schooling (EYS), offer a snapshot of an educational system, but lack either cohort-specific or temporal granularity. To address these limitations, we introduce the ultimate years of schooling (UYS)-a birth cohort-based metric targeting the final educational attainment of any individual cohort, including those with ongoing schooling trajectories. As with many attainment indicators, we propose to estimate UYS with cross-sectional household surveys. However, for younger cohorts, estimation fails, because these individuals are right-censored leading to severe downwards bias. To correct for this, we propose to re-frame educational attainment as a time-to-event process and deploy discrete-time survival models that explicitly account for censoring in the observations. At the national level, we estimate the parameters of the model using survey-weighted logistic regression, while for finer spatial resolutions, where sample sizes are smaller, we embed the discrete-time survival model within a Bayesian spatiotemporal framework to improve stability and precision. Applying our proposed methods to data from the 2022 Tanzania Demographic and Health Surveys, we estimate female educational trajectories corrected for censoring biases, and reveal substantial subnational disparities. By providing a dynamic, bias-corrected, and spatially disaggregated measure, our approach enhances education monitoring; it equips policymakers and researchers with a more precise tool for monitoring current progress towards education goals, and for designing future targeted policy interventions in LMICs. △ Less

Submitted 11 February, 2025; originally announced February 2025.

Time to Rethink AI for Combinatorial Optimization: Classical Algorithms Remain Tough to Match

Authors: Yikai Wu, Haoyu Zhao, Sanjeev Arora

Abstract: This position paper argues that the machine learning community should fundamentally rethink how AI-inspired methods are developed and evaluated for combinatorial optimization (CO). We present comprehensive empirical benchmarks comparing various recent AI-inspired GPU-based methods with several classical CPU-based solvers on the Maximum Independent Set (MIS) problem. Strikingly, even on in-distribu… ▽ More This position paper argues that the machine learning community should fundamentally rethink how AI-inspired methods are developed and evaluated for combinatorial optimization (CO). We present comprehensive empirical benchmarks comparing various recent AI-inspired GPU-based methods with several classical CPU-based solvers on the Maximum Independent Set (MIS) problem. Strikingly, even on in-distribution random graphs, leading AI-inspired methods are consistently outperformed by the state-of-the-art classical solver KaMIS, and some AI-inspired methods frequently fail to surpass even the simplest degree-based greedy heuristic. To better understand the source of these failures, we introduce a novel analysis, serialization, which reveals that non-backtracking AI methods, such as LTFT (based on GFlowNets), end up reasoning similarly to the simplest degree-based greedy heuristic, and thus worse than KaMIS. Our findings reveal three core issues: (1) Limited benchmarks and evaluation - AI-inspired methods are often tested only on small instances with very limited inference time, which covers up issues with scalability and resource usage; (2) Intrinsic hardness and learning limits - even under ideal, in-distribution conditions, learning-based approaches lag behind classical heuristics, highlighting inherent barriers that receive little attention; and (3) Insufficient use and understanding of classical heuristics - current learning frameworks often neglect to incorporate effective classical techniques. Although we use MIS as a testbed, similar gaps and challenges have been reported in other combinatorial optimization problems, suggesting broader relevance for our recommendations. We propose that future research must address these issues by rigorous benchmarking, deepening understanding of learning limitations, and integrating classical heuristics into AI-inspired methods. △ Less

Submitted 29 June, 2025; v1 submitted 5 February, 2025; originally announced February 2025.

Comments: 28 pages, 6 figures, 98 tables

Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization

Authors: Yu-Han Wu, Pierre Marion, Gérard Biau, Claire Boyer

Abstract: Denoising score matching plays a pivotal role in the performance of diffusion-based generative models. However, the empirical optimal score--the exact solution to the denoising score matching--leads to memorization, where generated samples replicate the training data. Yet, in practice, only a moderate degree of memorization is observed, even without explicit regularization. In this paper, we inves… ▽ More Denoising score matching plays a pivotal role in the performance of diffusion-based generative models. However, the empirical optimal score--the exact solution to the denoising score matching--leads to memorization, where generated samples replicate the training data. Yet, in practice, only a moderate degree of memorization is observed, even without explicit regularization. In this paper, we investigate this phenomenon by uncovering an implicit regularization mechanism driven by large learning rates. Specifically, we show that in the small-noise regime, the empirical optimal score exhibits high irregularity. We then prove that, when trained by stochastic gradient descent with a large enough learning rate, neural networks cannot stably converge to a local minimum with arbitrarily small excess risk. Consequently, the learned score cannot be arbitrarily close to the empirical optimal score, thereby mitigating memorization. To make the analysis tractable, we consider one-dimensional data and two-layer neural networks. Experiments validate the crucial role of the learning rate in preventing memorization, even beyond the one-dimensional setting. △ Less

Submitted 6 May, 2025; v1 submitted 5 February, 2025; originally announced February 2025.

Latent Thought Models with Variational Bayes Inference-Time Computation

Authors: Deqian Kong, Minglu Zhao, Dehong Xu, Bo Pang, Shu Wang, Edouardo Honig, Zhangzhang Si, Chuan Li, Jianwen Xie, Sirui Xie, Ying Nian Wu

Abstract: We propose a novel class of language models, Latent Thought Models (LTMs), which incorporate explicit latent thought vectors that follow an explicit prior model in latent space. These latent thought vectors guide the autoregressive generation of ground tokens through a Transformer decoder. Training employs a dual-rate optimization process within the classical variational Bayes framework: fast lear… ▽ More We propose a novel class of language models, Latent Thought Models (LTMs), which incorporate explicit latent thought vectors that follow an explicit prior model in latent space. These latent thought vectors guide the autoregressive generation of ground tokens through a Transformer decoder. Training employs a dual-rate optimization process within the classical variational Bayes framework: fast learning of local variational parameters for the posterior distribution of latent vectors (inference-time computation), and slow learning of global decoder parameters. Empirical studies reveal that LTMs possess additional scaling dimensions beyond traditional Large Language Models (LLMs), such as the number of iterations in inference-time computation and number of latent thought vectors. Higher sample efficiency can be achieved by increasing training compute per token, with further gains possible by trading model size for more inference steps. Designed based on these scaling properties, LTMs demonstrate superior sample and parameter efficiency compared to autoregressive models and discrete diffusion models. They significantly outperform these counterparts in validation perplexity and zero-shot language modeling tasks. Additionally, LTMs exhibit emergent few-shot in-context reasoning capabilities that scale with model size, and achieve competitive performance in conditional and unconditional text generation. △ Less

Submitted 6 June, 2025; v1 submitted 3 February, 2025; originally announced February 2025.

Black-box Optimization with Simultaneous Statistical Inference for Optimal Performance

Authors: Teng Lian, Jian-Qiang Hu, Yuhang Wu, Zeyu Zheng

Abstract: Black-box optimization is often encountered for decision-making in complex systems management, where the knowledge of system is limited. Under these circumstances, it is essential to balance the utilization of new information with computational efficiency. In practice, decision-makers often face the dual tasks of optimization and statistical inference for the optimal performance, in order to achie… ▽ More Black-box optimization is often encountered for decision-making in complex systems management, where the knowledge of system is limited. Under these circumstances, it is essential to balance the utilization of new information with computational efficiency. In practice, decision-makers often face the dual tasks of optimization and statistical inference for the optimal performance, in order to achieve it with a high reliability. Our goal is to address the dual tasks in an online fashion. Wu et al (2022) [arXiv preprint: 2210.06737] point out that the sample average of performance estimates generated by the optimization algorithm needs not to admit a central limit theorem. We propose an algorithm that not only tackles this issue, but also provides an online consistent estimator for the variance of the performance. Furthermore, we characterize the convergence rate of the coverage probabilities of the asymptotic confidence intervals. △ Less

Submitted 13 January, 2025; originally announced January 2025.

Testing and Improving the Robustness of Amortized Bayesian Inference for Cognitive Models

Authors: Yufei Wu, Stefan Radev, Francis Tuerlinckx

Abstract: Contaminant observations and outliers often cause problems when estimating the parameters of cognitive models, which are statistical models representing cognitive processes. In this study, we test and improve the robustness of parameter estimation using amortized Bayesian inference (ABI) with neural networks. To this end, we conduct systematic analyses on a toy example and analyze both synthetic a… ▽ More Contaminant observations and outliers often cause problems when estimating the parameters of cognitive models, which are statistical models representing cognitive processes. In this study, we test and improve the robustness of parameter estimation using amortized Bayesian inference (ABI) with neural networks. To this end, we conduct systematic analyses on a toy example and analyze both synthetic and real data using a popular cognitive model, the Drift Diffusion Models (DDM). First, we study the sensitivity of ABI to contaminants with tools from robust statistics: the empirical influence function and the breakdown point. Next, we propose a data augmentation or noise injection approach that incorporates a contamination distribution into the data-generating process during training. We examine several candidate distributions and evaluate their performance and cost in terms of accuracy and efficiency loss relative to a standard estimator. Introducing contaminants from a Cauchy distribution during training considerably increases the robustness of the neural density estimator as measured by bounded influence functions and a much higher breakdown point. Overall, the proposed method is straightforward and practical to implement and has a broad applicability in fields where outlier detection or removal is challenging. △ Less

Submitted 29 December, 2024; originally announced December 2024.

Towards counterfactual fairness through auxiliary variables

Authors: Bowei Tian, Ziyao Wang, Shwai He, Wanghao Ye, Guoheng Sun, Yucong Dai, Yongkai Wu, Ang Li

Abstract: The challenge of balancing fairness and predictive accuracy in machine learning models, especially when sensitive attributes such as race, gender, or age are considered, has motivated substantial research in recent years. Counterfactual fairness ensures that predictions remain consistent across counterfactual variations of sensitive attributes, which is a crucial concept in addressing societal bia… ▽ More The challenge of balancing fairness and predictive accuracy in machine learning models, especially when sensitive attributes such as race, gender, or age are considered, has motivated substantial research in recent years. Counterfactual fairness ensures that predictions remain consistent across counterfactual variations of sensitive attributes, which is a crucial concept in addressing societal biases. However, existing counterfactual fairness approaches usually overlook intrinsic information about sensitive features, limiting their ability to achieve fairness while simultaneously maintaining performance. To tackle this challenge, we introduce EXOgenous Causal reasoning (EXOC), a novel causal reasoning framework motivated by exogenous variables. It leverages auxiliary variables to uncover intrinsic properties that give rise to sensitive attributes. Our framework explicitly defines an auxiliary node and a control node that contribute to counterfactual fairness and control the information flow within the model. Our evaluation, conducted on synthetic and real-world datasets, validates EXOC's superiority, showing that it outperforms state-of-the-art approaches in achieving counterfactual fairness. Our code is available at https://github.com/CASE-Lab-UMD/counterfactual_fairness_2025. △ Less

Submitted 20 February, 2025; v1 submitted 5 December, 2024; originally announced December 2024.

Comments: arXiv admin note: text overlap with arXiv:2307.08232 by other authors

Journal ref: The International Conference on Learning Representations (ICLR 2025)

Unlocking the Potential of Text-to-Image Diffusion with PAC-Bayesian Theory

Authors: Eric Hanchen Jiang, Yasi Zhang, Zhi Zhang, Yixin Wan, Andrew Lizarraga, Shufan Li, Ying Nian Wu

Abstract: Text-to-image (T2I) diffusion models have revolutionized generative modeling by producing high-fidelity, diverse, and visually realistic images from textual prompts. Despite these advances, existing models struggle with complex prompts involving multiple objects and attributes, often misaligning modifiers with their corresponding nouns or neglecting certain elements. Recent attention-based methods… ▽ More Text-to-image (T2I) diffusion models have revolutionized generative modeling by producing high-fidelity, diverse, and visually realistic images from textual prompts. Despite these advances, existing models struggle with complex prompts involving multiple objects and attributes, often misaligning modifiers with their corresponding nouns or neglecting certain elements. Recent attention-based methods have improved object inclusion and linguistic binding, but still face challenges such as attribute misbinding and a lack of robust generalization guarantees. Leveraging the PAC-Bayes framework, we propose a Bayesian approach that designs custom priors over attention distributions to enforce desirable properties, including divergence between objects, alignment between modifiers and their corresponding nouns, minimal attention to irrelevant tokens, and regularization for better generalization. Our approach treats the attention mechanism as an interpretable component, enabling fine-grained control and improved attribute-object alignment. We demonstrate the effectiveness of our method on standard benchmarks, achieving state-of-the-art results across multiple metrics. By integrating custom priors into the denoising process, our method enhances image quality and addresses long-standing challenges in T2I diffusion models, paving the way for more reliable and interpretable generative models. △ Less

Submitted 25 November, 2024; originally announced November 2024.

Emergenet: A Digital Twin of Sequence Evolution for Scalable Emergence Risk Assessment of Animal Influenza A Strains

Authors: Kevin Yuanbo Wu, Jin Li, Aaron Esser-Kahn, Ishanu Chattopadhyay

Abstract: Despite having triggered devastating pandemics in the past, our ability to quantitatively assess the emergence potential of individual strains of animal influenza viruses remains limited. This study introduces Emergenet, a tool to infer a digital twin of sequence evolution to chart how new variants might emerge in the wild. Our predictions based on Emergenets built only using 220,151 Hemagglutinni… ▽ More Despite having triggered devastating pandemics in the past, our ability to quantitatively assess the emergence potential of individual strains of animal influenza viruses remains limited. This study introduces Emergenet, a tool to infer a digital twin of sequence evolution to chart how new variants might emerge in the wild. Our predictions based on Emergenets built only using 220,151 Hemagglutinnin (HA) sequences consistently outperform WHO seasonal vaccine recommendations for H1N1/H3N2 subtypes over two decades (average match-improvement: 3.73 AAs, 28.40\%), and are at par with state-of-the-art approaches that use more detailed phenotypic annotations. Finally, our generative models are used to scalably calculate the current odds of emergence of animal strains not yet in human circulation, which strongly correlates with CDC's expert-assessed Influenza Risk Assessment Tool (IRAT) scores (Pearson's $r = 0.721, p = 10^{-4}$). A minimum five orders of magnitude speedup over CDC's assessment (seconds vs months) then enabled us to analyze 6,354 animal strains collected post-2020 to identify 35 strains with high emergence scores ($> 7.7$). The Emergenet framework opens the door to preemptive pandemic mitigation through targeted inoculation of animal hosts before the first human infection. △ Less

Submitted 26 November, 2024; originally announced November 2024.

Comments: 35 pages, 15 figures

Robust Inference for High-dimensional Linear Models with Heavy-tailed Errors via Partial Gini Covariance

Authors: Yilin Zhang, Songshan Yang, Yunan Wu, Lan Wang

Abstract: This paper introduces the partial Gini covariance, a novel dependence measure that addresses the challenges of high-dimensional inference with heavy-tailed errors, often encountered in fields like finance, insurance, climate, and biology. Conventional high-dimensional regression inference methods suffer from inaccurate type I errors and reduced power in heavy-tailed contexts, limiting their effect… ▽ More This paper introduces the partial Gini covariance, a novel dependence measure that addresses the challenges of high-dimensional inference with heavy-tailed errors, often encountered in fields like finance, insurance, climate, and biology. Conventional high-dimensional regression inference methods suffer from inaccurate type I errors and reduced power in heavy-tailed contexts, limiting their effectiveness. Our proposed approach leverages the partial Gini covariance to construct a robust statistical inference framework that requires minimal tuning and does not impose restrictive moment conditions on error distributions. Unlike traditional methods, it circumvents the need for estimating the density of random errors and enhances the computational feasibility and robustness. Extensive simulations demonstrate the proposed method's superior power and robustness over standard high-dimensional inference approaches, such as those based on the debiased Lasso. The asymptotic relative efficiency analysis provides additional theoretical insight on the improved efficiency of the new approach in the heavy-tailed setting. Additionally, the partial Gini covariance extends to the multivariate setting, enabling chi-square testing for a group of coefficients. We illustrate the method's practical application with a real-world data example. △ Less

Submitted 20 November, 2024; v1 submitted 19 November, 2024; originally announced November 2024.

A minimalistic representation model for head direction system

Authors: Minglu Zhao, Dehong Xu, Deqian Kong, Wen-Hao Zhang, Ying Nian Wu

Abstract: We present a minimalistic representation model for the head direction (HD) system, aiming to learn a high-dimensional representation of head direction that captures essential properties of HD cells. Our model is a representation of rotation group $U(1)$, and we study both the fully connected version and convolutional version. We demonstrate the emergence of Gaussian-like tuning profiles and a 2D c… ▽ More We present a minimalistic representation model for the head direction (HD) system, aiming to learn a high-dimensional representation of head direction that captures essential properties of HD cells. Our model is a representation of rotation group $U(1)$, and we study both the fully connected version and convolutional version. We demonstrate the emergence of Gaussian-like tuning profiles and a 2D circle geometry in both versions of the model. We also demonstrate that the learned model is capable of accurate path integration. △ Less

Submitted 2 June, 2025; v1 submitted 15 November, 2024; originally announced November 2024.

Comments: Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci 2025)

Conditional regression for the Nonlinear Single-Variable Model

Authors: Yantao Wu, Mauro Maggioni

Abstract: Several statistical models for regression of a function $F$ on $\mathbb{R}^d$ without the statistical and computational curse of dimensionality exist, for example by imposing and exploiting geometric assumptions on the distribution of the data (e.g. that its support is low-dimensional), or strong smoothness assumptions on $F$, or a special structure $F$. Among the latter, compositional models assu… ▽ More Several statistical models for regression of a function $F$ on $\mathbb{R}^d$ without the statistical and computational curse of dimensionality exist, for example by imposing and exploiting geometric assumptions on the distribution of the data (e.g. that its support is low-dimensional), or strong smoothness assumptions on $F$, or a special structure $F$. Among the latter, compositional models assume $F=f\circ g$ with $g$ mapping to $\mathbb{R}^r$ with $r\ll d$, have been studied, and include classical single- and multi-index models and recent works on neural networks. While the case where $g$ is linear is rather well-understood, much less is known when $g$ is nonlinear, and in particular for which $g$'s the curse of dimensionality in estimating $F$, or both $f$ and $g$, may be circumvented. In this paper, we consider a model $F(X):=f(Π_γX) $ where $Π_γ:\mathbb{R}^d\to[0,\rm{len}_γ]$ is the closest-point projection onto the parameter of a regular curve $γ: [0,\rm{len}_γ]\to\mathbb{R}^d$ and $f:[0,\rm{len}_γ]\to\mathbb{R}^1$. The input data $X$ is not low-dimensional, far from $γ$, conditioned on $Π_γ(X)$ being well-defined. The distribution of the data, $γ$ and $f$ are unknown. This model is a natural nonlinear generalization of the single-index model, which corresponds to $γ$ being a line. We propose a nonparametric estimator, based on conditional regression, and show that under suitable assumptions, the strongest of which being that $f$ is coarsely monotone, it can achieve the $one$-$dimensional$ optimal min-max rate for non-parametric regression, up to the level of noise in the observations, and be constructed in time $\mathcal{O}(d^2n\log n)$. All the constants in the learning bounds, in the minimal number of samples required for our bounds to hold, and in the computational complexity are at most low-order polynomials in $d$. △ Less

Submitted 14 November, 2024; originally announced November 2024.

Comments: 55 pages, 10 figures

MSC Class: 62G08

Network Causal Effect Estimation In Graphical Models Of Contagion And Latent Confounding

Authors: Yufeng Wu, Rohit Bhattacharya

Abstract: A key question in many network studies is whether the observed correlations between units are primarily due to contagion or latent confounding. Here, we study this question using a segregated graph (Shpitser, 2015) representation of these mechanisms, and examine how uncertainty about the true underlying mechanism impacts downstream computation of network causal effects, particularly under full int… ▽ More A key question in many network studies is whether the observed correlations between units are primarily due to contagion or latent confounding. Here, we study this question using a segregated graph (Shpitser, 2015) representation of these mechanisms, and examine how uncertainty about the true underlying mechanism impacts downstream computation of network causal effects, particularly under full interference -- settings where we only have a single realization of a network and each unit may depend on any other unit in the network. Under certain assumptions about asymptotic growth of the network, we derive likelihood ratio tests that can be used to identify whether different sets of variables -- confounders, treatments, and outcomes -- across units exhibit dependence due to contagion or latent confounding. We then propose network causal effect estimation strategies that provide unbiased and consistent estimates if the dependence mechanisms are either known or correctly inferred using our proposed tests. Together, the proposed methods allow network effect estimation in a wider range of full interference scenarios that have not been considered in prior work. We evaluate the effectiveness of our methods with synthetic data and the validity of our assumptions using real-world networks. △ Less

Submitted 5 March, 2025; v1 submitted 2 November, 2024; originally announced November 2024.

Comments: 27 pages, in proceedings of the 4th Conference on Causal Learning and Reasoning

Conditional Uncertainty Quantification for Tensorized Topological Neural Networks

Authors: Yujia Wu, Bo Yang, Yang Zhao, Elynn Chen, Yuzhou Chen, Zheshi Zheng

Abstract: Graph Neural Networks (GNNs) have become the de facto standard for analyzing graph-structured data, leveraging message-passing techniques to capture both structural and node feature information. However, recent studies have raised concerns about the statistical reliability of uncertainty estimates produced by GNNs. This paper addresses this crucial challenge by introducing a novel technique for qu… ▽ More Graph Neural Networks (GNNs) have become the de facto standard for analyzing graph-structured data, leveraging message-passing techniques to capture both structural and node feature information. However, recent studies have raised concerns about the statistical reliability of uncertainty estimates produced by GNNs. This paper addresses this crucial challenge by introducing a novel technique for quantifying uncertainty in non-exchangeable graph-structured data, while simultaneously reducing the size of label prediction sets in graph classification tasks. We propose Conformalized Tensor-based Topological Neural Networks (CF-T2NN), a new approach for rigorous prediction inference over graphs. CF-T2NN employs tensor decomposition and topological knowledge learning to navigate and interpret the inherent uncertainty in decision-making processes. This method enables a more nuanced understanding and handling of prediction uncertainties, enhancing the reliability and interpretability of neural network outcomes. Our empirical validation, conducted across 10 real-world datasets, demonstrates the superiority of CF-T2NN over a wide array of state-of-the-art methods on various graph benchmarks. This work not only enhances the GNN framework with robust uncertainty quantification capabilities but also sets a new standard for reliability and precision in graph-structured data analysis. △ Less

Submitted 19 October, 2024; originally announced October 2024.

Comments: arXiv admin note: text overlap with arXiv:2401.12007

Conditional Prediction ROC Bands for Graph Classification

Authors: Yujia Wu, Bo Yang, Elynn Chen, Yuzhou Chen, Zheshi Zheng

Abstract: Graph classification in medical imaging and drug discovery requires accuracy and robust uncertainty quantification. To address this need, we introduce Conditional Prediction ROC (CP-ROC) bands, offering uncertainty quantification for ROC curves and robustness to distributional shifts in test data. Although developed for Tensorized Graph Neural Networks (TGNNs), CP-ROC is adaptable to general Graph… ▽ More Graph classification in medical imaging and drug discovery requires accuracy and robust uncertainty quantification. To address this need, we introduce Conditional Prediction ROC (CP-ROC) bands, offering uncertainty quantification for ROC curves and robustness to distributional shifts in test data. Although developed for Tensorized Graph Neural Networks (TGNNs), CP-ROC is adaptable to general Graph Neural Networks (GNNs) and other machine learning models. We establish statistically guaranteed coverage for CP-ROC under a local exchangeability condition. This addresses uncertainty challenges for ROC curves under non-iid setting, ensuring reliability when test graph distributions differ from training data. Empirically, to establish local exchangeability for TGNNs, we introduce a data-driven approach to construct local calibration sets for graphs. Comprehensive evaluations show that CP-ROC significantly improves prediction reliability across diverse tasks. This method enhances uncertainty quantification efficiency and reliability for ROC curves, proving valuable for real-world applications with non-iid objects. △ Less

Submitted 19 October, 2024; originally announced October 2024.

Counterfactual Generative Modeling with Variational Causal Inference

Authors: Yulun Wu, Louie McConnell, Claudia Iriondo

Abstract: Estimating an individual's counterfactual outcomes under interventions is a challenging task for traditional causal inference and supervised learning approaches when the outcome is high-dimensional (e.g. gene expressions, facial images) and covariates are relatively limited. In this case, to predict one's outcomes under counterfactual treatments, it is crucial to leverage individual information co… ▽ More Estimating an individual's counterfactual outcomes under interventions is a challenging task for traditional causal inference and supervised learning approaches when the outcome is high-dimensional (e.g. gene expressions, facial images) and covariates are relatively limited. In this case, to predict one's outcomes under counterfactual treatments, it is crucial to leverage individual information contained in the observed outcome in addition to the covariates. Prior works using variational inference in counterfactual generative modeling have been focusing on neural adaptations and model variants within the conditional variational autoencoder formulation, which we argue is fundamentally ill-suited to the notion of counterfactual in causal inference. In this work, we present a novel variational Bayesian causal inference framework and its theoretical backings to properly handle counterfactual generative modeling tasks, through which we are able to conduct counterfactual supervision end-to-end during training without any counterfactual samples, and encourage disentangled exogenous noise abduction that aids the correct identification of causal effect in counterfactual generations. In experiments, we demonstrate the advantage of our framework compared to state-of-the-art models in counterfactual generative modeling on multiple benchmarks. △ Less

Submitted 18 March, 2025; v1 submitted 16 October, 2024; originally announced October 2024.

Comments: Published as a conference paper at ICLR 2025

DODT: Enhanced Online Decision Transformer Learning through Dreamer's Actor-Critic Trajectory Forecasting

Authors: Eric Hanchen Jiang, Zhi Zhang, Dinghuai Zhang, Andrew Lizarraga, Chenheng Xu, Yasi Zhang, Siyan Zhao, Zhengjie Xu, Peiyu Yu, Yuer Tang, Deqian Kong, Ying Nian Wu

Abstract: Advancements in reinforcement learning have led to the development of sophisticated models capable of learning complex decision-making tasks. However, efficiently integrating world models with decision transformers remains a challenge. In this paper, we introduce a novel approach that combines the Dreamer algorithm's ability to generate anticipatory trajectories with the adaptive learning strength… ▽ More Advancements in reinforcement learning have led to the development of sophisticated models capable of learning complex decision-making tasks. However, efficiently integrating world models with decision transformers remains a challenge. In this paper, we introduce a novel approach that combines the Dreamer algorithm's ability to generate anticipatory trajectories with the adaptive learning strengths of the Online Decision Transformer. Our methodology enables parallel training where Dreamer-produced trajectories enhance the contextual decision-making of the transformer, creating a bidirectional enhancement loop. We empirically demonstrate the efficacy of our approach on a suite of challenging benchmarks, achieving notable improvements in sample efficiency and reward maximization over existing methods. Our results indicate that the proposed integrated framework not only accelerates learning but also showcases robustness in diverse and dynamic scenarios, marking a significant step forward in model-based reinforcement learning. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Sequential Design with Derived Win Statistics

Authors: Baoshan Zhang, Yuan Wu

Abstract: The Win Ratio has gained significant traction in cardiovascular trials as a novel method for analyzing composite endpoints (Pocock and others, 2012). Compared with conventional approaches based on time to the first event, the Win Ratio accommodates the varying priorities and types of outcomes among components, potentially offering greater statistical power by fully utilizing the information contai… ▽ More The Win Ratio has gained significant traction in cardiovascular trials as a novel method for analyzing composite endpoints (Pocock and others, 2012). Compared with conventional approaches based on time to the first event, the Win Ratio accommodates the varying priorities and types of outcomes among components, potentially offering greater statistical power by fully utilizing the information contained within each outcome. However, studies using Win Ratio have largely been confined to fixed design, limiting flexibility for early decisions, such as stopping for futility or efficacy. Our study proposes a sequential design framework incorporating multiple interim analyses based on Win Ratio or Net Benefit statistics. Moreover, we provide rigorous proof of the canonical joint distribution for sequential Win Ratio and Net Benefit statistics, and an algorithm for sample size determination is developed. We also provide results from a finite sample simulation study, which show that our proposed method controls Type I error maintains power level, and has a smaller average sample size than the fixed design. A real study of cardiovascular study is applied to illustrate the proposed method. △ Less

Submitted 8 October, 2024; originally announced October 2024.

Comments: 36 pages, 1 figure, 5 tables

Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models

Authors: Yuchen Wu, Yuxin Chen, Yuting Wei

Abstract: Diffusion models play a pivotal role in contemporary generative modeling, claiming state-of-the-art performance across various domains. Despite their superior sample quality, mainstream diffusion-based stochastic samplers like DDPM often require a large number of score function evaluations, incurring considerably higher computational cost compared to single-step generators like generative adversar… ▽ More Diffusion models play a pivotal role in contemporary generative modeling, claiming state-of-the-art performance across various domains. Despite their superior sample quality, mainstream diffusion-based stochastic samplers like DDPM often require a large number of score function evaluations, incurring considerably higher computational cost compared to single-step generators like generative adversarial networks. While several acceleration methods have been proposed in practice, the theoretical foundations for accelerating diffusion models remain underexplored. In this paper, we propose and analyze a training-free acceleration algorithm for SDE-style diffusion samplers, based on the stochastic Runge-Kutta method. The proposed sampler provably attains $\varepsilon^2$ error -- measured in KL divergence -- using $\widetilde O(d^{3/2} / \varepsilon)$ score function evaluations (for sufficiently small $\varepsilon$), strengthening the state-of-the-art guarantees $\widetilde O(d^{3} / \varepsilon)$ in terms of dimensional dependency. Numerical experiments validate the efficiency of the proposed method. △ Less

Submitted 7 October, 2024; originally announced October 2024.

Comments: 45 pages, 3 figures

BMRMM: An R Package for Bayesian Markov (Renewal) Mixed Models

Authors: Yutong Wu, Abhra Sarkar

Abstract: We introduce the BMRMM package implementing Bayesian inference for a class of Markov renewal mixed models which can characterize the stochastic dynamics of a collection of sequences, each comprising alternative instances of categorical states and associated continuous duration times, while being influenced by a set of exogenous factors as well as a 'random' individual. The default setting flexibly… ▽ More We introduce the BMRMM package implementing Bayesian inference for a class of Markov renewal mixed models which can characterize the stochastic dynamics of a collection of sequences, each comprising alternative instances of categorical states and associated continuous duration times, while being influenced by a set of exogenous factors as well as a 'random' individual. The default setting flexibly models the state transition probabilities using mixtures of Dirichlet distributions and the duration times using mixtures of gamma kernels while also allowing variable selection for both. Modeling such data using simpler Markov mixed models also remains an option, either by ignoring the duration times altogether or by replacing them with instances of an additional category obtained by discretizing them by a user-specified unit. The option is also useful when data on duration times may not be available in the first place. We demonstrate the package's utility using two data sets. △ Less

Submitted 16 September, 2024; originally announced September 2024.

Comments: 20 pages, 11 figures

Think Twice Before You Act: Improving Inverse Problem Solving With MCMC

Authors: Yaxuan Zhu, Zehao Dou, Haoxin Zheng, Yasi Zhang, Ying Nian Wu, Ruiqi Gao

Abstract: Recent studies demonstrate that diffusion models can serve as a strong prior for solving inverse problems. A prominent example is Diffusion Posterior Sampling (DPS), which approximates the posterior distribution of data given the measure using Tweedie's formula. Despite the merits of being versatile in solving various inverse problems without re-training, the performance of DPS is hindered by the… ▽ More Recent studies demonstrate that diffusion models can serve as a strong prior for solving inverse problems. A prominent example is Diffusion Posterior Sampling (DPS), which approximates the posterior distribution of data given the measure using Tweedie's formula. Despite the merits of being versatile in solving various inverse problems without re-training, the performance of DPS is hindered by the fact that this posterior approximation can be inaccurate especially for high noise levels. Therefore, we propose \textbf{D}iffusion \textbf{P}osterior \textbf{MC}MC (\textbf{DPMC}), a novel inference algorithm based on Annealed MCMC to solve inverse problems with pretrained diffusion models. We define a series of intermediate distributions inspired by the approximated conditional distributions used by DPS. Through annealed MCMC sampling, we encourage the samples to follow each intermediate distribution more closely before moving to the next distribution at a lower noise level, and therefore reduce the accumulated error along the path. We test our algorithm in various inverse problems, including super resolution, Gaussian deblurring, motion deblurring, inpainting, and phase retrieval. Our algorithm outperforms DPS with less number of evaluations across nearly all tasks, and is competitive among existing approaches. △ Less

Submitted 13 September, 2024; originally announced September 2024.

An Eigengap Ratio Test for Determining the Number of Communities in Network Data

Authors: Yujia Wu, Jingfei Zhang, Wei Lan, Chih-Ling Tsai

Abstract: To characterize the community structure in network data, researchers have introduced various block-type models, including the stochastic block model, degree-corrected stochastic block model, mixed membership block model, degree-corrected mixed membership block model, and others. A critical step in applying these models effectively is determining the number of communities in the network. However, t… ▽ More To characterize the community structure in network data, researchers have introduced various block-type models, including the stochastic block model, degree-corrected stochastic block model, mixed membership block model, degree-corrected mixed membership block model, and others. A critical step in applying these models effectively is determining the number of communities in the network. However, to our knowledge, existing methods for estimating the number of network communities often require model estimations or are unable to simultaneously account for network sparsity and a divergent number of communities. In this paper, we propose an eigengap-ratio based test that address these challenges. The test is straightforward to compute, requires no parameter tuning, and can be applied to a wide range of block models without the need to estimate network distribution parameters. Furthermore, it is effective for both dense and sparse networks with a divergent number of communities. We show that the proposed test statistic converges to a function of the type-I Tracy-Widom distributions under the null hypothesis, and that the test is asymptotically powerful under alternatives. Simulation studies on both dense and sparse networks demonstrate the efficacy of the proposed method. Three real-world examples are presented to illustrate the usefulness of the proposed test. △ Less

Submitted 8 September, 2024; originally announced September 2024.

2DSig-Detect: a semi-supervised framework for anomaly detection on image data using 2D-signatures

Authors: Xinheng Xie, Kureha Yamaguchi, Margaux Leblanc, Simon Malzard, Varun Chhabra, Victoria Nockles, Yue Wu

Abstract: The rapid advancement of machine learning technologies raises questions about the security of machine learning models, with respect to both training-time (poisoning) and test-time (evasion, impersonation, and inversion) attacks. Models performing image-related tasks, e.g. detection, and classification, are vulnerable to adversarial attacks that can degrade their performance and produce undesirable… ▽ More The rapid advancement of machine learning technologies raises questions about the security of machine learning models, with respect to both training-time (poisoning) and test-time (evasion, impersonation, and inversion) attacks. Models performing image-related tasks, e.g. detection, and classification, are vulnerable to adversarial attacks that can degrade their performance and produce undesirable outcomes. This paper introduces a novel technique for anomaly detection in images called 2DSig-Detect, which uses a 2D-signature-embedded semi-supervised framework rooted in rough path theory. We demonstrate our method in adversarial settings for training-time and test-time attacks, and benchmark our framework against other state of the art methods. Using 2DSig-Detect for anomaly detection, we show both superior performance and a reduction in the computation time to detect the presence of adversarial perturbations in images. △ Less

Submitted 20 March, 2025; v1 submitted 8 September, 2024; originally announced September 2024.

MSC Class: 60L99; 68T10

Latent Space Energy-based Neural ODEs

Authors: Sheng Cheng, Deqian Kong, Jianwen Xie, Kookjin Lee, Ying Nian Wu, Yezhou Yang

Abstract: This paper introduces novel deep dynamical models designed to represent continuous-time sequences. Our approach employs a neural emission model to generate each data point in the time series through a non-linear transformation of a latent state vector. The evolution of these latent states is implicitly defined by a neural ordinary differential equation (ODE), with the initial state drawn from an i… ▽ More This paper introduces novel deep dynamical models designed to represent continuous-time sequences. Our approach employs a neural emission model to generate each data point in the time series through a non-linear transformation of a latent state vector. The evolution of these latent states is implicitly defined by a neural ordinary differential equation (ODE), with the initial state drawn from an informative prior distribution parameterized by an Energy-based model (EBM). This framework is extended to disentangle dynamic states from underlying static factors of variation, represented as time-invariant variables in the latent space. We train the model using maximum likelihood estimation with Markov chain Monte Carlo (MCMC) in an end-to-end manner. Experimental results on oscillating systems, videos and real-world state sequences (MuJoCo) demonstrate that our model with the learnable energy-based prior outperforms existing counterparts, and can generalize to new dynamic parameterization, enabling long-horizon predictions. △ Less

Submitted 5 February, 2025; v1 submitted 5 September, 2024; originally announced September 2024.

PerturBench: Benchmarking Machine Learning Models for Cellular Perturbation Analysis

Authors: Yan Wu, Esther Wershof, Sebastian M Schmon, Marcel Nassar, Błażej Osiński, Ridvan Eksi, Zichao Yan, Rory Stark, Kun Zhang, Thore Graepel

Abstract: We introduce a comprehensive framework for perturbation response modeling in single cells, aimed at standardizing benchmarking in this rapidly evolving field. Our approach includes a modular and user-friendly model development and evaluation platform, a collection of diverse perturbational datasets, and a set of metrics designed to fairly compare models and dissect their performance nuances. Throu… ▽ More We introduce a comprehensive framework for perturbation response modeling in single cells, aimed at standardizing benchmarking in this rapidly evolving field. Our approach includes a modular and user-friendly model development and evaluation platform, a collection of diverse perturbational datasets, and a set of metrics designed to fairly compare models and dissect their performance nuances. Through extensive evaluation of both published and baseline models across diverse datasets, we highlight the limitations of widely used models, such as mode collapse. We also demonstrate the importance of rank metrics which complement traditional model fit measures, such as RMSE, for validating model effectiveness. Notably, our results show that while no single model architecture clearly outperforms others, simpler architectures are generally competitive and scale well with larger datasets. Overall, this benchmarking exercise sets new standards for model evaluation, supports robust model development, and advances the potential of these models to use high-throughput genetic and chemical screens for disease target discovery. △ Less

Submitted 16 June, 2025; v1 submitted 20 August, 2024; originally announced August 2024.

Comments: 10 pages plus 20 pages supplementary material. Code is available at https://github.com/altoslabs/perturbench

Towards Few-Shot Learning in the Open World: A Review and Beyond

Authors: Hui Xue, Yuexuan An, Yongchun Qin, Wenqian Li, Yixin Wu, Yongjuan Che, Pengfei Fang, Minling Zhang

Abstract: Human intelligence is characterized by our ability to absorb and apply knowledge from the world around us, especially in rapidly acquiring new concepts from minimal examples, underpinned by prior knowledge. Few-shot learning (FSL) aims to mimic this capacity by enabling significant generalizations and transferability. However, traditional FSL frameworks often rely on assumptions of clean, complete… ▽ More Human intelligence is characterized by our ability to absorb and apply knowledge from the world around us, especially in rapidly acquiring new concepts from minimal examples, underpinned by prior knowledge. Few-shot learning (FSL) aims to mimic this capacity by enabling significant generalizations and transferability. However, traditional FSL frameworks often rely on assumptions of clean, complete, and static data, conditions that are seldom met in real-world environments. Such assumptions falter in the inherently uncertain, incomplete, and dynamic contexts of the open world. This paper presents a comprehensive review of recent advancements designed to adapt FSL for use in open-world settings. We categorize existing methods into three distinct types of open-world few-shot learning: those involving varying instances, varying classes, and varying distributions. Each category is discussed in terms of its specific challenges and methods, as well as its strengths and weaknesses. We standardize experimental settings and metric benchmarks across scenarios, and provide a comparative analysis of the performance of various methods. In conclusion, we outline potential future research directions for this evolving field. It is our hope that this review will catalyze further development of effective solutions to these complex challenges, thereby advancing the field of artificial intelligence. △ Less

Submitted 19 August, 2024; originally announced August 2024.

Robust Offline Active Learning on Graphs

Authors: Yuanchen Wu, Yubai Yuan

Abstract: We consider the problem of active learning on graphs, which has crucial applications in many real-world networks where labeling node responses is expensive. In this paper, we propose an offline active learning method that selects nodes to query by explicitly incorporating information from both the network structure and node covariates. Building on graph signal recovery theories and the random spec… ▽ More We consider the problem of active learning on graphs, which has crucial applications in many real-world networks where labeling node responses is expensive. In this paper, we propose an offline active learning method that selects nodes to query by explicitly incorporating information from both the network structure and node covariates. Building on graph signal recovery theories and the random spectral sparsification technique, the proposed method adopts a two-stage biased sampling strategy that takes both informativeness and representativeness into consideration for node querying. Informativeness refers to the complexity of graph signals that are learnable from the responses of queried nodes, while representativeness refers to the capacity of queried nodes to control generalization errors given noisy node-level information. We establish a theoretical relationship between generalization error and the number of nodes selected by the proposed method. Our theoretical results demonstrate the trade-off between informativeness and representativeness in active learning. Extensive numerical experiments show that the proposed method is competitive with existing graph-based active learning methods, especially when node covariates and responses contain noises. Additionally, the proposed method is applicable to both regression and classification tasks on graphs. △ Less

Submitted 6 November, 2024; v1 submitted 15 August, 2024; originally announced August 2024.

Provably Efficient Posterior Sampling for Sparse Linear Regression via Measure Decomposition

Authors: Andrea Montanari, Yuchen Wu

Abstract: We consider the problem of sampling from the posterior distribution of a $d$-dimensional coefficient vector $\boldsymbolθ$, given linear observations $\boldsymbol{y} = \boldsymbol{X}\boldsymbolθ+\boldsymbol{\varepsilon}$. In general, such posteriors are multimodal, and therefore challenging to sample from. This observation has prompted the exploration of various heuristics that aim at approximatin… ▽ More We consider the problem of sampling from the posterior distribution of a $d$-dimensional coefficient vector $\boldsymbolθ$, given linear observations $\boldsymbol{y} = \boldsymbol{X}\boldsymbolθ+\boldsymbol{\varepsilon}$. In general, such posteriors are multimodal, and therefore challenging to sample from. This observation has prompted the exploration of various heuristics that aim at approximating the posterior distribution. In this paper, we study a different approach based on decomposing the posterior distribution into a log-concave mixture of simple product measures. This decomposition allows us to reduce sampling from a multimodal distribution of interest to sampling from a log-concave one, which is tractable and has been investigated in detail. We prove that, under mild conditions on the prior, for random designs, such measure decomposition is generally feasible when the number of samples per parameter $n/d$ exceeds a constant threshold. We thus obtain a provably efficient (polynomial time) sampling algorithm in a regime where this was previously not known. Numerical simulations confirm that the algorithm is practical, and reveal that it has attractive statistical properties compared to state-of-the-art methods. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Comments: 29 pages, 10 figures