Showing 1–50 of 665 results for author: Wang, X

A first-order method for nonconvex-nonconcave minimax problems under a local Kurdyka-Łojasiewicz condition

Authors: Zhaosong Lu, Xiangyuan Wang

Abstract: We study a class of nonconvex-nonconcave minimax problems in which the inner maximization problem satisfies a local Kurdyka-Łojasiewicz (KL) condition that may vary with the outer minimization variable. In contrast to the global KL or Polyak-Łojasiewicz (PL) conditions commonly assumed in the literature -- which are significantly stronger and often too restrictive in practice -- this local KL cond… ▽ More We study a class of nonconvex-nonconcave minimax problems in which the inner maximization problem satisfies a local Kurdyka-Łojasiewicz (KL) condition that may vary with the outer minimization variable. In contrast to the global KL or Polyak-Łojasiewicz (PL) conditions commonly assumed in the literature -- which are significantly stronger and often too restrictive in practice -- this local KL condition accommodates a broader range of practical scenarios. However, it also introduces new analytical challenges. In particular, as an optimization algorithm progresses toward a stationary point of the problem, the region over which the KL condition holds may shrink, resulting in a more intricate and potentially ill-conditioned landscape. To address this challenge, we show that the associated maximal function is locally Hölder smooth. Leveraging this key property, we develop an inexact proximal gradient method for solving the minimax problem, where the inexact gradient of the maximal function is computed by applying a proximal gradient method to a KL-structured subproblem. Under mild assumptions, we establish complexity guarantees for computing an approximate stationary point of the minimax problem. △ Less

Submitted 2 July, 2025; originally announced July 2025.

Comments: 26 pages

MSC Class: 90C26; 90C30; 90C47; 90C99; 65K05

Thompson Sampling in Function Spaces via Neural Operators

Authors: Rafael Oliveira, Xuesong Wang, Kian Ming A. Chai, Edwin V. Bonilla

Abstract: We propose an extension of Thompson sampling to optimization problems over function spaces where the objective is a known functional of an unknown operator's output. We assume that functional evaluations are inexpensive, while queries to the operator (such as running a high-fidelity simulator) are costly. Our algorithm employs a sample-then-optimize approach using neural operator surrogates. This… ▽ More We propose an extension of Thompson sampling to optimization problems over function spaces where the objective is a known functional of an unknown operator's output. We assume that functional evaluations are inexpensive, while queries to the operator (such as running a high-fidelity simulator) are costly. Our algorithm employs a sample-then-optimize approach using neural operator surrogates. This strategy avoids explicit uncertainty quantification by treating trained neural operators as approximate samples from a Gaussian process. We provide novel theoretical convergence guarantees, based on Gaussian processes in the infinite-dimensional setting, under minimal assumptions. We benchmark our method against existing baselines on functional optimization tasks involving partial differential equations and other nonlinear operator-driven phenomena, demonstrating improved sample efficiency and competitive performance. △ Less

Submitted 27 June, 2025; originally announced June 2025.

Comments: Under review

Inference for Error-Prone Count Data: Estimation under a Binomial Convolution Framework

Authors: Yuqiu Yang, Christina Vu, Cornelis J. Potgieter, Xinlei Wang, Akihito Kamata

Abstract: Measurement error in count data is common but underexplored in the literature, particularly in contexts where observed scores are bounded and arise from discrete scoring processes. Motivated by applications in oral reading fluency assessment, we propose a binomial convolution framework that extends binary misclassification models to settings where only the aggregate number of correct responses is… ▽ More Measurement error in count data is common but underexplored in the literature, particularly in contexts where observed scores are bounded and arise from discrete scoring processes. Motivated by applications in oral reading fluency assessment, we propose a binomial convolution framework that extends binary misclassification models to settings where only the aggregate number of correct responses is observed, and errors may involve both overcounting and undercounting the number of events. The model accommodates distinct true positive and true negative accuracy rates and preserves the bounded nature of the data. Assuming the availability of both contaminated and error-free scores on a subset of items, we develop and compare three estimation strategies: maximum likelihood estimation (MLE), linear regression, and generalized method of moments (GMM). Extensive simulations show that MLE is most accurate when the model is correctly specified but is computationally intensive and less robust to misspecification. Regression is simple and stable but less precise, while GMM offers a compromise in model dependence, though it is sensitive to outliers. In practice, this framework supports improved inference in unsupervised settings where contaminated scores serve as inputs to downstream analyses. By quantifying accuracy rates, the model enables score corrections even when no specific outcome is yet defined. We demonstrate its utility using real oral reading fluency data, comparing human and AI-generated scores. Findings highlight the practical implications of estimator choice and underscore the importance of explicitly modeling asymmetric measurement error in count data. △ Less

Submitted 25 June, 2025; originally announced June 2025.

Comments: 40 pages, 6 figures, 3 tables

Efficient Network Automatic Relevance Determination

Authors: Hongwei Zhang, Ziqi Ye, Xinyuan Wang, Xin Guo, Zenglin Xu, Yuan Cheng, Zixin Hu, Yuan Qi

Abstract: We propose Network Automatic Relevance Determination (NARD), an extension of ARD for linearly probabilistic models, to simultaneously model sparse relationships between inputs $X \in \mathbb R^{d \times N}$ and outputs $Y \in \mathbb R^{m \times N}$, while capturing the correlation structure among the $Y$. NARD employs a matrix normal prior which contains a sparsity-inducing parameter to identify… ▽ More We propose Network Automatic Relevance Determination (NARD), an extension of ARD for linearly probabilistic models, to simultaneously model sparse relationships between inputs $X \in \mathbb R^{d \times N}$ and outputs $Y \in \mathbb R^{m \times N}$, while capturing the correlation structure among the $Y$. NARD employs a matrix normal prior which contains a sparsity-inducing parameter to identify and discard irrelevant features, thereby promoting sparsity in the model. Algorithmically, it iteratively updates both the precision matrix and the relationship between $Y$ and the refined inputs. To mitigate the computational inefficiencies of the $\mathcal O(m^3 + d^3)$ cost per iteration, we introduce Sequential NARD, which evaluates features sequentially, and a Surrogate Function Method, leveraging an efficient approximation of the marginal likelihood and simplifying the calculation of determinant and inverse of an intermediate matrix. Combining the Sequential update with the Surrogate Function method further reduces computational costs. The computational complexity per iteration for these three methods is reduced to $\mathcal O(m^3+p^3)$, $\mathcal O(m^3 + d^2)$, $\mathcal O(m^3+p^2)$, respectively, where $p \ll d$ is the final number of features in the model. Our methods demonstrate significant improvements in computational efficiency with comparable performance on both synthetic and real-world datasets. △ Less

Submitted 14 June, 2025; originally announced June 2025.

Comments: ICML 2025

Regularized Estimation of the Loading Matrix in Factor Models for High-Dimensional Time Series

Authors: Xialu Liu, Xin Wang

Abstract: High-dimensional data analysis using traditional models suffers from overparameterization. Two types of techniques are commonly used to reduce the number of parameters - regularization and dimension reduction. In this project, we combine them by imposing a sparse factor structure and propose a regularized estimator to further reduce the number of parameters in factor models. A challenge limiting t… ▽ More High-dimensional data analysis using traditional models suffers from overparameterization. Two types of techniques are commonly used to reduce the number of parameters - regularization and dimension reduction. In this project, we combine them by imposing a sparse factor structure and propose a regularized estimator to further reduce the number of parameters in factor models. A challenge limiting the widespread application of factor models is that factors are hard to interpret, as both factors and the loading matrix are unobserved. To address this, we introduce a penalty term when estimating the loading matrix for a sparse estimate. As a result, each factor only drives a smaller subset of time series that exhibit the strongest correlation, improving the factor interpretability. The theoretical properties of the proposed estimator are investigated. The simulation results are presented to confirm that our algorithm performs well. We apply our method to Hawaii tourism data. △ Less

Submitted 7 July, 2025; v1 submitted 12 June, 2025; originally announced June 2025.

Accelerating Constrained Sampling: A Large Deviations Approach

Authors: Yingli Wang, Changwei Tu, Xiaoyu Wang, Lingjiong Zhu

Abstract: The problem of sampling a target probability distribution on a constrained domain arises in many applications including machine learning. For constrained sampling, various Langevin algorithms such as projected Langevin Monte Carlo (PLMC) based on the discretization of reflected Langevin dynamics (RLD) and more generally skew-reflected non-reversible Langevin Monte Carlo (SRNLMC) based on the discr… ▽ More The problem of sampling a target probability distribution on a constrained domain arises in many applications including machine learning. For constrained sampling, various Langevin algorithms such as projected Langevin Monte Carlo (PLMC) based on the discretization of reflected Langevin dynamics (RLD) and more generally skew-reflected non-reversible Langevin Monte Carlo (SRNLMC) based on the discretization of skew-reflected non-reversible Langevin dynamics (SRNLD) have been proposed and studied in the literature. This work focuses on the long-time behavior of SRNLD, where a skew-symmetric matrix is added to RLD. Although the non-asymptotic convergence analysis for SRNLD (and SRNLMC) and the acceleration compared to RLD (and PMLC) have been studied in the literature, it is not clear how one should design the skew-symmetric matrix in the dynamics to achieve good performance in practice. We establish a large deviation principle (LDP) for the empirical measure of SRNLD when the skew-symmetric matrix is chosen such that its product with the inward unit normal vector field on the boundary is zero. By explicitly characterizing the rate functions, we show that SRNLD can accelerate the convergence to the target distribution compared to RLD with this choice of the skew-symmetric matrix. Numerical experiments for SRNLMC based on the proposed skew-symmetric matrix show superior performance which validate the theoretical findings from the large deviations theory. △ Less

Submitted 9 June, 2025; originally announced June 2025.

Comments: 40 pages, 7 figures

Dynamic Modes as Time Representation for Spatiotemporal Forecasting

Authors: Menglin Kong, Vincent Zhihao Zheng, Xudong Wang, Lijun Sun

Abstract: This paper introduces a data-driven time embedding method for modeling long-range seasonal dependencies in spatiotemporal forecasting tasks. The proposed approach employs Dynamic Mode Decomposition (DMD) to extract temporal modes directly from observed data, eliminating the need for explicit timestamps or hand-crafted time features. These temporal modes serve as time representations that can be se… ▽ More This paper introduces a data-driven time embedding method for modeling long-range seasonal dependencies in spatiotemporal forecasting tasks. The proposed approach employs Dynamic Mode Decomposition (DMD) to extract temporal modes directly from observed data, eliminating the need for explicit timestamps or hand-crafted time features. These temporal modes serve as time representations that can be seamlessly integrated into deep spatiotemporal forecasting models. Unlike conventional embeddings such as time-of-day indicators or sinusoidal functions, our method captures complex multi-scale periodicity through spectral analysis of spatiotemporal data. Extensive experiments on urban mobility, highway traffic, and climate datasets demonstrate that the DMD-based embedding consistently improves long-horizon forecasting accuracy, reduces residual correlation, and enhances temporal generalization. The method is lightweight, model-agnostic, and compatible with any architecture that incorporates time covariates. △ Less

Submitted 1 June, 2025; originally announced June 2025.

Nearly-Linear Time Private Hypothesis Selection with the Optimal Approximation Factor

Authors: Maryam Aliakbarpour, Zhan Shi, Ria Stevens, Vincent X. Wang

Abstract: Estimating the density of a distribution from its samples is a fundamental problem in statistics. Hypothesis selection addresses the setting where, in addition to a sample set, we are given $n$ candidate distributions -- referred to as hypotheses -- and the goal is to determine which one best describes the underlying data distribution. This problem is known to be solvable very efficiently, requiri… ▽ More Estimating the density of a distribution from its samples is a fundamental problem in statistics. Hypothesis selection addresses the setting where, in addition to a sample set, we are given $n$ candidate distributions -- referred to as hypotheses -- and the goal is to determine which one best describes the underlying data distribution. This problem is known to be solvable very efficiently, requiring roughly $O(\log n)$ samples and running in $\tilde{O}(n)$ time. The quality of the output is measured via the total variation distance to the unknown distribution, and the approximation factor of the algorithm determines how large this distance is compared to the optimal distance achieved by the best candidate hypothesis. It is known that $α= 3$ is the optimal approximation factor for this problem. We study hypothesis selection under the constraint of differential privacy. We propose a differentially private algorithm in the central model that runs in nearly-linear time with respect to the number of hypotheses, achieves the optimal approximation factor, and incurs only a modest increase in sample complexity, which remains polylogarithmic in $n$. This resolves an open question posed by [Bun, Kamath, Steinke, Wu, NeurIPS 2019]. Prior to our work, existing upper bounds required quadratic time. △ Less

Submitted 1 June, 2025; originally announced June 2025.

Comments: 33 pages

Reconstruction and Prediction of Volterra Integral Equations Driven by Gaussian Noise

Authors: Zhihao Xu, Saisai Ding, Zhikun Zhang, Xiangjun Wang

Abstract: Integral equations are widely used in fields such as applied modeling, medical imaging, and system identification, providing a powerful framework for solving deterministic problems. While parameter identification for differential equations has been extensively studied, the focus on integral equations, particularly stochastic Volterra integral equations, remains limited. This research addresses the… ▽ More Integral equations are widely used in fields such as applied modeling, medical imaging, and system identification, providing a powerful framework for solving deterministic problems. While parameter identification for differential equations has been extensively studied, the focus on integral equations, particularly stochastic Volterra integral equations, remains limited. This research addresses the parameter identification problem, also known as the equation reconstruction problem, in Volterra integral equations driven by Gaussian noise. We propose an improved deep neural networks framework for estimating unknown parameters in the drift term of these equations. The network represents the primary variables and their integrals, enhancing parameter estimation accuracy by incorporating inter-output relationships into the loss function. Additionally, the framework extends beyond parameter identification to predict the system's behavior outside the integration interval. Prediction accuracy is validated by comparing predicted and true trajectories using a 95% confidence interval. Numerical experiments demonstrate the effectiveness of the proposed deep neural networks framework in both parameter identification and prediction tasks, showing robust performance under varying noise levels and providing accurate solutions for modeling stochastic systems. △ Less

Submitted 1 June, 2025; originally announced June 2025.

FLoE: Fisher-Based Layer Selection for Efficient Sparse Adaptation of Low-Rank Experts

Authors: Xinyi Wang, Lirong Gao, Haobo Wang, Yiming Zhang, Junbo Zhao

Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods have emerged as a widely adopted strategy for adapting pre-trained Large Language Models (LLMs) to downstream tasks, significantly reducing memory and computational costs. However, most existing PEFT techniques uniformly deploy LoRA adapters across all layers, disregarding the intrinsic heterogeneity of layer contributions and task-specific rank requi… ▽ More Parameter-Efficient Fine-Tuning (PEFT) methods have emerged as a widely adopted strategy for adapting pre-trained Large Language Models (LLMs) to downstream tasks, significantly reducing memory and computational costs. However, most existing PEFT techniques uniformly deploy LoRA adapters across all layers, disregarding the intrinsic heterogeneity of layer contributions and task-specific rank requirements. This uniform paradigm leads to redundant parameter allocation and suboptimal adaptation efficiency. To address these limitations, we propose FLoE, a novel PEFT framework that introduces two key innovations: (i) a Fisher information-guided importance scoring mechanism to dynamically identify task-critical transformer layers for MoE-based low-rank adaptation, enabling sparse adapter deployment; and (ii) a Bayesian optimization-driven rank allocator that automatically determines optimal LoRA ranks on specific datasets without exhaustive grid search. Extensive experiments across diverse LLMs and benchmarks reveal that FLoE achieves impressive efficiency-accuracy trade-offs, making FLoE particularly advantageous in resource-constrained environments that necessitate rapid adaptation. △ Less

Submitted 31 May, 2025; originally announced June 2025.

Comments: 17 pages, 9 figures

Bias as a Virtue: Rethinking Generalization under Distribution Shifts

Authors: Ruixuan Chen, Wentao Li, Jiahui Xiao, Yuchen Li, Yimin Tang, Xiaonan Wang

Abstract: Machine learning models often degrade when deployed on data distributions different from their training data. Challenging conventional validation paradigms, we demonstrate that higher in-distribution (ID) bias can lead to better out-of-distribution (OOD) generalization. Our Adaptive Distribution Bridge (ADB) framework implements this insight by introducing controlled statistical diversity during t… ▽ More Machine learning models often degrade when deployed on data distributions different from their training data. Challenging conventional validation paradigms, we demonstrate that higher in-distribution (ID) bias can lead to better out-of-distribution (OOD) generalization. Our Adaptive Distribution Bridge (ADB) framework implements this insight by introducing controlled statistical diversity during training, enabling models to develop bias profiles that effectively generalize across distributions. Empirically, we observe a robust negative correlation where higher ID bias corresponds to lower OOD error--a finding that contradicts standard practices focused on minimizing validation error. Evaluation on multiple datasets shows our approach significantly improves OOD generalization. ADB achieves robust mean error reductions of up to 26.8% compared to traditional cross-validation, and consistently identifies high-performing training strategies, evidenced by percentile ranks often exceeding 74.4%. Our work provides both a practical method for improving generalization and a theoretical framework for reconsidering the role of bias in robust machine learning. △ Less

Submitted 31 May, 2025; originally announced June 2025.

Comments: 14 pages

Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention

Authors: Stephan Rabanser, Ali Shahin Shamsabadi, Olive Franzese, Xiao Wang, Adrian Weller, Nicolas Papernot

Abstract: Cautious predictions -- where a machine learning model abstains when uncertain -- are crucial for limiting harmful errors in safety-critical applications. In this work, we identify a novel threat: a dishonest institution can exploit these mechanisms to discriminate or unjustly deny services under the guise of uncertainty. We demonstrate the practicality of this threat by introducing an uncertainty… ▽ More Cautious predictions -- where a machine learning model abstains when uncertain -- are crucial for limiting harmful errors in safety-critical applications. In this work, we identify a novel threat: a dishonest institution can exploit these mechanisms to discriminate or unjustly deny services under the guise of uncertainty. We demonstrate the practicality of this threat by introducing an uncertainty-inducing attack called Mirage, which deliberately reduces confidence in targeted input regions, thereby covertly disadvantaging specific individuals. At the same time, Mirage maintains high predictive performance across all data points. To counter this threat, we propose Confidential Guardian, a framework that analyzes calibration metrics on a reference dataset to detect artificially suppressed confidence. Additionally, it employs zero-knowledge proofs of verified inference to ensure that reported confidence scores genuinely originate from the deployed model. This prevents the provider from fabricating arbitrary model confidence values while protecting the model's proprietary details. Our results confirm that Confidential Guardian effectively prevents the misuse of cautious predictions, providing verifiable assurances that abstention reflects genuine model uncertainty rather than malicious intent. △ Less

Submitted 29 May, 2025; originally announced May 2025.

Comments: Proceedings of the 42nd International Conference on Machine Learning

Learnable Kernel Density Estimation for Graphs

Authors: Xudong Wang, Ziheng Sun, Chris Ding, Jicong Fan

Abstract: This work proposes a framework LGKDE that learns kernel density estimation for graphs. The key challenge in graph density estimation lies in effectively capturing both structural patterns and semantic variations while maintaining theoretical guarantees. Combining graph kernels and kernel density estimation (KDE) is a standard approach to graph density estimation, but has unsatisfactory performance… ▽ More This work proposes a framework LGKDE that learns kernel density estimation for graphs. The key challenge in graph density estimation lies in effectively capturing both structural patterns and semantic variations while maintaining theoretical guarantees. Combining graph kernels and kernel density estimation (KDE) is a standard approach to graph density estimation, but has unsatisfactory performance due to the handcrafted and fixed features of kernels. Our method LGKDE leverages graph neural networks to represent each graph as a discrete distribution and utilizes maximum mean discrepancy to learn the graph metric for multi-scale KDE, where all parameters are learned by maximizing the density of graphs relative to the density of their well-designed perturbed counterparts. The perturbations are conducted on both node features and graph spectra, which helps better characterize the boundary of normal density regions. Theoretically, we establish consistency and convergence guarantees for LGKDE, including bounds on the mean integrated squared error, robustness, and complexity. We validate LGKDE by demonstrating its effectiveness in recovering the underlying density of synthetic graph distributions and applying it to graph anomaly detection across diverse benchmark datasets. Extensive empirical evaluation shows that LGKDE demonstrates superior performance compared to state-of-the-art baselines on most benchmark datasets. △ Less

Submitted 27 May, 2025; originally announced May 2025.

Comments: Under Review

ACM Class: I.2; I.5.1; I.5.2

Offline Clustering of Linear Bandits: Unlocking the Power of Clusters in Data-Limited Environments

Authors: Jingyuan Liu, Zeyu Zhang, Xuchuang Wang, Xutong Liu, John C. S. Lui, Mohammad Hajiesmaili, Carlee Joe-Wong

Abstract: Contextual linear multi-armed bandits are a learning framework for making a sequence of decisions, e.g., advertising recommendations for a sequence of arriving users. Recent works have shown that clustering these users based on the similarity of their learned preferences can significantly accelerate the learning. However, prior work has primarily focused on the online setting, which requires conti… ▽ More Contextual linear multi-armed bandits are a learning framework for making a sequence of decisions, e.g., advertising recommendations for a sequence of arriving users. Recent works have shown that clustering these users based on the similarity of their learned preferences can significantly accelerate the learning. However, prior work has primarily focused on the online setting, which requires continually collecting user data, ignoring the offline data widely available in many applications. To tackle these limitations, we study the offline clustering of bandits (Off-ClusBand) problem, which studies how to use the offline dataset to learn cluster properties and improve decision-making across multiple users. The key challenge in Off-ClusBand arises from data insufficiency for users: unlike the online case, in the offline case, we have a fixed, limited dataset to work from and thus must determine whether we have enough data to confidently cluster users together. To address this challenge, we propose two algorithms: Off-C$^2$LUB, which we analytically show performs well for arbitrary amounts of user data, and Off-CLUB, which is prone to bias when data is limited but, given sufficient data, matches a theoretical lower bound that we derive for the offline clustered MAB problem. We experimentally validate these results on both real and synthetic datasets. △ Less

Submitted 25 May, 2025; originally announced May 2025.

Scale-invariant Attention

Authors: Ben Anson, Xi Wang, Laurence Aitchison

Abstract: One persistent challenge in LLM research is the development of attention mechanisms that are able to generalise from training on shorter contexts to inference on longer contexts. We propose two conditions that we expect all effective long context attention mechanisms to have: scale-invariant total attention, and scale-invariant attention sparsity. Under a Gaussian assumption, we show that a simple… ▽ More One persistent challenge in LLM research is the development of attention mechanisms that are able to generalise from training on shorter contexts to inference on longer contexts. We propose two conditions that we expect all effective long context attention mechanisms to have: scale-invariant total attention, and scale-invariant attention sparsity. Under a Gaussian assumption, we show that a simple position-dependent transformation of the attention logits is sufficient for these conditions to hold. Experimentally we find that the resulting scale-invariant attention scheme gives considerable benefits in terms of validation loss when zero-shot generalising from training on short contexts to validation on longer contexts, and is effective at long-context retrieval. △ Less

Submitted 20 May, 2025; originally announced May 2025.

Comments: Preprint

A system identification approach to clustering vector autoregressive time series

Authors: Zuogong Yue, Xinyi Wang, Victor Solo

Abstract: Clustering of time series based on their underlying dynamics is keeping attracting researchers due to its impacts on assisting complex system modelling. Most current time series clustering methods handle only scalar time series, treat them as white noise, or rely on domain knowledge for high-quality feature construction, where the autocorrelation pattern/feature is mostly ignored. Instead of relyi… ▽ More Clustering of time series based on their underlying dynamics is keeping attracting researchers due to its impacts on assisting complex system modelling. Most current time series clustering methods handle only scalar time series, treat them as white noise, or rely on domain knowledge for high-quality feature construction, where the autocorrelation pattern/feature is mostly ignored. Instead of relying on heuristic feature/metric construction, the system identification approach allows treating vector time series clustering by explicitly considering their underlying autoregressive dynamics. We first derive a clustering algorithm based on a mixture autoregressive model. Unfortunately it turns out to have significant computational problems. We then derive a `small-noise' limiting version of the algorithm, which we call k-LMVAR (Limiting Mixture Vector AutoRegression), that is computationally manageable. We develop an associated BIC criterion for choosing the number of clusters and model order. The algorithm performs very well in comparative simulations and also scales well computationally. △ Less

Submitted 20 May, 2025; originally announced May 2025.

LoD: Loss-difference OOD Detection by Intentionally Label-Noisifying Unlabeled Wild Data

Authors: Chuanxing Geng, Qifei Li, Xinrui Wang, Dong Liang, Songcan Chen, Pong C. Yuen

Abstract: Using unlabeled wild data containing both in-distribution (ID) and out-of-distribution (OOD) data to improve the safety and reliability of models has recently received increasing attention. Existing methods either design customized losses for labeled ID and unlabeled wild data then perform joint optimization, or first filter out OOD data from the latter then learn an OOD detector. While achieving… ▽ More Using unlabeled wild data containing both in-distribution (ID) and out-of-distribution (OOD) data to improve the safety and reliability of models has recently received increasing attention. Existing methods either design customized losses for labeled ID and unlabeled wild data then perform joint optimization, or first filter out OOD data from the latter then learn an OOD detector. While achieving varying degrees of success, two potential issues remain: (i) Labeled ID data typically dominates the learning of models, inevitably making models tend to fit OOD data as IDs; (ii) The selection of thresholds for identifying OOD data in unlabeled wild data usually faces dilemma due to the unavailability of pure OOD samples. To address these issues, we propose a novel loss-difference OOD detection framework (LoD) by \textit{intentionally label-noisifying} unlabeled wild data. Such operations not only enable labeled ID data and OOD data in unlabeled wild data to jointly dominate the models' learning but also ensure the distinguishability of the losses between ID and OOD samples in unlabeled wild data, allowing the classic clustering technique (e.g., K-means) to filter these OOD samples without requiring thresholds any longer. We also provide theoretical foundation for LoD's viability, and extensive experiments verify its superiority. △ Less

Submitted 19 May, 2025; originally announced May 2025.

Comments: Accepted by IJCAI2025

Topology Guidance: Controlling the Outputs of Generative Models via Vector Field Topology

Authors: Xiaohan Wang, Matthew Berger

Abstract: For domains that involve numerical simulation, it can be computationally expensive to run an ensemble of simulations spanning a parameter space of interest to a user. To this end, an attractive surrogate for simulation is the generative modeling of fields produced by an ensemble, allowing one to synthesize fields in a computationally cheap, yet accurate, manner. However, for the purposes of visual… ▽ More For domains that involve numerical simulation, it can be computationally expensive to run an ensemble of simulations spanning a parameter space of interest to a user. To this end, an attractive surrogate for simulation is the generative modeling of fields produced by an ensemble, allowing one to synthesize fields in a computationally cheap, yet accurate, manner. However, for the purposes of visual analysis, a limitation of generative models is their lack of control, as it is unclear what one should expect when sampling a field from a model. In this paper we study how to make generative models of fields more controllable, so that users can specify features of interest, in particular topological features, that they wish to see in the output. We propose topology guidance, a method for guiding the sampling process of a generative model, specifically a diffusion model, such that a topological description specified as input is satisfied in the generated output. Central to our method, we couple a coordinate-based neural network used to represent fields, with a diffusion model used for generation. We show how to use topologically-relevant signals provided by the coordinate-based network to help guide the denoising process of a diffusion model. This enables us to faithfully represent a user's specified topology, while ensuring that the output field remains within the generative data distribution. Specifically, we study 2D vector field topology, evaluating our method over an ensemble of fluid flows, where we show that generated vector fields faithfully adhere to the location, and type, of critical points over the spatial domain. We further show the benefits of our method in aiding the comparison of ensembles, allowing one to explore commonalities and differences in distributions along prescribed topological features. △ Less

Submitted 10 May, 2025; originally announced May 2025.

Data Envelopment Analysis with Robust and Closest Targets:Integrating Full-Dimensional Efficient Facets for Risk-Resilient Benchmarking

Authors: Xiuquan Huang, Xi Wang, Tao Zhang, Xiaocang Xu, Ali Emrouznejad

Abstract: As the external environment become increasingly volatile and unpredictable, the selection of benchmarking targets in data envelopment analysis should account for their ability to consider risks; however, this aspect has not received sufficient attention. We propose a robust benchmarking target defined by the intersection of the maximum number of full-dimensional efficient facets, each representing… ▽ More As the external environment become increasingly volatile and unpredictable, the selection of benchmarking targets in data envelopment analysis should account for their ability to consider risks; however, this aspect has not received sufficient attention. We propose a robust benchmarking target defined by the intersection of the maximum number of full-dimensional efficient facets, each representing a unique marginal substitution relationship. These targets can serve as robust projections for decision making units that are lacking prior risk information because they incorporate the maximum number of marginal substitution relationships. This enables decision makers to adjust their production through these relationships, thereby maximizing the likelihood of achieving globally optimal outcomes. Furthermore, we propose a novel, well-defined efficiency measure based on robust and closest targets. Finally, we demonstrate the application of the proposed measure using a dataset comprising 38 universities from China's 985 Project. △ Less

Submitted 18 June, 2025; v1 submitted 9 May, 2025; originally announced May 2025.

Machine learning bridging battery field data and laboratory data

Authors: Yanbin Zhao, Hao Liu, Zhihua Deng, Tong Li, Haoyi Jiang, Zhenfei Ling, Xingkai Wang, Lei Zhang, Xiaoping Ouyang

Abstract: Aiming at the dilemma that most laboratory data-driven diagnostic and prognostic methods cannot be applied to field batteries in passenger cars and energy storage systems, this paper proposes a method to bridge field data and laboratory data using machine learning. Only two field real impedances corresponding to a medium frequency and a high frequency are needed to predict laboratory real impedanc… ▽ More Aiming at the dilemma that most laboratory data-driven diagnostic and prognostic methods cannot be applied to field batteries in passenger cars and energy storage systems, this paper proposes a method to bridge field data and laboratory data using machine learning. Only two field real impedances corresponding to a medium frequency and a high frequency are needed to predict laboratory real impedance curve, laboratory charge/discharge curve, and laboratory relaxation curve. Based on the predicted laboratory data, laboratory data-driven methods can be used for field battery diagnosis and prognosis. Compared with the field data-driven methods based on massive historical field data, the proposed method has the advantages of higher accuracy, lower cost, faster speed, readily available, and no use of private data. The proposed method is tested using two open-source datasets containing 249 NMC cells. For a test set containing 76 cells, the mean absolute percentage errors of laboratory real impedance curve, charge curve, and discharge curve prediction results are 0.85%, 4.72%, and 2.69%, respectively. This work fills the gap between laboratory data-driven diagnostic and prognostic methods and field battery applications, making all laboratory data-driven methods applicable to field battery diagnosis and prognosis. Furthermore, this work overturns the fixed path of developing field battery diagnostic and prognostic methods based on massive field historical data, opening up new research and breakthrough directions for field battery diagnosis and prognosis. △ Less

Submitted 13 May, 2025; v1 submitted 8 May, 2025; originally announced May 2025.

Comments: 73 pages, 21 figures

Estimating the Heritability of Longitudinal Rate-of-Change: Genetic Insights into PSA Velocity in Prostate Cancer-Free Individuals

Authors: Pei Zhang, Xiaoyu Wang, Jianxin Shi, Paul S. Albert

Abstract: Serum prostate-specific antigen (PSA) is widely used for prostate cancer screening. While the genetics of PSA levels has been studied to enhance screening accuracy, the genetic basis of PSA velocity, the rate of PSA change over time, remains unclear. The Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial, a large, randomized study with longitudinal PSA data (15,260 cancer-free m… ▽ More Serum prostate-specific antigen (PSA) is widely used for prostate cancer screening. While the genetics of PSA levels has been studied to enhance screening accuracy, the genetic basis of PSA velocity, the rate of PSA change over time, remains unclear. The Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial, a large, randomized study with longitudinal PSA data (15,260 cancer-free males, averaging 5.34 samples per subject) and genome-wide genotype data, provides a unique opportunity to estimate PSA velocity heritability. We developed a mixed model to jointly estimate heritability of PSA levels at age 54 and PSA velocity. To accommodate the large dataset, we implemented two efficient computational approaches: a partitioning and meta-analysis strategy using average information restricted maximum likelihood (AI-REML), and a fast restricted Haseman-Elston (REHE) regression method. Simulations showed that both methods yield unbiased estimates of both heritability metrics, with AI-REML providing smaller variability in the estimation of velocity heritability than REHE. Applying AI-REML to PLCO data, we estimated heritability at 0.32 (s.e. = 0.07) for baseline PSA and 0.45 (s.e. = 0.18) for PSA velocity. These findings reveal a substantial genetic contribution to PSA velocity, supporting future genome-wide studies to identify variants affecting PSA dynamics and improve PSA-based screening. △ Less

Submitted 7 May, 2025; originally announced May 2025.

Residual-based Alternative Partial Least Squares for Generalized Functional Linear Models

Authors: Yue Wang, Xiao Wang, Joseph G. Ibrahim, Hongtu Zhu

Abstract: Many biomedical studies collect high-dimensional medical imaging data to identify biomarkers for the detection, diagnosis, and treatment of human diseases. Consequently, it is crucial to develop accurate models that can predict a wide range of clinical outcomes (both discrete and continuous) based on imaging data. By treating imaging predictors as functional data, we propose a residual-based alter… ▽ More Many biomedical studies collect high-dimensional medical imaging data to identify biomarkers for the detection, diagnosis, and treatment of human diseases. Consequently, it is crucial to develop accurate models that can predict a wide range of clinical outcomes (both discrete and continuous) based on imaging data. By treating imaging predictors as functional data, we propose a residual-based alternative partial least squares (RAPLS) model for a broad class of generalized functional linear models that incorporate both functional and scalar covariates. Our RAPLS method extends the alternative partial least squares (APLS) algorithm iteratively to accommodate additional scalar covariates and non-continuous outcomes. We establish the convergence rate of the RAPLS estimator for the unknown slope function and, with an additional calibration step, we prove the asymptotic normality and efficiency of the calibrated RAPLS estimator for the scalar parameters. The effectiveness of the RAPLS algorithm is demonstrated through multiple simulation studies and an application predicting Alzheimer's disease progression using neuroimaging data from the Alzheimer's Disease Neuroimaging Initiative (ADNI). △ Less

Submitted 1 May, 2025; originally announced May 2025.

Comments: 40 pages, 1 figure; accepted in Statistica Sinica

AI-Assisted Decision-Making for Clinical Assessment of Auto-Segmented Contour Quality

Authors: Biling Wang, Austen Maniscalco, Ti Bai, Siqiu Wang, Michael Dohopolski, Mu-Han Lin, Chenyang Shen, Dan Nguyen, Junzhou Huang, Steve Jiang, Xinlei Wang

Abstract: Purpose: This study presents a Deep Learning (DL)-based quality assessment (QA) approach for evaluating auto-generated contours (auto-contours) in radiotherapy, with emphasis on Online Adaptive Radiotherapy (OART). Leveraging Bayesian Ordinal Classification (BOC) and calibrated uncertainty thresholds, the method enables confident QA predictions without relying on ground truth contours or extensive… ▽ More Purpose: This study presents a Deep Learning (DL)-based quality assessment (QA) approach for evaluating auto-generated contours (auto-contours) in radiotherapy, with emphasis on Online Adaptive Radiotherapy (OART). Leveraging Bayesian Ordinal Classification (BOC) and calibrated uncertainty thresholds, the method enables confident QA predictions without relying on ground truth contours or extensive manual labeling. Methods: We developed a BOC model to classify auto-contour quality and quantify prediction uncertainty. A calibration step was used to optimize uncertainty thresholds that meet clinical accuracy needs. The method was validated under three data scenarios: no manual labels, limited labels, and extensive labels. For rectum contours in prostate cancer, we applied geometric surrogate labels when manual labels were absent, transfer learning when limited, and direct supervision when ample labels were available. Results: The BOC model delivered robust performance across all scenarios. Fine-tuning with just 30 manual labels and calibrating with 34 subjects yielded over 90% accuracy on test data. Using the calibrated threshold, over 93% of the auto-contours' qualities were accurately predicted in over 98% of cases, reducing unnecessary manual reviews and highlighting cases needing correction. Conclusion: The proposed QA model enhances contouring efficiency in OART by reducing manual workload and enabling fast, informed clinical decisions. Through uncertainty quantification, it ensures safer, more reliable radiotherapy workflows. △ Less

Submitted 11 May, 2025; v1 submitted 1 May, 2025; originally announced May 2025.

Robust Estimation and Inference in Hybrid Controlled Trials for Binary Outcomes: A Case Study on Non-Small Cell Lung Cancer

Authors: Jiajun Liu, Ke Zhu, Shu Yang, Xiaofei Wang

Abstract: Hybrid controlled trials (HCTs), which augment randomized controlled trials (RCTs) with external controls (ECs), are increasingly receiving attention as a way to address limited power, slow accrual, and ethical concerns in clinical research. However, borrowing from ECs raises critical statistical challenges in estimation and inference, especially for binary outcomes where hidden bias is harder to… ▽ More Hybrid controlled trials (HCTs), which augment randomized controlled trials (RCTs) with external controls (ECs), are increasingly receiving attention as a way to address limited power, slow accrual, and ethical concerns in clinical research. However, borrowing from ECs raises critical statistical challenges in estimation and inference, especially for binary outcomes where hidden bias is harder to detect and estimands such as risk difference, risk ratio, and odds ratio are of primary interest. We propose a novel framework that combines doubly robust estimators for various estimands under covariate shift of ECs with conformal selective borrowing (CSB) to address outcome incomparability. CSB uses conformal inference with nearest-neighbor-based conformal scores and their label-conditional extensions to perform finite-sample exact individual-level EC selection, addressing the limited information in binary outcomes. To ensure strict type I error rate control for testing treatment effects while gaining power, we use a Fisher randomization test with the CSB estimator as the test statistic. Extensive simulations demonstrate the robust performance of our methods. We apply our method to data from CALGB 9633 and the National Cancer Database to evaluate chemotherapy effects in Stage IB non-small-cell lung cancer patients and show that the proposed method effectively mitigates hidden bias introduced by full-borrowing approaches, strictly controls the type I error rate, and improves the power over RCT-only analysis. △ Less

Submitted 30 April, 2025; originally announced May 2025.

Machine learning accelerates fuel cell life testing

Authors: Yanbin Zhao, Hao Liu, Zhihua Deng, Haoyi Jiang, Zhenfei Ling, Zhiyang Liu, Xingkai Wang, Tong Li, Xiaoping Ouyang

Abstract: Accelerated life testing (ALT) can significantly reduce the economic, time, and labor costs of life testing in the process of equipment, device, and material research and development (R&D), and improve R&D efficiency. This paper proposes a performance characterization data prediction (PCDP) method and a life prediction-driven ALT (LP-ALT) method to accelerate the life test of polymer electrolyte m… ▽ More Accelerated life testing (ALT) can significantly reduce the economic, time, and labor costs of life testing in the process of equipment, device, and material research and development (R&D), and improve R&D efficiency. This paper proposes a performance characterization data prediction (PCDP) method and a life prediction-driven ALT (LP-ALT) method to accelerate the life test of polymer electrolyte membrane fuel cells (PEMFCs). The PCDP method can accurately predict different PCD using only four impedances (real and imaginary) corresponding to a high frequency and a medium frequency, greatly shortening the measurement time of offline PCD and reducing the difficulty of life testing. The test results on an open source life test dataset containing 42 PEMFCs show that compared with the determination coefficient (R^2) results of predicted aging indicators, including limiting current, total mass transport resistance, electrochemically active surface area, and crossover current, obtained based on the measured PCD, the R^2 results of predicted aging indicators based on the predicted PCD is only reduced by 0.04, 0.01, 0.05, and 0.06, respectively. The LP-ALT method can shorten the life test time through early life prediction. Test results on the same open-source life test dataset of PEMFCs show that the acceleration ratio of the LP-ALT method can reach 30 times under the premise of ensuring that the minimum R^2 of the prediction results of different aging indicators, including limiting current, total mass transport resistance, and electrochemically active surface area, is not less than 0.89. Combining the different performance characterization data predicted by the PCDP method and the life prediction of the LP-ALT method, the diagnosis and prognosis of PEMFCs and their components can be achieved. △ Less

Submitted 7 May, 2025; v1 submitted 26 April, 2025; originally announced April 2025.

Comments: 39 pages, 25 figures

Optimizing LLM Inference: Fluid-Guided Online Scheduling with Memory Constraints

Authors: Ruicheng Ao, Gan Luo, David Simchi-Levi, Xinshang Wang

Abstract: Large Language Models (LLMs) are indispensable in today's applications, but their inference procedure -- generating responses by processing text in segments and using a memory-heavy Key-Value (KV) cache -- demands significant computational resources, particularly under memory constraints. This paper formulates LLM inference optimization as a multi-stage online scheduling problem where sequential p… ▽ More Large Language Models (LLMs) are indispensable in today's applications, but their inference procedure -- generating responses by processing text in segments and using a memory-heavy Key-Value (KV) cache -- demands significant computational resources, particularly under memory constraints. This paper formulates LLM inference optimization as a multi-stage online scheduling problem where sequential prompt arrivals and KV cache growth render conventional scheduling ineffective. We develop a fluid dynamics approximation to provide a tractable benchmark that guides algorithm design. Building on this, we propose the Waiting for Accumulated Inference Threshold (WAIT) algorithm, which uses multiple thresholds to schedule incoming prompts optimally when output lengths are known, and extend it to Nested WAIT for cases with unknown output lengths. Theoretical analysis shows that both algorithms achieve near-optimal performance against the fluid benchmark in heavy traffic conditions, balancing throughput, latency, and Time to First Token (TTFT). Experiments with the Llama-7B model on an A100 GPU using both synthetic and real-world datasets demonstrate improved throughput and reduced latency relative to established baselines like vLLM and Sarathi. This work bridges operations research and machine learning, offering a rigorous framework for the efficient deployment of LLMs under memory constraints. △ Less

Submitted 15 April, 2025; originally announced April 2025.

Comments: 42 pages, 18 figures

Regularization and Selection in A Directed Network Model with Nodal Homophily and Nodal Effects

Authors: Zhaoyu Xing, Y. X. Rachel Wang, Andrew T. A. Wood, Tao Zou

Abstract: This article introduces a regularization and selection methods for directed networks with nodal homophily and nodal effects. The proposed approach not only preserves the statistical efficiency of the resulting estimator, but also ensures that the selection of nodal homophily and nodal effects is scalable with large-scale network data and multiple nodal features. In particular, we propose a directe… ▽ More This article introduces a regularization and selection methods for directed networks with nodal homophily and nodal effects. The proposed approach not only preserves the statistical efficiency of the resulting estimator, but also ensures that the selection of nodal homophily and nodal effects is scalable with large-scale network data and multiple nodal features. In particular, we propose a directed random network model with nodal homophily and nodal effects, which includes the nodal features in the probability density of random networks. Subsequently, we propose a regularized maximum likelihood estimator with an adaptive LASSO-type regularizer. We demonstrate that the regularized estimator exhibits the consistency and possesses the oracle properties. In addition, we propose a network Bayesian information criterion which ensures the selection consistency while tuning the model. Simulation experiments are conducted to demonstrate the excellent numerical performance. An online friendship network among musicians with nodal musical preference is used to illustrate the usefulness of the proposed new network model in network-related empirical analysis. △ Less

Submitted 6 April, 2025; originally announced April 2025.

Comments: 26 pages

MSC Class: 62J07; 62F12; 05C82

Balanced Direction from Multifarious Choices: Arithmetic Meta-Learning for Domain Generalization

Authors: Xiran Wang, Jian Zhang, Lei Qi, Yinghuan Shi

Abstract: Domain generalization is proposed to address distribution shift, arising from statistical disparities between training source and unseen target domains. The widely used first-order meta-learning algorithms demonstrate strong performance for domain generalization by leveraging the gradient matching theory, which aims to establish balanced parameters across source domains to reduce overfitting to an… ▽ More Domain generalization is proposed to address distribution shift, arising from statistical disparities between training source and unseen target domains. The widely used first-order meta-learning algorithms demonstrate strong performance for domain generalization by leveraging the gradient matching theory, which aims to establish balanced parameters across source domains to reduce overfitting to any particular domain. However, our analysis reveals that there are actually numerous directions to achieve gradient matching, with current methods representing just one possible path. These methods actually overlook another critical factor that the balanced parameters should be close to the centroid of optimal parameters of each source domain. To address this, we propose a simple yet effective arithmetic meta-learning with arithmetic-weighted gradients. This approach, while adhering to the principles of gradient matching, promotes a more precise balance by estimating the centroid between domain-specific optimal parameters. Experimental results validate the effectiveness of our strategy. △ Less

Submitted 23 March, 2025; originally announced March 2025.

Integrative Analysis of High-dimensional RCT and RWD Subject to Censoring and Hidden Confounding

Authors: Xin Ye, Shu Yang, Xiaofei Wang, Yanyan Liu

Abstract: In this study, we focus on estimating the heterogeneous treatment effect (HTE) for survival outcome. The outcome is subject to censoring and the number of covariates is high-dimensional. We utilize data from both the randomized controlled trial (RCT), considered as the gold standard, and real-world data (RWD), possibly affected by hidden confounding factors. To achieve a more efficient HTE estimat… ▽ More In this study, we focus on estimating the heterogeneous treatment effect (HTE) for survival outcome. The outcome is subject to censoring and the number of covariates is high-dimensional. We utilize data from both the randomized controlled trial (RCT), considered as the gold standard, and real-world data (RWD), possibly affected by hidden confounding factors. To achieve a more efficient HTE estimate, such integrative analysis requires great insight into the data generation mechanism, particularly the accurate characterization of unmeasured confounding effects/bias. With this aim, we propose a penalized-regression-based integrative approach that allows for the simultaneous estimation of parameters, selection of variables, and identification of the existence of unmeasured confounding effects. The consistency, asymptotic normality, and efficiency gains are rigorously established for the proposed estimate. Finally, we apply the proposed method to estimate the HTE of lobar/sublobar resection on the survival of lung cancer patients. The RCT is a multicenter non-inferiority randomized phase 3 trial, and the RWD comes from a clinical oncology cancer registry in the United States. The analysis reveals that the unmeasured confounding exists and the integrative approach does enhance the efficiency for the HTE estimation. △ Less

Submitted 20 March, 2025; originally announced March 2025.

Statistical Inference for Heterogeneous Treatment Effect with Right-censored Data from Synthesizing Randomized Clinical Trials and Real-world Data

Authors: Guangcai Mao, Shu Yang, Xiaofei Wang

Abstract: The heterogeneous treatment effect plays a crucial role in precision medicine. There is evidence that real-world data, even subject to biases, can be employed as supplementary evidence for randomized clinical trials to improve the statistical efficiency of the heterogeneous treatment effect estimation. In this paper, for survival data with right censoring, we consider estimating the heterogeneous… ▽ More The heterogeneous treatment effect plays a crucial role in precision medicine. There is evidence that real-world data, even subject to biases, can be employed as supplementary evidence for randomized clinical trials to improve the statistical efficiency of the heterogeneous treatment effect estimation. In this paper, for survival data with right censoring, we consider estimating the heterogeneous treatment effect, defined as the difference of the treatment-specific conditional restricted mean survival times given covariates, by synthesizing evidence from randomized clinical trials and the real-world data with possible biases. We define an omnibus confounding function to characterize the effect of biases caused by unmeasured confounders, censoring, outcome heterogeneity, and measurement error, and further, identify it by combining the trial and real-world data. We propose a penalized sieve method to estimate the heterogeneous treatment effect and the confounding function and further study the theoretical properties of the proposed integrative estimators based on the theory of reproducing kernel Hilbert space and empirical process. The proposed methodology is shown to outperform the approach solely based on the trial data through simulation studies and an integrative analysis of the data from a randomized trial and a real-world registry on early-stage non-small-cell lung cancer. △ Less

Submitted 19 March, 2025; originally announced March 2025.

Machine learning algorithms to predict stroke in China based on causal inference of time series analysis

Authors: Qizhi Zheng, Ayang Zhao, Xinzhu Wang, Yanhong Bai, Zikun Wang, Xiuying Wang, Xianzhang Zeng, Guanghui Dong

Abstract: Participants: This study employed a combination of Vector Autoregression (VAR) model and Graph Neural Networks (GNN) to systematically construct dynamic causal inference. Multiple classic classification algorithms were compared, including Random Forest, Logistic Regression, XGBoost, Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Gradient Boosting, and Multi Layer Perceptron (MLP). The SMO… ▽ More Participants: This study employed a combination of Vector Autoregression (VAR) model and Graph Neural Networks (GNN) to systematically construct dynamic causal inference. Multiple classic classification algorithms were compared, including Random Forest, Logistic Regression, XGBoost, Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Gradient Boosting, and Multi Layer Perceptron (MLP). The SMOTE algorithm was used to undersample a small number of samples and employed Stratified K-fold Cross Validation. Results: This study included a total of 11,789 participants, including 6,334 females (53.73%) and 5,455 males (46.27%), with an average age of 65 years. Introduction of dynamic causal inference features has significantly improved the performance of almost all models. The area under the ROC curve of each model ranged from 0.78 to 0.83, indicating significant difference (P < 0.01). Among all the models, the Gradient Boosting model demonstrated the highest performance and stability. Model explanation and feature importance analysis generated model interpretation that illustrated significant contributors associated with risks of stroke. Conclusions and Relevance: This study proposes a stroke risk prediction method that combines dynamic causal inference with machine learning models, significantly improving prediction accuracy and revealing key health factors that affect stroke. The research results indicate that dynamic causal inference features have important value in predicting stroke risk, especially in capturing the impact of changes in health status over time on stroke risk. By further optimizing the model and introducing more variables, this study provides theoretical basis and practical guidance for future stroke prevention and intervention strategies. △ Less

Submitted 10 March, 2025; originally announced March 2025.

Comments: 17 pages

CAD-VAE: Leveraging Correlation-Aware Latents for Comprehensive Fair Disentanglement

Authors: Chenrui Ma, Rongchang Zhao, Xi Xiao, Hongyang Xie, Tianyang Wang, Xiao Wang, Hao Zhang, Yanning Shen

Abstract: While deep generative models have significantly advanced representation learning, they may inherit or amplify biases and fairness issues by encoding sensitive attributes alongside predictive features. Enforcing strict independence in disentanglement is often unrealistic when target and sensitive factors are naturally correlated. To address this challenge, we propose CAD-VAE (Correlation-Aware Dise… ▽ More While deep generative models have significantly advanced representation learning, they may inherit or amplify biases and fairness issues by encoding sensitive attributes alongside predictive features. Enforcing strict independence in disentanglement is often unrealistic when target and sensitive factors are naturally correlated. To address this challenge, we propose CAD-VAE (Correlation-Aware Disentangled VAE), which introduces a correlated latent code to capture the shared information between target and sensitive attributes. Given this correlated latent, our method effectively separates overlapping factors without extra domain knowledge by directly minimizing the conditional mutual information between target and sensitive codes. A relevance-driven optimization strategy refines the correlated code by efficiently capturing essential correlated features and eliminating redundancy. Extensive experiments on benchmark datasets demonstrate that CAD-VAE produces fairer representations, realistic counterfactuals, and improved fairness-aware image editing. △ Less

Submitted 10 March, 2025; originally announced March 2025.

Bayesian Machine Learning for Estimating Optimal Dynamic Treatment Regimes with Ordinal Outcomes

Authors: Xinru Wang, Tanujit Chakraborty, Bibhas Chakraborty

Abstract: Dynamic treatment regimes (DTRs) are sequences of decision rules designed to tailor treatment based on patients' treatment history and evolving disease status. Ordinal outcomes frequently serve as primary endpoints in clinical trials and observational studies. However, constructing optimal DTRs for ordinal outcomes has been underexplored. This paper introduces a Bayesian machine learning (BML) fra… ▽ More Dynamic treatment regimes (DTRs) are sequences of decision rules designed to tailor treatment based on patients' treatment history and evolving disease status. Ordinal outcomes frequently serve as primary endpoints in clinical trials and observational studies. However, constructing optimal DTRs for ordinal outcomes has been underexplored. This paper introduces a Bayesian machine learning (BML) framework to handle ordinal outcomes in the DTR setting. To deal with potential nonlinear associations between outcomes and predictors, we first introduce ordinal Bayesian additive regression trees (OBART), a new model that integrates the latent variable framework within the traditional Bayesian additive regression trees (BART). We then incorporate OBART into the BML to estimate optimal DTRs based on ordinal data and quantify the associated uncertainties. Extensive simulation studies are conducted to evaluate the performance of the proposed approach against existing methods. We demonstrate the application of the proposed approach using data from a smoking cessation trial and provide the OBART R package along with R code for implementation. △ Less

Submitted 8 March, 2025; originally announced March 2025.

Identification of Genetic Factors Associated with Corpus Callosum Morphology: Conditional Strong Independence Screening for Non-Euclidean Responses

Authors: Zhe Gao, Jin Zhu, Yue Hu, Wenliang Pan, Xueqin Wang

Abstract: The corpus callosum, the largest white matter structure in the brain, plays a critical role in interhemispheric communication. Variations in its morphology are associated with various neurological and psychological conditions, making it a key focus in neurogenetics. Age is known to influence the structure and morphology of the corpus callosum significantly, complicating the identification of speci… ▽ More The corpus callosum, the largest white matter structure in the brain, plays a critical role in interhemispheric communication. Variations in its morphology are associated with various neurological and psychological conditions, making it a key focus in neurogenetics. Age is known to influence the structure and morphology of the corpus callosum significantly, complicating the identification of specific genetic factors that contribute to its shape and size. We propose a conditional strong independence screening method to address these challenges for ultrahigh-dimensional predictors and non-Euclidean responses. Our approach incorporates prior knowledge, such as age. It introduces a novel concept of conditional metric dependence, quantifying non-linear conditional dependencies among random objects in metric spaces without relying on predefined models. We apply this framework to identify genetic factors associated with the morphology of the corpus callosum. Simulation results demonstrate the efficacy of this method across various non-Euclidean data types, highlighting its potential to drive genetic discovery in neuroscience. △ Less

Submitted 3 March, 2025; originally announced March 2025.

DeepSuM: Deep Sufficient Modality Learning Framework

Authors: Zhe Gao, Jian Huang, Ting Li, Xueqin Wang

Abstract: Multimodal learning has become a pivotal approach in developing robust learning models with applications spanning multimedia, robotics, large language models, and healthcare. The efficiency of multimodal systems is a critical concern, given the varying costs and resource demands of different modalities. This underscores the necessity for effective modality selection to balance performance gains ag… ▽ More Multimodal learning has become a pivotal approach in developing robust learning models with applications spanning multimedia, robotics, large language models, and healthcare. The efficiency of multimodal systems is a critical concern, given the varying costs and resource demands of different modalities. This underscores the necessity for effective modality selection to balance performance gains against resource expenditures. In this study, we propose a novel framework for modality selection that independently learns the representation of each modality. This approach allows for the assessment of each modality's significance within its unique representation space, enabling the development of tailored encoders and facilitating the joint analysis of modalities with distinct characteristics. Our framework aims to enhance the efficiency and effectiveness of multimodal learning by optimizing modality integration and selection. △ Less

Submitted 3 March, 2025; originally announced March 2025.

Local Information for Global Network Estimation in Latent Space Models

Authors: Lijia Wang, Xiao Han, Yanhui Wu, Y. X. Rachel Wang

Abstract: In social networks, neighborhood is crucial for understanding individual behavior in response to environments, and thus it is essential to analyze an individual's local perspective within the global network. This paper studies how to utilize a partial information network centered around a given individual for global network estimation by fitting a general latent space model. Compared to the entire… ▽ More In social networks, neighborhood is crucial for understanding individual behavior in response to environments, and thus it is essential to analyze an individual's local perspective within the global network. This paper studies how to utilize a partial information network centered around a given individual for global network estimation by fitting a general latent space model. Compared to the entire network, the partial information network contains a significant proportion of missing edges with its structure depending on a random, potentially sparse neighborhood, posing significant challenges for estimation. We address the challenges by proposing a projected gradient descent algorithm for maximizing the likelihood of the observed data and develop theoretical guarantees for its convergence under different neighborhood structures. Our convergence rates and estimation error bounds highlight the impact of bias in an individual's local view of the global network, and we further show that the bias can be quantified with an imbalance measure. Using simulated and real networks, we demonstrate the performance of our estimation method and how our approach enables researchers to gain additional insights into the structure of social networks, such as the tradeoff between degrees and imbalance. △ Less

Submitted 23 February, 2025; originally announced February 2025.

Flow-based Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems

Authors: Xintong Wang, Xiaofei Guan, Ling Guo, Hao Wu

Abstract: Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems is a fundamental yet challenging problem in many fields of science and engineering. Existing methods face significant obstacles: Gaussian-based filters struggle with non-Gaussian distributions, while sequential Monte Carlo methods are computationally intensive and prone to particle degeneracy in high dimensions. Althoug… ▽ More Bayesian filtering for high-dimensional nonlinear stochastic dynamical systems is a fundamental yet challenging problem in many fields of science and engineering. Existing methods face significant obstacles: Gaussian-based filters struggle with non-Gaussian distributions, while sequential Monte Carlo methods are computationally intensive and prone to particle degeneracy in high dimensions. Although generative models in machine learning have made significant progress in modeling high-dimensional non-Gaussian distributions, their inefficiency in online updating limits their applicability to filtering problems. To address these challenges, we propose a flow-based Bayesian filter (FBF) that integrates normalizing flows to construct a novel latent linear state-space model with Gaussian filtering distributions. This framework facilitates efficient density estimation and sampling using invertible transformations provided by normalizing flows, and it enables the construction of filters in a data-driven manner, without requiring prior knowledge of system dynamics or observation models. Numerical experiments demonstrate the superior accuracy and efficiency of FBF. △ Less

Submitted 5 March, 2025; v1 submitted 22 February, 2025; originally announced February 2025.

Censor Dependent Variational Inference

Authors: Chuanhui Liu, Xiao Wang

Abstract: This paper provides a comprehensive analysis of variational inference in latent variable models for survival analysis, emphasizing the distinctive challenges associated with applying variational methods to survival data. We identify a critical weakness in the existing methodology, demonstrating how a poorly designed variational distribution may hinder the objective of survival analysis tasks - mod… ▽ More This paper provides a comprehensive analysis of variational inference in latent variable models for survival analysis, emphasizing the distinctive challenges associated with applying variational methods to survival data. We identify a critical weakness in the existing methodology, demonstrating how a poorly designed variational distribution may hinder the objective of survival analysis tasks - modeling time-to-event distributions. We prove that the optimal variational distribution, which perfectly bounds the log-likelihood, may depend on the censoring mechanism. To address this issue, we propose censor-dependent variational inference (CDVI), tailored for latent variable models in survival analysis. More practically, we introduce CD-CVAE, a V-structure Variational Autoencoder (VAE) designed for the scalable implementation of CDVI. Further discussion extends some existing theories and training techniques to survival analysis. Extensive experiments validate our analysis and demonstrate significant improvements in the estimation of individual survival distributions. △ Less

Submitted 3 June, 2025; v1 submitted 13 February, 2025; originally announced February 2025.

Factor Modelling for Biclustering Large-dimensional Matrix-valued Time Series

Authors: Yong He, Xiaoyang Ma, Xingheng Wang, Yalin Wang

Abstract: A novel unsupervised learning method is proposed in this paper for biclustering large-dimensional matrix-valued time series based on an entirely new latent two-way factor structure. Each block cluster is characterized by its own row and column cluster-specific factors in addition to some common matrix factors which impact on all the matrix time series. We first estimate the global loading spaces b… ▽ More A novel unsupervised learning method is proposed in this paper for biclustering large-dimensional matrix-valued time series based on an entirely new latent two-way factor structure. Each block cluster is characterized by its own row and column cluster-specific factors in addition to some common matrix factors which impact on all the matrix time series. We first estimate the global loading spaces by projecting the observation matrices onto the row or column loading space corresponding to common factors. The loading spaces for cluster-specific factors are then further recovered by projecting the observation matrices onto the orthogonal complement space of the estimated global loading spaces. To identify the latent row/column clusters simultaneously for matrix-valued time series, we provide a $K$-means algorithm based on the estimated row/column factor loadings of the cluster-specific weak factors. Theoretically, we derive faster convergence rates for global loading matrices than those of the state-of-the-art methods available in the literature under mild conditions. We also propose an one-pass eigenvalue-ratio method to estimate the numbers of global and cluster-specific factors. The consistency with explicit convergence rates is also established for the estimators of the local loading matrices, the factor numbers and the latent cluster memberships. Numerical experiments with both simulated data as well as a real data example are also reported to illustrate the usefulness of our proposed method. △ Less

Submitted 10 February, 2025; originally announced February 2025.

Distributed Primal-Dual Algorithms: Unification, Connections, and Insights

Authors: Runxiong Wu, Dong Liu, Xueqin Wang, Andi Wang

Abstract: We study primal-dual algorithms for general empirical risk minimization problems in distributed settings, focusing on two prominent classes of algorithms. The first class is the communication-efficient distributed dual coordinate ascent (CoCoA), derived from the coordinate ascent method for solving the dual problem. The second class is the alternating direction method of multipliers (ADMM), includ… ▽ More We study primal-dual algorithms for general empirical risk minimization problems in distributed settings, focusing on two prominent classes of algorithms. The first class is the communication-efficient distributed dual coordinate ascent (CoCoA), derived from the coordinate ascent method for solving the dual problem. The second class is the alternating direction method of multipliers (ADMM), including consensus ADMM, linearized ADMM, and proximal ADMM. We demonstrate that both classes of algorithms can be transformed into a unified update form that involves only primal and dual variables. This discovery reveals key connections between the two classes of algorithms: CoCoA can be interpreted as a special case of proximal ADMM for solving the dual problem, while consensus ADMM is closely related to a proximal ADMM algorithm. This discovery provides the insight that by adjusting the augmented Lagrangian parameter, we can easily enable the ADMM variants to outperform the CoCoA variants. We further explore linearized versions of ADMM and analyze the effects of tuning parameters on these ADMM variants in the distributed setting. Our theoretical findings are supported by extensive simulation studies and real-world data analysis. △ Less

Submitted 1 February, 2025; originally announced February 2025.

Comments: 15 pages, 4 figures, 1 table

Multi-agent Multi-armed Bandit with Fully Heavy-tailed Dynamics

Authors: Xingyu Wang, Mengfan Xu

Abstract: We study decentralized multi-agent multi-armed bandits in fully heavy-tailed settings, where clients communicate over sparse random graphs with heavy-tailed degree distributions and observe heavy-tailed (homogeneous or heterogeneous) reward distributions with potentially infinite variance. The objective is to maximize system performance by pulling the globally optimal arm with the highest global r… ▽ More We study decentralized multi-agent multi-armed bandits in fully heavy-tailed settings, where clients communicate over sparse random graphs with heavy-tailed degree distributions and observe heavy-tailed (homogeneous or heterogeneous) reward distributions with potentially infinite variance. The objective is to maximize system performance by pulling the globally optimal arm with the highest global reward mean across all clients. We are the first to address such fully heavy-tailed scenarios, which capture the dynamics and challenges in communication and inference among multiple clients in real-world systems. In homogeneous settings, our algorithmic framework exploits hub-like structures unique to heavy-tailed graphs, allowing clients to aggregate rewards and reduce noises via hub estimators when constructing UCB indices; under $M$ clients and degree distributions with power-law index $α> 1$, our algorithm attains a regret bound (almost) of order $O(M^{1 -\frac{1}α} \log{T})$. Under heterogeneous rewards, clients synchronize by communicating with neighbors, aggregating exchanged estimators in UCB indices; With our newly established information delay bounds on sparse random graphs, we prove a regret bound of $O(M \log{T})$. Our results improve upon existing work, which only address time-invariant connected graphs, or light-tailed dynamics in dense graphs and rewards. △ Less

Submitted 31 January, 2025; originally announced January 2025.

Comments: 40 pages

Noise-Adaptive Conformal Classification with Marginal Coverage

Authors: Teresa Bortolotti, Y. X. Rachel Wang, Xin Tong, Alessandra Menafoglio, Simone Vantini, Matteo Sesia

Abstract: Conformal inference provides a rigorous statistical framework for uncertainty quantification in machine learning, enabling well-calibrated prediction sets with precise coverage guarantees for any classification model. However, its reliance on the idealized assumption of perfect data exchangeability limits its effectiveness in the presence of real-world complications, such as low-quality labels --… ▽ More Conformal inference provides a rigorous statistical framework for uncertainty quantification in machine learning, enabling well-calibrated prediction sets with precise coverage guarantees for any classification model. However, its reliance on the idealized assumption of perfect data exchangeability limits its effectiveness in the presence of real-world complications, such as low-quality labels -- a widespread issue in modern large-scale data sets. This work tackles this open problem by introducing an adaptive conformal inference method capable of efficiently handling deviations from exchangeability caused by random label noise, leading to informative prediction sets with tight marginal coverage guarantees even in those challenging scenarios. We validate our method through extensive numerical experiments demonstrating its effectiveness on synthetic and real data sets, including CIFAR-10H and BigEarthNet. △ Less

Submitted 29 January, 2025; originally announced January 2025.

Joint Pricing and Resource Allocation: An Optimal Online-Learning Approach

Authors: Jianyu Xu, Xuan Wang, Yu-Xiang Wang, Jiashuo Jiang

Abstract: We study an online learning problem on dynamic pricing and resource allocation, where we make joint pricing and inventory decisions to maximize the overall net profit. We consider the stochastic dependence of demands on the price, which complicates the resource allocation process and introduces significant non-convexity and non-smoothness to the problem. To solve this problem, we develop an effici… ▽ More We study an online learning problem on dynamic pricing and resource allocation, where we make joint pricing and inventory decisions to maximize the overall net profit. We consider the stochastic dependence of demands on the price, which complicates the resource allocation process and introduces significant non-convexity and non-smoothness to the problem. To solve this problem, we develop an efficient algorithm that utilizes a "Lower-Confidence Bound (LCB)" meta-strategy over multiple OCO agents. Our algorithm achieves $\tilde{O}(\sqrt{Tmn})$ regret (for $m$ suppliers and $n$ consumers), which is optimal with respect to the time horizon $T$. Our results illustrate an effective integration of statistical learning methodologies with complex operations research problems. △ Less

Submitted 21 May, 2025; v1 submitted 29 January, 2025; originally announced January 2025.

MSC Class: 91B06; 90B22; 91B24; 90B50; 90B80; 62P20 ACM Class: I.2.6

Distributionally Robust Graph Out-of-Distribution Recommendation via Diffusion Model

Authors: Chu Zhao, Enneng Yang, Yuliang Liang, Jianzhe Zhao, Guibing Guo, Xingwei Wang

Abstract: The distributionally robust optimization (DRO)-based graph neural network methods improve recommendation systems' out-of-distribution (OOD) generalization by optimizing the model's worst-case performance. However, these studies fail to consider the impact of noisy samples in the training data, which results in diminished generalization capabilities and lower accuracy. Through experimental and theo… ▽ More The distributionally robust optimization (DRO)-based graph neural network methods improve recommendation systems' out-of-distribution (OOD) generalization by optimizing the model's worst-case performance. However, these studies fail to consider the impact of noisy samples in the training data, which results in diminished generalization capabilities and lower accuracy. Through experimental and theoretical analysis, this paper reveals that current DRO-based graph recommendation methods assign greater weight to noise distribution, leading to model parameter learning being dominated by it. When the model overly focuses on fitting noise samples in the training data, it may learn irrelevant or meaningless features that cannot be generalized to OOD data. To address this challenge, we design a Distributionally Robust Graph model for OOD recommendation (DRGO). Specifically, our method first employs a simple and effective diffusion paradigm to alleviate the noisy effect in the latent space. Additionally, an entropy regularization term is introduced in the DRO objective function to avoid extreme sample weights in the worst-case distribution. Finally, we provide a theoretical proof of the generalization error bound of DRGO as well as a theoretical analysis of how our approach mitigates noisy sample effects, which helps to better understand the proposed framework from a theoretical perspective. We conduct extensive experiments on four datasets to evaluate the effectiveness of our framework against three typical distribution shifts, and the results demonstrate its superiority in both independently and identically distributed distributions (IID) and OOD. △ Less

Submitted 26 January, 2025; originally announced January 2025.

Comments: 14 pages, Accepted by WWW'25

Estimating Committor Functions via Deep Adaptive Sampling on Rare Transition Paths

Authors: Yueyang Wang, Kejun Tang, Xili Wang, Xiaoliang Wan, Weiqing Ren, Chao Yang

Abstract: The committor functions are central to investigating rare but important events in molecular simulations. It is known that computing the committor function suffers from the curse of dimensionality. Recently, using neural networks to estimate the committor function has gained attention due to its potential for high-dimensional problems. Training neural networks to approximate the committor function… ▽ More The committor functions are central to investigating rare but important events in molecular simulations. It is known that computing the committor function suffers from the curse of dimensionality. Recently, using neural networks to estimate the committor function has gained attention due to its potential for high-dimensional problems. Training neural networks to approximate the committor function needs to sample transition data from straightforward simulations of rare events, which is very inefficient. The scarcity of transition data makes it challenging to approximate the committor function. To address this problem, we propose an efficient framework to generate data points in the transition state region that helps train neural networks to approximate the committor function. We design a Deep Adaptive Sampling method for TRansition paths (DASTR), where deep generative models are employed to generate samples to capture the information of transitions effectively. In particular, we treat a non-negative function in the integrand of the loss functional as an unnormalized probability density function and approximate it with the deep generative model. The new samples from the deep generative model are located in the transition state region and fewer samples are located in the other region. This distribution provides effective samples for approximating the committor function and significantly improves the accuracy. We demonstrate the effectiveness of the proposed method through both simulations and realistic examples. △ Less

Submitted 26 January, 2025; originally announced January 2025.

Quantitative Error Bounds for Scaling Limits of Stochastic Iterative Algorithms

Authors: Xiaoyu Wang, Mikolaj J. Kasprzak, Jeffrey Negrea, Solesne Bourguin, Jonathan H. Huggins

Abstract: Stochastic iterative algorithms, including stochastic gradient descent (SGD) and stochastic gradient Langevin dynamics (SGLD), are widely utilized for optimization and sampling in large-scale and high-dimensional problems in machine learning, statistics, and engineering. Numerous works have bounded the parameter error in, and characterized the uncertainty of, these approximations. One common appro… ▽ More Stochastic iterative algorithms, including stochastic gradient descent (SGD) and stochastic gradient Langevin dynamics (SGLD), are widely utilized for optimization and sampling in large-scale and high-dimensional problems in machine learning, statistics, and engineering. Numerous works have bounded the parameter error in, and characterized the uncertainty of, these approximations. One common approach has been to use scaling limit analyses to relate the distribution of algorithm sample paths to a continuous-time stochastic process approximation, particularly in asymptotic setups. Focusing on the univariate setting, in this paper, we build on previous work to derive non-asymptotic functional approximation error bounds between the algorithm sample paths and the Ornstein-Uhlenbeck approximation using an infinite-dimensional version of Stein's method of exchangeable pairs. We show that this bound implies weak convergence under modest additional assumptions and leads to a bound on the error of the variance of the iterate averages of the algorithm. Furthermore, we use our main result to construct error bounds in terms of two common metrics: the Lévy-Prokhorov and bounded Wasserstein distances. Our results provide a foundation for developing similar error bounds for the multivariate setting and for more sophisticated stochastic approximation algorithms. △ Less

Submitted 21 January, 2025; originally announced January 2025.

MSC Class: 60F17 (Primary) 60J60; 62-08; 68T05; 62E17 (Secondary)

Non-Reversible Langevin Algorithms for Constrained Sampling

Authors: Hengrong Du, Qi Feng, Changwei Tu, Xiaoyu Wang, Lingjiong Zhu

Abstract: We consider the constrained sampling problem where the goal is to sample from a target distribution on a constrained domain. We propose skew-reflected non-reversible Langevin dynamics (SRNLD), a continuous-time stochastic differential equation with skew-reflected boundary. We obtain non-asymptotic convergence rate of SRNLD to the target distribution in both total variation and 1-Wasserstein distan… ▽ More We consider the constrained sampling problem where the goal is to sample from a target distribution on a constrained domain. We propose skew-reflected non-reversible Langevin dynamics (SRNLD), a continuous-time stochastic differential equation with skew-reflected boundary. We obtain non-asymptotic convergence rate of SRNLD to the target distribution in both total variation and 1-Wasserstein distances. By breaking reversibility, we show that the convergence is faster than the special case of the reversible dynamics. Based on the discretization of SRNLD, we propose skew-reflected non-reversible Langevin Monte Carlo (SRNLMC), and obtain non-asymptotic discretization error from SRNLD, and convergence guarantees to the target distribution in 1-Wasserstein distance. We show better performance guarantees than the projected Langevin Monte Carlo in the literature that is based on the reversible dynamics. Numerical experiments are provided for both synthetic and real datasets to show efficiency of the proposed algorithms. △ Less

Submitted 14 April, 2025; v1 submitted 20 January, 2025; originally announced January 2025.

Comments: 35 pages, 9 figures, typos corrected

Bayesian analysis of nonlinear structured latent factor models using a Gaussian Process Prior

Authors: Yimang Zhang, Xiaorui Wang, Jian Qing Shi

Abstract: Factor analysis models are widely utilized in social and behavioral sciences, such as psychology, education, and marketing, to measure unobservable latent traits. In this article, we introduce a nonlinear structured latent factor analysis model which is more flexible to characterize the relationship between manifest variables and latent factors. The confirmatory identifiability of the latent facto… ▽ More Factor analysis models are widely utilized in social and behavioral sciences, such as psychology, education, and marketing, to measure unobservable latent traits. In this article, we introduce a nonlinear structured latent factor analysis model which is more flexible to characterize the relationship between manifest variables and latent factors. The confirmatory identifiability of the latent factor is discussed, ensuring the substantive interpretation of the latent factors. A Bayesian approach with a Gaussian process prior is proposed to estimate the unknown nonlinear function and the unknown parameters. Asymptotic results are established, including structural identifiability of the latent factors, consistency of the estimates of the unknown parameters and the unknown nonlinear function. Simulation studies and a real data analysis are conducted to investigate the performance of the proposed method. Simulation studies show our proposed method performs well in handling nonlinear model and successfully identifies the latent factors. Our analysis incorporates oil flow data, allowing us to uncover the underlying structure of latent nonlinear patterns. △ Less

Submitted 6 January, 2025; originally announced January 2025.

AverageTime: Enhance Long-Term Time Series Forecasting with Simple Averaging

Authors: Gaoxiang Zhao, Li Zhou, Xiaoqiang Wang

Abstract: Long-term time series forecasting focuses on leveraging historical data to predict future trends. The core challenge lies in effectively modeling dependencies both within sequences and channels. Convolutional Neural Networks and Linear models often excel in sequence modeling but frequently fall short in capturing complex channel dependencies. In contrast, Transformer-based models, with their atten… ▽ More Long-term time series forecasting focuses on leveraging historical data to predict future trends. The core challenge lies in effectively modeling dependencies both within sequences and channels. Convolutional Neural Networks and Linear models often excel in sequence modeling but frequently fall short in capturing complex channel dependencies. In contrast, Transformer-based models, with their attention mechanisms applied to both sequences and channels, have demonstrated strong predictive performance. Our research proposes a new approach for capturing sequence and channel dependencies: AverageTime, an exceptionally simple yet effective structure. By employing mixed channel embedding and averaging operations, AverageTime separately captures correlations for sequences and channels through channel mapping and result averaging. In addition, we integrate clustering methods to further accelerate the model's training process. Experiments on real-world datasets demonstrate that AverageTime surpasses state-of-the-art models in predictive performance while maintaining efficiency comparable to lightweight linear models. This provides a new and effective framework for modeling long time series. △ Less

Submitted 2 April, 2025; v1 submitted 30 December, 2024; originally announced December 2024.

On the Convergence of Min-Max Langevin Dynamics and Algorithm

Authors: Yang Cai, Siddharth Mitra, Xiuyuan Wang, Andre Wibisono

Abstract: We study zero-sum games in the space of probability distributions over the Euclidean space $\mathbb{R}^d$ with entropy regularization, in the setting when the interaction function between the players is smooth and strongly convex-strongly concave. We prove an exponential convergence guarantee for the mean-field min-max Langevin dynamics to compute the equilibrium distribution of the zero-sum game.… ▽ More We study zero-sum games in the space of probability distributions over the Euclidean space $\mathbb{R}^d$ with entropy regularization, in the setting when the interaction function between the players is smooth and strongly convex-strongly concave. We prove an exponential convergence guarantee for the mean-field min-max Langevin dynamics to compute the equilibrium distribution of the zero-sum game. We also study the finite-particle approximation of the mean-field min-max Langevin dynamics, both in continuous and discrete times. We prove biased convergence guarantees for the continuous-time finite-particle min-max Langevin dynamics to the stationary mean-field equilibrium distribution with an explicit bias term which does not scale with the number of particles. We also prove biased convergence guarantees for the discrete-time finite-particle min-max Langevin algorithm to the stationary mean-field equilibrium distribution with an additional bias term which scales with the step size and the number of particles. This provides an explicit iteration complexity for the average particle along the finite-particle algorithm to approximately compute the equilibrium distribution of the zero-sum game. △ Less

Submitted 27 June, 2025; v1 submitted 29 December, 2024; originally announced December 2024.

Comments: v3: Accepted for presentation at the Conference on Learning Theory (COLT) 2025. v2: Revised introduction and presentation of results