Skip to main content

Showing 1–10 of 10 results for author: Tong, X T

Searching in archive cs. Search in all archives.
.
  1. arXiv:2505.04417  [pdf, other

    cs.LG math.NA stat.ML

    Localized Diffusion Models for High Dimensional Distributions Generation

    Authors: Georg A. Gottwald, Shuigen Liu, Youssef Marzouk, Sebastian Reich, Xin T. Tong

    Abstract: Diffusion models are the state-of-the-art tools for various generative tasks. However, estimating high-dimensional score functions makes them potentially suffer from the curse of dimensionality (CoD). This underscores the importance of better understanding and exploiting low-dimensional structure in the target distribution. In this work, we consider locality structure, which describes sparse depen… ▽ More

    Submitted 7 May, 2025; originally announced May 2025.

  2. arXiv:2505.02508  [pdf, ps, other

    stat.ML cs.LG math.ST

    Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces

    Authors: Yang Lyu, Yuchun Qian, Tan Minh Nguyen, Xin T. Tong

    Abstract: Diffusion models is a popular computational tool to generate new data samples. It utilizes a forward diffusion process that add noise to the data distribution and then use a reverse process to remove noises to produce samples from the data distribution. However, when the empirical data distribution consists of $n$ data point, using the empirical diffusion model will necessarily produce one of the… ▽ More

    Submitted 6 May, 2025; v1 submitted 5 May, 2025; originally announced May 2025.

  3. arXiv:2410.03292  [pdf, other

    cs.LG

    Demystifying the Token Dynamics of Deep Selective State Space Models

    Authors: Thieu N Vo, Tung D. Pham, Xin T. Tong, Tan Minh Nguyen

    Abstract: Selective state space models (SSM), such as Mamba, have gained prominence for their effectiveness in modeling sequential data. Despite their outstanding empirical performance, a comprehensive theoretical understanding of deep selective SSM remains elusive, hindering their further development and adoption for applications that need high fidelity. In this paper, we investigate the dynamical properti… ▽ More

    Submitted 7 March, 2025; v1 submitted 4 October, 2024; originally announced October 2024.

    Comments: Accepted at ICLR 2025 (spotlight)

  4. arXiv:2410.01195  [pdf, other

    cs.LG math.OC

    Stochastic Gradient Descent with Adaptive Data

    Authors: Ethan Che, Jing Dong, Xin T. Tong

    Abstract: Stochastic gradient descent (SGD) is a powerful optimization technique that is particularly useful in online learning scenarios. Its convergence analysis is relatively well understood under the assumption that the data samples are independent and identically distributed (iid). However, applying SGD to policy optimization problems in operations research involves a distinct challenge: the policy cha… ▽ More

    Submitted 1 October, 2024; originally announced October 2024.

  5. arXiv:2406.00914  [pdf, other

    math.OC cs.AI

    Wasserstein gradient flow for optimal probability measure decomposition

    Authors: Jiangze Han, Christopher Thomas Ryan, Xin T. Tong

    Abstract: We examine the infinite-dimensional optimization problem of finding a decomposition of a probability measure into K probability sub-measures to minimize specific loss functions inspired by applications in clustering and user grouping. We analytically explore the structures of the support of optimal sub-measures and introduce algorithms based on Wasserstein gradient flow, demonstrating their conver… ▽ More

    Submitted 2 June, 2024; originally announced June 2024.

  6. arXiv:2210.06447  [pdf, other

    cs.LG stat.ML

    Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descent

    Authors: Ruqi Zhang, Qiang Liu, Xin T. Tong

    Abstract: Sampling methods, as important inference and learning techniques, are typically designed for unconstrained domains. However, constraints are ubiquitous in machine learning problems, such as those on safety, fairness, robustness, and many other properties that must be satisfied to apply sampling results in real-life applications. Enforcing these constraints often leads to implicitly-defined manifol… ▽ More

    Submitted 12 October, 2022; originally announced October 2022.

    Comments: NeurIPS 2022

  7. arXiv:2205.08098  [pdf, other

    cs.LG stat.ML

    Can We Do Better Than Random Start? The Power of Data Outsourcing

    Authors: Yi Chen, Jing Dong, Xin T. Tong

    Abstract: Many organizations have access to abundant data but lack the computational power to process the data. While they can outsource the computational task to other facilities, there are various constraints on the amount of data that can be shared. It is natural to ask what can data outsourcing accomplish under such constraints. We address this question from a machine learning perspective. When training… ▽ More

    Submitted 17 May, 2022; originally announced May 2022.

    Comments: 22 pages, 5 figures

  8. arXiv:2202.02850  [pdf, ps, other

    cs.LG math.OC

    Stochastic Gradient Descent with Dependent Data for Offline Reinforcement Learning

    Authors: Jing Dong, Xin T. Tong

    Abstract: In reinforcement learning (RL), offline learning decoupled learning from data collection and is useful in dealing with exploration-exploitation tradeoff and enables data reuse in many applications. In this work, we study two offline learning tasks: policy evaluation and policy learning. For policy evaluation, we formulate it as a stochastic optimization problem and show that it can be solved using… ▽ More

    Submitted 6 February, 2022; originally announced February 2022.

  9. arXiv:2003.11196  [pdf, ps, other

    stat.ML cs.LG math.ST

    Dimension Independent Generalization Error by Stochastic Gradient Descent

    Authors: Xi Chen, Qiang Liu, Xin T. Tong

    Abstract: One classical canon of statistics is that large models are prone to overfitting, and model selection procedures are necessary for high dimensional data. However, many overparameterized models, such as neural networks, perform very well in practice, although they are often trained with simple online methods and regularization. The empirical success of overparameterized models, which is often known… ▽ More

    Submitted 4 January, 2021; v1 submitted 24 March, 2020; originally announced March 2020.

    Comments: 60 pages, 2 figures

  10. arXiv:1904.13016  [pdf, ps, other

    stat.ML cs.LG

    On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamics

    Authors: Xi Chen, Simon S. Du, Xin T. Tong

    Abstract: Stochastic gradient Langevin dynamics (SGLD) is a fundamental algorithm in stochastic optimization. Recent work by Zhang et al. [2017] presents an analysis for the hitting time of SGLD for the first and second order stationary points. The proof in Zhang et al. [2017] is a two-stage procedure through bounding the Cheeger's constant, which is rather complicated and leads to loose bounds. In this pap… ▽ More

    Submitted 15 March, 2020; v1 submitted 29 April, 2019; originally announced April 2019.

    Comments: 41 pages