-
Statistical and Computational Guarantees of Kernel Max-Sliced Wasserstein Distances
Authors:
Jie Wang,
March Boedihardjo,
Yao Xie
Abstract:
Optimal transport has been very successful for various machine learning tasks; however, it is known to suffer from the curse of dimensionality. Hence, dimensionality reduction is desirable when applied to high-dimensional data with low-dimensional structures. The kernel max-sliced (KMS) Wasserstein distance is developed for this purpose by finding an optimal nonlinear mapping that reduces data int…
▽ More
Optimal transport has been very successful for various machine learning tasks; however, it is known to suffer from the curse of dimensionality. Hence, dimensionality reduction is desirable when applied to high-dimensional data with low-dimensional structures. The kernel max-sliced (KMS) Wasserstein distance is developed for this purpose by finding an optimal nonlinear mapping that reduces data into $1$ dimension before computing the Wasserstein distance. However, its theoretical properties have not yet been fully developed. In this paper, we provide sharp finite-sample guarantees under milder technical assumptions compared with state-of-the-art for the KMS $p$-Wasserstein distance between two empirical distributions with $n$ samples for general $p\in[1,\infty)$. Algorithm-wise, we show that computing the KMS $2$-Wasserstein distance is NP-hard, and then we further propose a semidefinite relaxation (SDR) formulation (which can be solved efficiently in polynomial time) and provide a relaxation gap for the obtained solution. We provide numerical examples to demonstrate the good performance of our scheme for high-dimensional two-sample testing.
△ Less
Submitted 2 February, 2025; v1 submitted 24 May, 2024;
originally announced May 2024.
-
Max-sliced 2-Wasserstein distance
Authors:
March T. Boedihardjo
Abstract:
This note is a continuation of the author's previous work on "Sharp bounds for the max-sliced Wasserstein distance." We use the same technique to obtain an upper bound for the expected max-sliced 2-Wasserstein distance between a compactly supported symmetric probability measure on a Euclidean space and its symmetrized empirical distribution.
This note is a continuation of the author's previous work on "Sharp bounds for the max-sliced Wasserstein distance." We use the same technique to obtain an upper bound for the expected max-sliced 2-Wasserstein distance between a compactly supported symmetric probability measure on a Euclidean space and its symmetrized empirical distribution.
△ Less
Submitted 14 March, 2024; v1 submitted 4 March, 2024;
originally announced March 2024.
-
Sharp bounds for max-sliced Wasserstein distances
Authors:
March T. Boedihardjo
Abstract:
We obtain essentially matching upper and lower bounds for the expected max-sliced 1-Wasserstein distance between a probability measure on a separable Hilbert space and its empirical distribution from $n$ samples. By proving a Banach space version of this result, we also obtain an upper bound, that is sharp up to a log factor, for the expected max-sliced 2-Wasserstein distance between a symmetric p…
▽ More
We obtain essentially matching upper and lower bounds for the expected max-sliced 1-Wasserstein distance between a probability measure on a separable Hilbert space and its empirical distribution from $n$ samples. By proving a Banach space version of this result, we also obtain an upper bound, that is sharp up to a log factor, for the expected max-sliced 2-Wasserstein distance between a symmetric probability measure $μ$ on a Euclidean space and its symmetrized empirical distribution in terms of the operator norm of the covariance matrix of $μ$ and the diameter of the support of $μ$.
△ Less
Submitted 26 November, 2024; v1 submitted 1 March, 2024;
originally announced March 2024.
-
A Performance Guarantee for Spectral Clustering
Authors:
March Boedihardjo,
Shaofeng Deng,
Thomas Strohmer
Abstract:
The two-step spectral clustering method, which consists of the Laplacian eigenmap and a rounding step, is a widely used method for graph partitioning. It can be seen as a natural relaxation to the NP-hard minimum ratio cut problem. In this paper we study the central question: when is spectral clustering able to find the global solution to the minimum ratio cut problem? First we provide a condition…
▽ More
The two-step spectral clustering method, which consists of the Laplacian eigenmap and a rounding step, is a widely used method for graph partitioning. It can be seen as a natural relaxation to the NP-hard minimum ratio cut problem. In this paper we study the central question: when is spectral clustering able to find the global solution to the minimum ratio cut problem? First we provide a condition that naturally depends on the intra- and inter-cluster connectivities of a given partition under which we may certify that this partition is the solution to the minimum ratio cut problem. Then we develop a deterministic two-to-infinity norm perturbation bound for the the invariant subspace of the graph Laplacian that corresponds to the $k$ smallest eigenvalues. Finally by combining these two results we give a condition under which spectral clustering is guaranteed to output the global solution to the minimum ratio cut problem, which serves as a performance guarantee for spectral clustering.
△ Less
Submitted 10 July, 2020;
originally announced July 2020.
-
DP-LSSGD: A Stochastic Optimization Method to Lift the Utility in Privacy-Preserving ERM
Authors:
Bao Wang,
Quanquan Gu,
March Boedihardjo,
Farzin Barekat,
Stanley J. Osher
Abstract:
Machine learning (ML) models trained by differentially private stochastic gradient descent (DP-SGD) have much lower utility than the non-private ones. To mitigate this degradation, we propose a DP Laplacian smoothing SGD (DP-LSSGD) to train ML models with differential privacy (DP) guarantees. At the core of DP-LSSGD is the Laplacian smoothing, which smooths out the Gaussian noise used in the Gauss…
▽ More
Machine learning (ML) models trained by differentially private stochastic gradient descent (DP-SGD) have much lower utility than the non-private ones. To mitigate this degradation, we propose a DP Laplacian smoothing SGD (DP-LSSGD) to train ML models with differential privacy (DP) guarantees. At the core of DP-LSSGD is the Laplacian smoothing, which smooths out the Gaussian noise used in the Gaussian mechanism. Under the same amount of noise used in the Gaussian mechanism, DP-LSSGD attains the same DP guarantee, but in practice, DP-LSSGD makes training both convex and nonconvex ML models more stable and enables the trained models to generalize better. The proposed algorithm is simple to implement and the extra computational complexity and memory overhead compared with DP-SGD are negligible. DP-LSSGD is applicable to train a large variety of ML models, including DNNs. The code is available at \url{https://github.com/BaoWangMath/DP-LSSGD}.
△ Less
Submitted 7 December, 2019; v1 submitted 28 June, 2019;
originally announced June 2019.
