-
Gaussian-Smoothed Sliced Probability Divergences
Authors:
Mokhtar Z. Alaya,
Alain Rakotomamonjy,
Maxime Berar,
Gilles Gasso
Abstract:
Gaussian smoothed sliced Wasserstein distance has been recently introduced for comparing probability distributions, while preserving privacy on the data. It has been shown that it provides performances similar to its non-smoothed (non-private) counterpart. However, the computationaland statistical properties of such a metric have not yet been well-established. This work investigates the theoretica…
▽ More
Gaussian smoothed sliced Wasserstein distance has been recently introduced for comparing probability distributions, while preserving privacy on the data. It has been shown that it provides performances similar to its non-smoothed (non-private) counterpart. However, the computationaland statistical properties of such a metric have not yet been well-established. This work investigates the theoretical properties of this distance as well as those of generalized versions denoted as Gaussian-smoothed sliced divergences. We first show that smoothing and slicing preserve the metric property and the weak topology. To study the sample complexity of such divergences, we then introduce $\hat{\hatμ}_{n}$ the double empirical distribution for the smoothed-projected $μ$. The distribution $\hat{\hatμ}_{n}$ is a result of a double sampling process: one from sampling according to the origin distribution $μ$ and the second according to the convolution of the projection of $μ$ on the unit sphere and the Gaussian smoothing. We particularly focus on the Gaussian smoothed sliced Wasserstein distance and prove that it converges with a rate $O(n^{-1/2})$. We also derive other properties, including continuity, of different divergences with respect to the smoothing parameter. We support our theoretical findings with empirical studies in the context of privacy-preserving domain adaptation.
△ Less
Submitted 25 April, 2024; v1 submitted 4 April, 2024;
originally announced April 2024.
-
Unbalanced Optimal Transport through Non-negative Penalized Linear Regression
Authors:
Laetitia Chapel,
Rémi Flamary,
Haoran Wu,
Cédric Févotte,
Gilles Gasso
Abstract:
This paper addresses the problem of Unbalanced Optimal Transport (UOT) in which the marginal conditions are relaxed (using weighted penalties in lieu of equality) and no additional regularization is enforced on the OT plan. In this context, we show that the corresponding optimization problem can be reformulated as a non-negative penalized linear regression problem. This reformulation allows us to…
▽ More
This paper addresses the problem of Unbalanced Optimal Transport (UOT) in which the marginal conditions are relaxed (using weighted penalties in lieu of equality) and no additional regularization is enforced on the OT plan. In this context, we show that the corresponding optimization problem can be reformulated as a non-negative penalized linear regression problem. This reformulation allows us to propose novel algorithms inspired from inverse problems and nonnegative matrix factorization. In particular, we consider majorization-minimization which leads in our setting to efficient multiplicative updates for a variety of penalties. Furthermore, we derive for the first time an efficient algorithm to compute the regularization path of UOT with quadratic penalties. The proposed algorithm provides a continuity of piece-wise linear OT plans converging to the solution of balanced OT (corresponding to infinite penalty weights). We perform several numerical experiments on simulated and real data illustrating the new algorithms, and provide a detailed discussion about more sophisticated optimization tools that can further be used to solve OT problems thanks to our reformulation.
△ Less
Submitted 8 June, 2021;
originally announced June 2021.
-
Chance Constrained Optimization for Energy Management in Electric Vehicles
Authors:
Erfan Mohagheghi,
Joan Gubianes Gasso,
Abebe Geletu,
Pu Li
Abstract:
E-powertrain of future electric vehicles could consist of energy generation units (e.g., fuel cells and photovoltaic modules), energy storage systems (e.g., batteries and supercapacitors), energy conversion units (e.g., bidirectional DC/DC converters and DC/AC inverters) and an electric machine, which can work in both generating and motoring modes [1- 6]. An energy management system is responsible…
▽ More
E-powertrain of future electric vehicles could consist of energy generation units (e.g., fuel cells and photovoltaic modules), energy storage systems (e.g., batteries and supercapacitors), energy conversion units (e.g., bidirectional DC/DC converters and DC/AC inverters) and an electric machine, which can work in both generating and motoring modes [1- 6]. An energy management system is responsible to operate the above-mentioned components in a way that the technical constraints are satisfied. This task should be accomplished by solving an optimization problem, which could aim at minimizing the total operation costs [5]. The optimization problem has been widely addressed by deterministic approaches [7], which take into account the forecasted values of active-reactive load profile. However, as shown in Figure 1 (a), it is impossible to accurately forecast the values, meaning that the solutions coming from deterministic approaches could lead to infeasible operations (i.e., constraint violations). Therefore, stochastic optimization approaches [8] should be utilized to fi nd optimal solution strategies while considering uncertain parameters.
△ Less
Submitted 5 December, 2020;
originally announced December 2020.
-
Importance sampling strategy for non-convex randomized block-coordinate descent
Authors:
Rémi Flamary,
Alain Rakotomamonjy,
Gilles Gasso
Abstract:
As the number of samples and dimensionality of optimization problems related to statistics an machine learning explode, block coordinate descent algorithms have gained popularity since they reduce the original problem to several smaller ones. Coordinates to be optimized are usually selected randomly according to a given probability distribution. We introduce an importance sampling strategy that he…
▽ More
As the number of samples and dimensionality of optimization problems related to statistics an machine learning explode, block coordinate descent algorithms have gained popularity since they reduce the original problem to several smaller ones. Coordinates to be optimized are usually selected randomly according to a given probability distribution. We introduce an importance sampling strategy that helps randomized coordinate descent algorithms to focus on blocks that are still far from convergence. The framework applies to problems composed of the sum of two possibly non-convex terms, one being separable and non-smooth. We have compared our algorithm to a full gradient proximal approach as well as to a randomized block coordinate algorithm that considers uniform sampling and cyclic block coordinate descent. Experimental evidences show the clear benefit of using an importance sampling strategy.
△ Less
Submitted 23 June, 2016;
originally announced June 2016.
-
DC Proximal Newton for Non-Convex Optimization Problems
Authors:
Alain Rakotomamonjy,
Remi Flamary,
Gilles Gasso
Abstract:
We introduce a novel algorithm for solving learning problems where both the loss function and the regularizer are non-convex but belong to the class of difference of convex (DC) functions. Our contribution is a new general purpose proximal Newton algorithm that is able to deal with such a situation. The algorithm consists in obtaining a descent direction from an approximation of the loss function…
▽ More
We introduce a novel algorithm for solving learning problems where both the loss function and the regularizer are non-convex but belong to the class of difference of convex (DC) functions. Our contribution is a new general purpose proximal Newton algorithm that is able to deal with such a situation. The algorithm consists in obtaining a descent direction from an approximation of the loss function and then in performing a line search to ensure sufficient descent. A theoretical analysis is provided showing that the iterates of the proposed algorithm {admit} as limit points stationary points of the DC objective function. Numerical experiments show that our approach is more efficient than current state of the art for a problem with a convex loss functions and non-convex regularizer. We have also illustrated the benefit of our algorithm in high-dimensional transductive learning problem where both loss function and regularizers are non-convex.
△ Less
Submitted 2 July, 2015;
originally announced July 2015.
