-
On the Wasserstein Geodesic Principal Component Analysis of probability measures
Authors:
Nina Vesseron,
Elsa Cazelles,
Alice Le Brigant,
Thierry Klein
Abstract:
This paper focuses on Geodesic Principal Component Analysis (GPCA) on a collection of probability distributions using the Otto-Wasserstein geometry. The goal is to identify geodesic curves in the space of probability measures that best capture the modes of variation of the underlying dataset. We first address the case of a collection of Gaussian distributions, and show how to lift the computations…
▽ More
This paper focuses on Geodesic Principal Component Analysis (GPCA) on a collection of probability distributions using the Otto-Wasserstein geometry. The goal is to identify geodesic curves in the space of probability measures that best capture the modes of variation of the underlying dataset. We first address the case of a collection of Gaussian distributions, and show how to lift the computations in the space of invertible linear maps. For the more general setting of absolutely continuous probability measures, we leverage a novel approach to parameterizing geodesics in Wasserstein space with neural networks. Finally, we compare to classical tangent PCA through various examples and provide illustrations on real-world datasets.
△ Less
Submitted 4 June, 2025;
originally announced June 2025.
-
Active Third-Person Imitation Learning
Authors:
Timo Klein,
Susanna Weinberger,
Adish Singla,
Sebastian Tschiatschek
Abstract:
We consider the problem of third-person imitation learning with the additional challenge that the learner must select the perspective from which they observe the expert. In our setting, each perspective provides only limited information about the expert's behavior, and the learning agent must carefully select and combine information from different perspectives to achieve competitive performance. T…
▽ More
We consider the problem of third-person imitation learning with the additional challenge that the learner must select the perspective from which they observe the expert. In our setting, each perspective provides only limited information about the expert's behavior, and the learning agent must carefully select and combine information from different perspectives to achieve competitive performance. This setting is inspired by real-world imitation learning applications, e.g., in robotics, a robot might observe a human demonstrator via camera and receive information from different perspectives depending on the camera's position. We formalize the aforementioned active third-person imitation learning problem, theoretically analyze its characteristics, and propose a generative adversarial network-based active learning approach. Empirically, we demstrate that our proposed approach can effectively learn from expert demonstrations and explore the importance of different architectural choices for the learner's performance.
△ Less
Submitted 26 December, 2023;
originally announced December 2023.
-
Mixture-of-experts VAEs can disregard variation in surjective multimodal data
Authors:
Jannik Wolff,
Tassilo Klein,
Moin Nabi,
Rahul G. Krishnan,
Shinichi Nakajima
Abstract:
Machine learning systems are often deployed in domains that entail data from multiple modalities, for example, phenotypic and genotypic characteristics describe patients in healthcare. Previous works have developed multimodal variational autoencoders (VAEs) that generate several modalities. We consider subjective data, where single datapoints from one modality (such as class labels) describe multi…
▽ More
Machine learning systems are often deployed in domains that entail data from multiple modalities, for example, phenotypic and genotypic characteristics describe patients in healthcare. Previous works have developed multimodal variational autoencoders (VAEs) that generate several modalities. We consider subjective data, where single datapoints from one modality (such as class labels) describe multiple datapoints from another modality (such as images). We theoretically and empirically demonstrate that multimodal VAEs with a mixture of experts posterior can struggle to capture variability in such surjective data.
△ Less
Submitted 11 April, 2022;
originally announced April 2022.
-
Simultaneous x, y Pixel Estimation and Feature Extraction for Multiple Small Objects in a Scene: A Description of the ALIEN Network
Authors:
Seth Zuckerman,
Timothy Klein,
Alexander Boxer,
Christopher Goldman,
Brian Lang
Abstract:
We present a deep-learning network that detects multiple small objects (hundreds to thousands) in a scene while simultaneously estimating their x,y pixel locations together with a characteristic feature-set (for instance, target orientation and color). All estimations are performed in a single, forward pass which makes implementing the network fast and efficient. In this paper, we describe the arc…
▽ More
We present a deep-learning network that detects multiple small objects (hundreds to thousands) in a scene while simultaneously estimating their x,y pixel locations together with a characteristic feature-set (for instance, target orientation and color). All estimations are performed in a single, forward pass which makes implementing the network fast and efficient. In this paper, we describe the architecture of our network --- nicknamed ALIEN --- and detail its performance when applied to vehicle detection.
△ Less
Submitted 6 February, 2019;
originally announced February 2019.
-
A case study : Influence of Dimension Reduction on regression trees-based Algorithms -Predicting Aeronautics Loads of a Derivative Aircraft
Authors:
Edouard Fournier,
Stéphane Grihon,
Thierry Klein
Abstract:
In aircraft industry, market needs evolve quickly in a high competitiveness context. This requires adapting a given aircraft model in minimum time considering for example an increase of range or the number of passengers (cf A330 NEO family). The computation of loads and stress to resize the airframe is on the critical path of this aircraft variant definition: this is a consuming and costly process…
▽ More
In aircraft industry, market needs evolve quickly in a high competitiveness context. This requires adapting a given aircraft model in minimum time considering for example an increase of range or the number of passengers (cf A330 NEO family). The computation of loads and stress to resize the airframe is on the critical path of this aircraft variant definition: this is a consuming and costly process, one of the reason being the high dimen-sionality and the large amount of data. This is why Airbus has invested since a couple of years in Big Data approaches (statistic methods up to machine learning) to improve the speed, the data value extraction and the responsiveness of this process. This paper presents recent advances in this work made in cooperation between Airbus, ENAC and Institut de Math{é}-matiques de Toulouse in the framework of a proof of value sprint project. It compares the influence of three dimensional reduction techniques (PCA, polynomial fitting, combined) on the extrapolation capabilities of Regression Trees based algorithms for loads prediction. It shows that AdaBoost with Random Forest offers promising results in average in terms of accuracy and computational time to estimate loads on which a PCA is applied only on the outputs.
△ Less
Submitted 16 November, 2018;
originally announced December 2018.
-
Prediction of Preliminary Maximum Wing Bending Moments under Discrete Gust
Authors:
Edouard Fournier,
Stéphane Grihon,
Christian Bes,
Thierry Klein
Abstract:
Many methodologies have been proposed to quickly identify among a very large number of flight conditions and maneuvers (i.e., steady, quasi-steady and unsteady loads cases) the ones which give the worst values for structural sizing (e.g., bending moments, shear forces, torques,...). All of these methods use both the simulation model of the aircraft under development and efficient algorithms to fin…
▽ More
Many methodologies have been proposed to quickly identify among a very large number of flight conditions and maneuvers (i.e., steady, quasi-steady and unsteady loads cases) the ones which give the worst values for structural sizing (e.g., bending moments, shear forces, torques,...). All of these methods use both the simulation model of the aircraft under development and efficient algorithms to find out the critical points of the flight envelope. At the preliminary structural design phases detailed models are not available and airframe's loads are estimated by empirical relationships or engineering judgments. These approximations can induce load uncertainties and may lead to expensive redesign activities through the upcoming detailed sizing process. In the context of preliminary design phase for a weight aircraft variant without geometric change, to overcome this likely drawback, we propose a method based on the huge and reliable database of an initial aircraft from which the weight variant belongs. More precisely, from the load cases of this initial database, response surfaces are identified as functions of preliminary parameters (flight conditions and structural parameters). Then, these response surfaces are used to predict quickly the weight aircraft variant quantities of interest for preliminary structural design studies. Although the proposed method can be readily extended to any structural quantity of interest and to any flight conditions and maneuvers, it is presented here for the prediction of the bending moments due to discrete gust at different locations along a wing span.
△ Less
Submitted 13 November, 2018;
originally announced November 2018.
-
Semi Parametric Estimations of rotating and scaling parameters for aeronautic loads
Authors:
Edouard Fournier,
Stéphane Grihon,
Thierry Klein
Abstract:
In this paper, we perform registration of noisy curves. We provide an appropriate model in estimating the rotation and scaling parameters to adjust a set of curves through a M-estimation procedure. We prove the consistency and the asymptotic normality of our estimators. Numerical simulation and a real life aeronautic example are given to illustrate our methodology.
In this paper, we perform registration of noisy curves. We provide an appropriate model in estimating the rotation and scaling parameters to adjust a set of curves through a M-estimation procedure. We prove the consistency and the asymptotic normality of our estimators. Numerical simulation and a real life aeronautic example are given to illustrate our methodology.
△ Less
Submitted 15 November, 2018; v1 submitted 24 September, 2018;
originally announced September 2018.
-
Differentially Private Federated Learning: A Client Level Perspective
Authors:
Robin C. Geyer,
Tassilo Klein,
Moin Nabi
Abstract:
Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to share the data. However, the protocol is vulnerable to differential attacks, whic…
▽ More
Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to share the data. However, the protocol is vulnerable to differential attacks, which could originate from any party contributing during federated optimization. In such an attack, a client's contribution during training and information about their data set is revealed through analyzing the distributed model. We tackle this problem and propose an algorithm for client sided differential privacy preserving federated optimization. The aim is to hide clients' contributions during training, balancing the trade-off between privacy loss and model performance. Empirical studies suggest that given a sufficiently large number of participating clients, our proposed procedure can maintain client-level differential privacy at only a minor cost in model performance.
△ Less
Submitted 1 March, 2018; v1 submitted 20 December, 2017;
originally announced December 2017.
-
Classification with the nearest neighbor rule in general finite dimensional spaces: necessary and sufficient conditions
Authors:
Sébastien Gadat,
Thierry Klein,
Clément Marteau
Abstract:
Given an $n$-sample of random vectors $(X_i,Y_i)_{1 \leq i \leq n}$ whose joint law is unknown, the long-standing problem of supervised classification aims to \textit{optimally} predict the label $Y$ of a given a new observation $X$. In this context, the nearest neighbor rule is a popular flexible and intuitive method in non-parametric situations.
Even if this algorithm is commonly used in the m…
▽ More
Given an $n$-sample of random vectors $(X_i,Y_i)_{1 \leq i \leq n}$ whose joint law is unknown, the long-standing problem of supervised classification aims to \textit{optimally} predict the label $Y$ of a given a new observation $X$. In this context, the nearest neighbor rule is a popular flexible and intuitive method in non-parametric situations.
Even if this algorithm is commonly used in the machine learning and statistics communities, less is known about its prediction ability in general finite dimensional spaces, especially when the support of the density of the observations is $\mathbb{R}^d$. This paper is devoted to the study of the statistical properties of the nearest neighbor rule in various situations. In particular, attention is paid to the marginal law of $X$, as well as the smoothness and margin properties of the \textit{regression function} $η(X) = \mathbb{E}[Y | X]$. We identify two necessary and sufficient conditions to obtain uniform consistency rates of classification and to derive sharp estimates in the case of the nearest neighbor rule. Some numerical experiments are proposed at the end of the paper to help illustrate the discussion.
△ Less
Submitted 5 November, 2014; v1 submitted 4 November, 2014;
originally announced November 2014.
-
Sensitivity analysis for multidimensional and functional outputs
Authors:
Fabrice Gamboa,
Alexandre Janon,
Thierry Klein,
Agnès Lagnoux
Abstract:
Let $X:=(X_1, \ldots, X_p)$ be random objects (the inputs), defined on some probability space $(Ω,{\mathcal{F}}, \mathbb P)$ and valued in some measurable space $E=E_1\times\ldots \times E_p$. Further, let $Y:=Y = f(X_1, \ldots, X_p)$ be the output. Here, $f$ is a measurable function from $E$ to some Hilbert space $\mathbb{H}$ ($\mathbb{H}$ could be either of finite or infinite dimension). In this…
▽ More
Let $X:=(X_1, \ldots, X_p)$ be random objects (the inputs), defined on some probability space $(Ω,{\mathcal{F}}, \mathbb P)$ and valued in some measurable space $E=E_1\times\ldots \times E_p$. Further, let $Y:=Y = f(X_1, \ldots, X_p)$ be the output. Here, $f$ is a measurable function from $E$ to some Hilbert space $\mathbb{H}$ ($\mathbb{H}$ could be either of finite or infinite dimension). In this work, we give a natural generalization of the Sobol indices (that are classically defined when $Y\in\mathbb R$ ), when the output belongs to $\mathbb{H}$. These indices have very nice properties. First, they are invariant. under isometry and scaling. Further they can be, as in dimension $1$, easily estimated by using the so-called Pick and Freeze method. We investigate the asymptotic behaviour of such estimation scheme.
△ Less
Submitted 14 November, 2013; v1 submitted 7 November, 2013;
originally announced November 2013.
-
Geodesic PCA in the Wasserstein space
Authors:
Jérémie Bigot,
Raúl Gouet,
Thierry Klein,
Alfredo López
Abstract:
We introduce the method of Geodesic Principal Component Analysis (GPCA) on the space of probability measures on the line, with finite second moment, endowed with the Wasserstein metric. We discuss the advantages of this approach, over a standard functional PCA of probability densities in the Hilbert space of square-integrable functions. We establish the consistency of the method by showing that th…
▽ More
We introduce the method of Geodesic Principal Component Analysis (GPCA) on the space of probability measures on the line, with finite second moment, endowed with the Wasserstein metric. We discuss the advantages of this approach, over a standard functional PCA of probability densities in the Hilbert space of square-integrable functions. We establish the consistency of the method by showing that the empirical GPCA converges to its population counterpart, as the sample size tends to infinity. A key property in the study of GPCA is the isometry between the Wasserstein space and a closed convex subset of the space of square-integrable functions, with respect to an appropriate measure. Therefore, we consider the general problem of PCA in a closed convex subset of a separable Hilbert space, which serves as basis for the analysis of GPCA and also has interest in its own right. We provide illustrative examples on simple statistical models, to show the benefits of this approach for data analysis. The method is also applied to a real dataset of population pyramids.
△ Less
Submitted 3 October, 2014; v1 submitted 29 July, 2013;
originally announced July 2013.
-
New sensitivity analysis subordinated to a contrast
Authors:
Jean-Claude Fort,
Thierry Klein,
Nabil Rachdi
Abstract:
In a model of the form $Y=h(X_1,\ldots,X_d)$ where the goal is to estimate a parameter of the probability distribution of $Y$, we define new sensitivity indices which quantify the importance of each variable $X_i$ with respect to this parameter of interest. The aim of this paper is to define {\it goal oriented sensitivity indices} and we will show that Sobol indices are sensitivity indices associa…
▽ More
In a model of the form $Y=h(X_1,\ldots,X_d)$ where the goal is to estimate a parameter of the probability distribution of $Y$, we define new sensitivity indices which quantify the importance of each variable $X_i$ with respect to this parameter of interest. The aim of this paper is to define {\it goal oriented sensitivity indices} and we will show that Sobol indices are sensitivity indices associated to a particular characteristic of the distribution $Y$. We name the framework we present as {\it Goal Oriented Sensitivity Analysis} (GOSA).
△ Less
Submitted 10 May, 2013;
originally announced May 2013.
-
Asymptotic normality and efficiency of two Sobol index estimators
Authors:
Alexandre Janon,
Thierry Klein,
Agnes Lagnoux-Renaudie,
Maëlle Nodet,
Clémentine Prieur
Abstract:
Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest (output of the model). One of the statistical tools used to quantify the influence of each input variable on the output is the Sobol sensitivity index. We consider the stat…
▽ More
Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest (output of the model). One of the statistical tools used to quantify the influence of each input variable on the output is the Sobol sensitivity index. We consider the statistical estimation of this index from a finite sample of model outputs: we present two estimators and state a central limit theorem for each. We show that one of these estimators has an optimal asymptotic variance. We also generalize our results to the case where the true output is not observable, and is replaced by a noisy version.
△ Less
Submitted 26 March, 2013;
originally announced March 2013.
-
Statistical inference for Sobol pick freeze Monte Carlo method
Authors:
Fabrice Gamboa,
Alexandre Janon,
Thierry Klein,
Agnes Lagnoux-Renaudie,
Clémentine Prieur,
Clémentine Prieur
Abstract:
Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest (output of the model). One of the statistical tools used to quantify the influence of each input variable on the output is the Sobol sensitivity index. We consider the stat…
▽ More
Many mathematical models involve input parameters, which are not precisely known. Global sensitivity analysis aims to identify the parameters whose uncertainty has the largest impact on the variability of a quantity of interest (output of the model). One of the statistical tools used to quantify the influence of each input variable on the output is the Sobol sensitivity index. We consider the statistical estimation of this index from a finite sample of model outputs. We study asymptotic and non-asymptotic properties of two estimators of Sobol indices. These properties are applied to significance tests and estimation by confidence intervals.
△ Less
Submitted 26 March, 2013;
originally announced March 2013.
-
Sensitivity indices for multivariate outputs
Authors:
Fabrice Gamboa,
Alexandre Janon,
Thierry Klein,
Agnès Lagnoux
Abstract:
We define and study a generalization of Sobol sensitivity indices for the case of a vector output.
We define and study a generalization of Sobol sensitivity indices for the case of a vector output.
△ Less
Submitted 17 April, 2013; v1 submitted 14 March, 2013;
originally announced March 2013.
