-
Interpretable sparse SIR for functional data
Authors:
Victor Picheny,
Rémi Servien,
Nathalie Villa-Vialaneix
Abstract:
This work focuses on the issue of variable selection in functional regression. Unlike most work in this framework, our approach does not select isolated points in the definition domain of the predictors, nor does it rely on the expansion of the predictors in a given functional basis. It provides an approach to select full intervals made of consecutive points. This feature improves the interpretabi…
▽ More
This work focuses on the issue of variable selection in functional regression. Unlike most work in this framework, our approach does not select isolated points in the definition domain of the predictors, nor does it rely on the expansion of the predictors in a given functional basis. It provides an approach to select full intervals made of consecutive points. This feature improves the interpretability of the estimated coefficients and is desirable in the functional framework for which small shifts are frequent when comparing one predictor (curve) to another. Our method is described in a semiparametric framework based on Sliced Inverse Regression (SIR). SIR is an effective method for dimension reduction of high-dimensional data which computes a linear projection of the predictors in a low-dimensional space, without loss on regression information. We extend the approaches of variable selection developed for multidimensional SIR to select intervals rather than separated evaluation points in the definition domain of the functional predictors. Different and equivalent formulations of SIR are combined in a shrinkage approach with a group-LASSO-like penalty. Finally, a fully automated iterative procedure is also proposed to find the critical (interpretable) intervals. The approach is proved efficient on simulated and real data.
△ Less
Submitted 2 March, 2018; v1 submitted 2 June, 2016;
originally announced June 2016.
-
Random Forests for Big Data
Authors:
Robin Genuer,
Jean-Michel Poggi,
Christine Tuleau-Malot,
Nathalie Villa-Vialaneix
Abstract:
Big Data is one of the major challenges of statistical science and has numerous consequences from algorithmic and theoretical viewpoints. Big Data always involve massive data but they also often include online data and data heterogeneity. Recently some statistical methods have been adapted to process Big Data, like linear regression models, clustering methods and bootstrapping schemes. Based on d…
▽ More
Big Data is one of the major challenges of statistical science and has numerous consequences from algorithmic and theoretical viewpoints. Big Data always involve massive data but they also often include online data and data heterogeneity. Recently some statistical methods have been adapted to process Big Data, like linear regression models, clustering methods and bootstrapping schemes. Based on decision trees combined with aggregation and bootstrap ideas, random forests were introduced by Breiman in 2001. They are a powerful nonparametric statistical method allowing to consider in a single and versatile framework regression problems, as well as two-class and multi-class classification problems. Focusing on classification problems, this paper proposes a selective review of available proposals that deal with scaling random forests to Big Data problems. These proposals rely on parallel environments or on online adaptations of random forests. We also describe how related quantities -- such as out-of-bag error and variable importance -- are addressed in these methods. Then, we formulate various remarks for random forests in the Big Data context. Finally, we experiment five variants on two massive datasets (15 and 120 millions of observations), a simulated one as well as real world data. One variant relies on subsampling while three others are related to parallel implementations of random forests and involve either various adaptations of bootstrap to Big Data or to "divide-and-conquer" approaches. The fifth variant relates on online learning of random forests. These numerical experiments lead to highlight the relative performance of the different variants, as well as some of their limitations.
△ Less
Submitted 22 March, 2017; v1 submitted 26 November, 2015;
originally announced November 2015.
-
Statistique et Big Data Analytics; Volumétrie, L'Attaque des Clones
Authors:
Philippe Besse,
Nathalie Villa-Vialaneix
Abstract:
This article assumes acquired the skills and expertise of a statistician in unsupervised (NMF, k-means, SVD) and supervised learning (regression, CART, random forest). What skills and knowledge do a statistician must acquire to reach the "Volume" scale of big data? After a quick overview of the different strategies available and especially of those imposed by Hadoop, the algorithms of some availab…
▽ More
This article assumes acquired the skills and expertise of a statistician in unsupervised (NMF, k-means, SVD) and supervised learning (regression, CART, random forest). What skills and knowledge do a statistician must acquire to reach the "Volume" scale of big data? After a quick overview of the different strategies available and especially of those imposed by Hadoop, the algorithms of some available learning methods are outlined in order to understand how they are adapted to the strong stresses of the Map-Reduce functionalities
△ Less
Submitted 5 October, 2014; v1 submitted 26 May, 2014;
originally announced May 2014.
-
On-line relational SOM for dissimilarity data
Authors:
Madalina Olteanu,
Nathalie Villa-Vialaneix,
Marie Cottrell
Abstract:
In some applications and in order to address real world situations better, data may be more complex than simple vectors. In some examples, they can be known through their pairwise dissimilarities only. Several variants of the Self Organizing Map algorithm were introduced to generalize the original algorithm to this framework. Whereas median SOM is based on a rough representation of the prototypes,…
▽ More
In some applications and in order to address real world situations better, data may be more complex than simple vectors. In some examples, they can be known through their pairwise dissimilarities only. Several variants of the Self Organizing Map algorithm were introduced to generalize the original algorithm to this framework. Whereas median SOM is based on a rough representation of the prototypes, relational SOM allows representing these prototypes by a virtual combination of all elements in the data set. However, this latter approach suffers from two main drawbacks. First, its complexity can be large. Second, only a batch version of this algorithm has been studied so far and it often provides results having a bad topographic organization. In this article, an on-line version of relational SOM is described and justified. The algorithm is tested on several datasets, including categorical data and graphs, and compared with the batch version and with other SOM algorithms for non vector data.
△ Less
Submitted 27 December, 2012;
originally announced December 2012.
-
Neural Networks for Complex Data
Authors:
Marie Cottrell,
Madalina Olteanu,
Fabrice Rossi,
Joseph Rynkiewicz,
Nathalie Villa-Vialaneix
Abstract:
Artificial neural networks are simple and efficient machine learning tools. Defined originally in the traditional setting of simple vector data, neural network models have evolved to address more and more difficulties of complex real world problems, ranging from time evolving data to sophisticated data structures such as graphs and functions. This paper summarizes advances on those themes from the…
▽ More
Artificial neural networks are simple and efficient machine learning tools. Defined originally in the traditional setting of simple vector data, neural network models have evolved to address more and more difficulties of complex real world problems, ranging from time evolving data to sophisticated data structures such as graphs and functions. This paper summarizes advances on those themes from the last decade, with a focus on results obtained by members of the SAMM team of Université Paris 1
△ Less
Submitted 24 October, 2012;
originally announced October 2012.
-
Optimizing an Organized Modularity Measure for Topographic Graph Clustering: a Deterministic Annealing Approach
Authors:
Fabrice Rossi,
Nathalie Villa-Vialaneix
Abstract:
This paper proposes an organized generalization of Newman and Girvan's modularity measure for graph clustering. Optimized via a deterministic annealing scheme, this measure produces topologically ordered graph clusterings that lead to faithful and readable graph representations based on clustering induced graphs. Topographic graph clustering provides an alternative to more classical solutions in w…
▽ More
This paper proposes an organized generalization of Newman and Girvan's modularity measure for graph clustering. Optimized via a deterministic annealing scheme, this measure produces topologically ordered graph clusterings that lead to faithful and readable graph representations based on clustering induced graphs. Topographic graph clustering provides an alternative to more classical solutions in which a standard graph clustering method is applied to build a simpler graph that is then represented with a graph layout algorithm. A comparative study on four real world graphs ranging from 34 to 1 133 vertices shows the interest of the proposed approach with respect to classical solutions and to self-organizing maps for graphs.
△ Less
Submitted 7 September, 2010;
originally announced September 2010.
