-
Classification non supervis{é}e des processus d'{é}v{é}nements r{é}currents
Authors:
Génia Babykina,
Vincent Vandewalle
Abstract:
Event of the same type occurring several times for one individual (recurrent events) are present in various domains (industrial systems reliability, episodes of unemployment, political conflicts, chronic diseases episodes). Analysis of such kind of data should account for the whole recurrence process dynamics rather than only focusing on the number of observed events. Statistical models for recurr…
▽ More
Event of the same type occurring several times for one individual (recurrent events) are present in various domains (industrial systems reliability, episodes of unemployment, political conflicts, chronic diseases episodes). Analysis of such kind of data should account for the whole recurrence process dynamics rather than only focusing on the number of observed events. Statistical models for recurrent events analysis are developed in the counting process probabilistic framework. One of the often-used models is the Andersen-Gill model, a generalization of the well-known Cox model for durations, which assumes that the baseline intensity of the recurrence process is time-dependent and is adjusted for covariates. For an individual i with covariates Xi, the intensity is as follows: $λ$_{ik}(t;$θ$) = $λ$_0(t) exp (X_i $β$). The baseline intensity can be specified parametrically, in a form of Weibull: $λ$_0 (t) = $γ$_{1} $γ$_{2} t^{$γ$_2-1}, with $γ$1 scale parameter et $γ$2 shape parameter. However, the observed covariates are often insufficient to explain the observed heterogeneity in data. This is often the case of clinical trials data containing information on patients. In this article a mixture model for recurrent events analysis is proposed. This model allows to account for unobserved heterogeneity and to cluster individuals according to their recurrence process. The intensity of the process is parametrically specified within each class and depend on observed covariates. Thus, the intensity becomes specific to class k: $λ$_{ik} (t; $θ$_k) = $γ$_{1k} $γ$_{2k} t^{$γ$_{2k}-1} exp (X_i $β$_k). The model parameters are estimated by the Maximum Likelihood method, using the EM algorithm. The BIC criterion is employed to choose the optimal number of classes. Model feasibility is verified by Monte Carlo simulations. An application to real data concerning hospital readmissions of elderly patients is proposed. The proposed model feasibility is empirically verified (the optimization algorithm converges, providing non-biased estimates). The real data application allows to identify two clinically relevant classes of patients.
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
Non-parametric Multi-Partitions Clustering
Authors:
Marie du Roy de Chaumaray,
Vincent Vandewalle
Abstract:
In the framework of model-based clustering, a model, called multi-partitions clustering, allowing several latent class variables has been proposed. This model assumes that the distribution of the observed data can be factorized into several independent blocks of variables, each block following its own mixture model. In this paper, we assume that each block follows a non parametric latent class mod…
▽ More
In the framework of model-based clustering, a model, called multi-partitions clustering, allowing several latent class variables has been proposed. This model assumes that the distribution of the observed data can be factorized into several independent blocks of variables, each block following its own mixture model. In this paper, we assume that each block follows a non parametric latent class model, {\it i.e.} independence of the variables in each component of the mixture with no parametric assumption on their class conditional distribution. The purpose is to deduce, from the observation of a sample, the number of blocks, the partition of the variables into the blocks and the number of components in each block, which characterise the proposed model. By following recent literature on model and variable selection in non-parametric mixture models, we propose to discretize the data into bins. This permits to apply the classical multi-partition clustering procedure for parametric multinomials, which are based on a penalized likelihood method (\emph{e.g.} BIC). The consistency of the procedure is obtained and an efficient optimization is proposed. The performances of the model are investigated on simulated data.
△ Less
Submitted 6 January, 2023;
originally announced January 2023.
-
Simultaneous semi-parametric estimation of clustering and regression
Authors:
Matthieu Marbac,
Mohammed Sedki,
Christophe Biernacki,
Vincent Vandewalle
Abstract:
We investigate the parameter estimation of regression models with fixed group effects, when the group variable is missing while group related variables are available. This problem involves clustering to infer the missing group variable based on the group related variables, and regression to build a model on the target variable given the group and eventually additional variables. Thus, this problem…
▽ More
We investigate the parameter estimation of regression models with fixed group effects, when the group variable is missing while group related variables are available. This problem involves clustering to infer the missing group variable based on the group related variables, and regression to build a model on the target variable given the group and eventually additional variables. Thus, this problem can be formulated as the joint distribution modeling of the target and of the group related variables. The usual parameter estimation strategy for this joint model is a two-step approach starting by learning the group variable (clustering step) and then plugging in its estimator for fitting the regression model (regression step). However, this approach is suboptimal (providing in particular biased regression estimates) since it does not make use of the target variable for clustering. Thus, we claim for a simultaneous estimation approach of both clustering and regression, in a semi-parametric framework. Numerical experiments illustrate the benefits of our proposition by considering wide ranges of distributions and regression models. The relevance of our new method is illustrated on real data dealing with problems associated with high blood pressure prevention.
△ Less
Submitted 28 December, 2020;
originally announced December 2020.
-
An end-to-end data-driven optimisation framework for constrained trajectories
Authors:
Florent Dewez,
Benjamin Guedj,
Arthur Talpaert,
Vincent Vandewalle
Abstract:
Many real-world problems require to optimise trajectories under constraints. Classical approaches are based on optimal control methods but require an exact knowledge of the underlying dynamics, which could be challenging or even out of reach. In this paper, we leverage data-driven approaches to design a new end-to-end framework which is dynamics-free for optimised and realistic trajectories. We fi…
▽ More
Many real-world problems require to optimise trajectories under constraints. Classical approaches are based on optimal control methods but require an exact knowledge of the underlying dynamics, which could be challenging or even out of reach. In this paper, we leverage data-driven approaches to design a new end-to-end framework which is dynamics-free for optimised and realistic trajectories. We first decompose the trajectories on function basis, trading the initial infinite dimension problem on a multivariate functional space for a parameter optimisation problem. A maximum \emph{a posteriori} approach which incorporates information from data is used to obtain a new optimisation problem which is regularised. The penalised term focuses the search on a region centered on data and includes estimated linear constraints in the problem. We apply our data-driven approach to two settings in aeronautics and sailing routes optimisation, yielding commanding results. The developed approach has been implemented in the Python library PyRotor.
△ Less
Submitted 5 February, 2021; v1 submitted 23 November, 2020;
originally announced November 2020.
-
From industry-wide parameters to aircraft-centric on-flight inference: improving aeronautics performance prediction with machine learning
Authors:
Florent Dewez,
Benjamin Guedj,
Vincent Vandewalle
Abstract:
Aircraft performance models play a key role in airline operations, especially in planning a fuel-efficient flight. In practice, manufacturers provide guidelines which are slightly modified throughout the aircraft life cycle via the tuning of a single factor, enabling better fuel predictions. However this has limitations, in particular they do not reflect the evolution of each feature impacting the…
▽ More
Aircraft performance models play a key role in airline operations, especially in planning a fuel-efficient flight. In practice, manufacturers provide guidelines which are slightly modified throughout the aircraft life cycle via the tuning of a single factor, enabling better fuel predictions. However this has limitations, in particular they do not reflect the evolution of each feature impacting the aircraft performance. Our goal here is to overcome this limitation. The key contribution of the present article is to foster the use of machine learning to leverage the massive amounts of data continuously recorded during flights performed by an aircraft and provide models reflecting its actual and individual performance. We illustrate our approach by focusing on the estimation of the drag and lift coefficients from recorded flight data. As these coefficients are not directly recorded, we resort to aerodynamics approximations. As a safety check, we provide bounds to assess the accuracy of both the aerodynamics approximation and the statistical performance of our approach. We provide numerical results on a collection of machine learning algorithms. We report excellent accuracy on real-life data and exhibit empirical evidence to support our modelling, in coherence with aerodynamics principles.
△ Less
Submitted 4 February, 2021; v1 submitted 11 May, 2020;
originally announced May 2020.
-
Feature quantization for parsimonious and interpretable predictive models
Authors:
Adrien Ehrhardt,
Christophe Biernacki,
Vincent Vandewalle,
Philippe Heinrich
Abstract:
For regulatory and interpretability reasons, logistic regression is still widely used. To improve prediction accuracy and interpretability, a preprocessing step quantizing both continuous and categorical data is usually performed: continuous features are discretized and, if numerous, levels of categorical features are grouped. An even better predictive accuracy can be reached by embedding this qua…
▽ More
For regulatory and interpretability reasons, logistic regression is still widely used. To improve prediction accuracy and interpretability, a preprocessing step quantizing both continuous and categorical data is usually performed: continuous features are discretized and, if numerous, levels of categorical features are grouped. An even better predictive accuracy can be reached by embedding this quantization estimation step directly into the predictive estimation step itself. But doing so, the predictive loss has to be optimized on a huge set. To overcome this difficulty, we introduce a specific two-step optimization strategy: first, the optimization problem is relaxed by approximating discontinuous quantization functions by smooth functions; second, the resulting relaxed optimization problem is solved via a particular neural network. The good performances of this approach, which we call glmdisc, are illustrated on simulated and real data from the UCI library and Crédit Agricole Consumer Finance (a major European historic player in the consumer credit market).
△ Less
Submitted 21 March, 2019;
originally announced March 2019.
-
A tractable Multi-Partitions Clustering
Authors:
Matthieu Marbac,
Vincent Vandewalle
Abstract:
In the framework of model-based clustering, a model allowing several latent class variables is proposed. This model assumes that the distribution of the observed data can be factorized into several independent blocks of variables. Each block is assumed to follow a latent class model ({\it i.e.,} mixture with conditional independence assumption). The proposed model includes variable selection, as a…
▽ More
In the framework of model-based clustering, a model allowing several latent class variables is proposed. This model assumes that the distribution of the observed data can be factorized into several independent blocks of variables. Each block is assumed to follow a latent class model ({\it i.e.,} mixture with conditional independence assumption). The proposed model includes variable selection, as a special case, and is able to cope with the mixed-data setting. The simplicity of the model allows to estimate the repartition of the variables into blocks and the mixture parameters simultaneously, thus avoiding to run EM algorithms for each possible repartition of variables into blocks. For the proposed method, a model is defined by the number of blocks, the number of clusters inside each block and the repartition of variables into block. Model selection can be done with two information criteria, the BIC and the MICL, for which an efficient optimization is proposed. The performances of the model are investigated on simulated and real data. It is shown that the proposed method gives a rich interpretation of the dataset at hand ({\it i.e.,} analysis of the repartition of the variables into blocks and analysis of the clusters produced by each block of variables).
△ Less
Submitted 22 January, 2018;
originally announced January 2018.
-
Model-based clustering of Gaussian copulas for mixed data
Authors:
Matthieu Marbac,
Christophe Biernacki,
Vincent Vandewalle
Abstract:
Clustering task of mixed data is a challenging problem. In a probabilistic framework, the main difficulty is due to a shortage of conventional distributions for such data. In this paper, we propose to achieve the mixed data clustering with a Gaussian copula mixture model, since copulas, and in particular the Gaussian ones, are powerful tools for easily modelling the distribution of multivariate va…
▽ More
Clustering task of mixed data is a challenging problem. In a probabilistic framework, the main difficulty is due to a shortage of conventional distributions for such data. In this paper, we propose to achieve the mixed data clustering with a Gaussian copula mixture model, since copulas, and in particular the Gaussian ones, are powerful tools for easily modelling the distribution of multivariate variables. Indeed, considering a mixing of continuous, integer and ordinal variables (thus all having a cumulative distribution function), this copula mixture model defines intra-component dependencies similar to a Gaussian mixture, so with classical correlation meaning. Simultaneously, it preserves standard margins associated to continuous, integer and ordered features, namely the Gaussian, the Poisson and the ordered multinomial distributions. As an interesting by-product, the proposed mixture model generalizes many well-known ones and also provides tools of visualization based on the parameters. At a practical level, the Bayesian inference is retained and it is achieved with a Metropolis-within-Gibbs sampler. Experiments on simulated and real data sets finally illustrate the expected advantages of the proposed model for mixed data: flexible and meaningful parametrization combined with visualization features.
△ Less
Submitted 29 September, 2015; v1 submitted 6 May, 2014;
originally announced May 2014.
-
Finite mixture model of conditional dependencies modes to cluster categorical data
Authors:
Matthieu Marbac,
Christophe Biernacki,
Vincent Vandewalle
Abstract:
We propose a parsimonious extension of the classical latent class model to cluster categorical data by relaxing the class conditional independence assumption. Under this new mixture model, named Conditional Modes Model, variables are grouped into conditionally independent blocks. The corresponding block distribution is a parsimonious multinomial distribution where the few free parameters correspon…
▽ More
We propose a parsimonious extension of the classical latent class model to cluster categorical data by relaxing the class conditional independence assumption. Under this new mixture model, named Conditional Modes Model, variables are grouped into conditionally independent blocks. The corresponding block distribution is a parsimonious multinomial distribution where the few free parameters correspond to the most likely modality crossings, while the remaining probability mass is uniformly spread over the other modality crossings. Thus, the proposed model allows to bring out the intra-class dependency between variables and to summarize each class by a few characteristic modality crossings. The model selection is performed via a Metropolis-within-Gibbs sampler to overcome the computational intractability of the block structure search. As this approach involves the computation of the integrated complete-data likelihood, we propose a new method (exact for the continuous parameters and approximated for the discrete ones) which avoids the biases of the \textsc{bic} criterion pointed out by our experiments. Finally, the parameters are only estimated for the best model via an \textsc{em} algorithm. The characteristics of the new model are illustrated on simulated data and on two biological data sets. These results strengthen the idea that this simple model allows to reduce biases involved by the conditional independence assumption and gives meaningful parameters. Both applications were performed with the R package \texttt{CoModes}
△ Less
Submitted 20 February, 2014;
originally announced February 2014.
-
Model-based clustering for conditionally correlated categorical data
Authors:
Matthieu Marbac,
Christophe Biernacki,
Vincent Vandewalle
Abstract:
An extension of the latent class model is presented for clustering categorical data by relaxing the classical "class conditional independence assumption" of variables. This model consists in grouping the variables into inter-independent and intra-dependent blocks, in order to consider the main intra-class correlations. The dependency between variables grouped inside the same block of a class is ta…
▽ More
An extension of the latent class model is presented for clustering categorical data by relaxing the classical "class conditional independence assumption" of variables. This model consists in grouping the variables into inter-independent and intra-dependent blocks, in order to consider the main intra-class correlations. The dependency between variables grouped inside the same block of a class is taken into account by mixing two extreme distributions, which are respectively the independence and the maximum dependency. When the variables are dependent given the class, this approach is expected to reduce the biases of the latent class model. Indeed, it produces a meaningful dependency model with only a few additional parameters. The parameters are estimated, by maximum likelihood, by means of an EM algorithm. Moreover, a Gibbs sampler is used for model selection in order to overcome the computational intractability of the combinatorial problems involved by the block structure search. Two applications on medical and biological data sets show the relevance of this new model. The results strengthen the view that this model is meaningful and that it reduces the biases induced by the conditional independence assumption of the latent class model.
△ Less
Submitted 10 July, 2014; v1 submitted 22 January, 2014;
originally announced January 2014.
