Showing 1–18 of 18 results for author: Mihaylova, L

Review of Recent Advances in Gaussian Process Regression Methods

Authors: Chenyi Lyu, Xingchi Liu, Lyudmila Mihaylova

Abstract: Gaussian process (GP) methods have been widely studied recently, especially for large-scale systems with big data and even more extreme cases when data is sparse. Key advantages of these methods consist in: 1) the ability to provide inherent ways to assess the impact of uncertainties (especially in the data, and environment) on the solutions, 2) have efficient factorisation based implementations a… ▽ More Gaussian process (GP) methods have been widely studied recently, especially for large-scale systems with big data and even more extreme cases when data is sparse. Key advantages of these methods consist in: 1) the ability to provide inherent ways to assess the impact of uncertainties (especially in the data, and environment) on the solutions, 2) have efficient factorisation based implementations and 3) can be implemented easily in distributed manners and hence provide scalable solutions. This paper reviews the recently developed key factorised GP methods such as the hierarchical off-diagonal low-rank approximation methods and GP with Kronecker structures. An example illustrates the performance of these methods with respect to accuracy and computational complexity. △ Less

Submitted 12 September, 2024; originally announced September 2024.

Gaussian Process Upper Confidence Bounds in Distributed Point Target Tracking over Wireless Sensor Networks

Authors: Xingchi Liu, Lyudmila Mihaylova, Jemin George, Tien Pham

Abstract: Uncertainty quantification plays a key role in the development of autonomous systems, decision-making, and tracking over wireless sensor networks (WSNs). However, there is a need of providing uncertainty confidence bounds, especially for distributed machine learning-based tracking, dealing with different volumes of data collected by sensors. This paper aims to fill in this gap and proposes a distr… ▽ More Uncertainty quantification plays a key role in the development of autonomous systems, decision-making, and tracking over wireless sensor networks (WSNs). However, there is a need of providing uncertainty confidence bounds, especially for distributed machine learning-based tracking, dealing with different volumes of data collected by sensors. This paper aims to fill in this gap and proposes a distributed Gaussian process (DGP) approach for point target tracking and derives upper confidence bounds (UCBs) of the state estimates. A unique contribution of this paper includes the derived theoretical guarantees on the proposed approach and its maximum accuracy for tracking with and without clutter measurements. Particularly, the developed approaches with uncertainty bounds are generic and can provide trustworthy solutions with an increased level of reliability. A novel hybrid Bayesian filtering method is proposed to enhance the DGP approach by adopting a Poisson measurement likelihood model. The proposed approaches are validated over a WSN case study, where sensors have limited sensing ranges. Numerical results demonstrate the tracking accuracy and robustness of the proposed approaches. The derived UCBs constitute a tool for trustworthiness evaluation of DGP approaches. The simulation results reveal that the proposed UCBs successfully encompass the true target states with 88% and 42% higher probability in X and Y coordinates, respectively, when compared to the confidence interval-based method. △ Less

Submitted 11 September, 2024; originally announced September 2024.

Variational Bayesian inference of hidden stochastic processes with unknown parameters

Authors: Komlan Atitey, Pavel Loskot, Lyudmila Mihaylova

Abstract: Estimating hidden processes from non-linear noisy observations is particularly difficult when the parameters of these processes are not known. This paper adopts a machine learning approach to devise variational Bayesian inference for such scenarios. In particular, a random process generated by the autoregressive moving average (ARMA) linear model is inferred from non-linearity noise observations.… ▽ More Estimating hidden processes from non-linear noisy observations is particularly difficult when the parameters of these processes are not known. This paper adopts a machine learning approach to devise variational Bayesian inference for such scenarios. In particular, a random process generated by the autoregressive moving average (ARMA) linear model is inferred from non-linearity noise observations. The posterior distribution of hidden states are approximated by a set of weighted particles generated by the sequential Monte carlo (SMC) algorithm involving sampling with importance sampling resampling (SISR). Numerical efficiency and estimation accuracy of the proposed inference method are evaluated by computer simulations. Furthermore, the proposed inference method is demonstrated on a practical problem of estimating the missing values in the gene expression time series assuming vector autoregressive (VAR) data model. △ Less

Submitted 2 November, 2019; originally announced November 2019.

Uncertainty propagation in neural networks for sparse coding

Authors: Danil Kuzin, Olga Isupova, Lyudmila Mihaylova

Abstract: A novel method to propagate uncertainty through the soft-thresholding nonlinearity is proposed in this paper. At every layer the current distribution of the target vector is represented as a spike and slab distribution, which represents the probabilities of each variable being zero, or Gaussian-distributed. Using the proposed method of uncertainty propagation, the gradients of the logarithms of no… ▽ More A novel method to propagate uncertainty through the soft-thresholding nonlinearity is proposed in this paper. At every layer the current distribution of the target vector is represented as a spike and slab distribution, which represents the probabilities of each variable being zero, or Gaussian-distributed. Using the proposed method of uncertainty propagation, the gradients of the logarithms of normalisation constants are derived, that can be used to update a weight distribution. A novel Bayesian neural network for sparse coding is designed utilising both the proposed method of uncertainty propagation and Bayesian inference algorithm. △ Less

Submitted 29 November, 2018; originally announced November 2018.

Comments: Presented at the third workshop on Bayesian Deep Learning (NeurIPS 2018)

Spatio-Temporal Structured Sparse Regression with Hierarchical Gaussian Process Priors

Authors: Danil Kuzin, Olga Isupova, Lyudmila Mihaylova

Abstract: This paper introduces a new sparse spatio-temporal structured Gaussian process regression framework for online and offline Bayesian inference. This is the first framework that gives a time-evolving representation of the interdependencies between the components of the sparse signal of interest. A hierarchical Gaussian process describes such structure and the interdependencies are represented via th… ▽ More This paper introduces a new sparse spatio-temporal structured Gaussian process regression framework for online and offline Bayesian inference. This is the first framework that gives a time-evolving representation of the interdependencies between the components of the sparse signal of interest. A hierarchical Gaussian process describes such structure and the interdependencies are represented via the covariance matrices of the prior distributions. The inference is based on the expectation propagation method and the theoretical derivation of the posterior distribution is provided in the paper. The inference framework is thoroughly evaluated over synthetic, real video and electroencephalography (EEG) data where the spatio-temporal evolving patterns need to be reconstructed with high accuracy. It is shown that it achieves 15% improvement of the F-measure compared with the alternating direction method of multipliers, spatio-temporal sparse Bayesian learning method and one-level Gaussian process model. Additionally, the required memory for the proposed algorithm is less than in the one-level Gaussian process model. This structured sparse regression framework is of broad applicability to source localisation and object detection problems with sparse signals. △ Less

Submitted 15 July, 2018; originally announced July 2018.

Ensemble Kalman Filtering for Online Gaussian Process Regression and Learning

Authors: Danil Kuzin, Le Yang, Olga Isupova, Lyudmila Mihaylova

Abstract: Gaussian process regression is a machine learning approach which has been shown its power for estimation of unknown functions. However, Gaussian processes suffer from high computational complexity, as in a basic form they scale cubically with the number of observations. Several approaches based on inducing points were proposed to handle this problem in a static context. These methods though face c… ▽ More Gaussian process regression is a machine learning approach which has been shown its power for estimation of unknown functions. However, Gaussian processes suffer from high computational complexity, as in a basic form they scale cubically with the number of observations. Several approaches based on inducing points were proposed to handle this problem in a static context. These methods though face challenges with real-time tasks and when the data is received sequentially over time. In this paper, a novel online algorithm for training sparse Gaussian process models is presented. It treats the mean and hyperparameters of the Gaussian process as the state and parameters of the ensemble Kalman filter, respectively. The online evaluation of the parameters and the state is performed on new upcoming samples of data. This procedure iteratively improves the accuracy of parameter estimates. The ensemble Kalman filter reduces the computational complexity required to obtain predictions with Gaussian processes preserving the accuracy level of these predictions. The performance of the proposed method is demonstrated on the synthetic dataset and real large dataset of UK house prices. △ Less

Submitted 9 July, 2018; originally announced July 2018.

Comments: FUSION 2018

Compressive Sensing Approaches for Autonomous Object Detection in Video Sequences

Authors: Danil Kuzin, Olga Isupova, Lyudmila Mihaylova

Abstract: Video analytics requires operating with large amounts of data. Compressive sensing allows to reduce the number of measurements required to represent the video using the prior knowledge of sparsity of the original signal, but it imposes certain conditions on the design matrix. The Bayesian compressive sensing approach relaxes the limitations of the conventional approach using the probabilistic reas… ▽ More Video analytics requires operating with large amounts of data. Compressive sensing allows to reduce the number of measurements required to represent the video using the prior knowledge of sparsity of the original signal, but it imposes certain conditions on the design matrix. The Bayesian compressive sensing approach relaxes the limitations of the conventional approach using the probabilistic reasoning and allows to include different prior knowledge about the signal structure. This paper presents two Bayesian compressive sensing methods for autonomous object detection in a video sequence from a static camera. Their performance is compared on the real datasets with the non-Bayesian greedy algorithm. It is shown that the Bayesian methods can provide the same accuracy as the greedy algorithm but much faster; or if the computational time is not critical they can provide more accurate results. △ Less

Submitted 27 April, 2017; originally announced May 2017.

Comments: SDF 2015

Structured Sparse Modelling with Hierarchical GP

Authors: Danil Kuzin, Olga Isupova, Lyudmila Mihaylova

Abstract: In this paper a new Bayesian model for sparse linear regression with a spatio-temporal structure is proposed. It incorporates the structural assumptions based on a hierarchical Gaussian process prior for spike and slab coefficients. We design an inference algorithm based on Expectation Propagation and evaluate the model over the real data. In this paper a new Bayesian model for sparse linear regression with a spatio-temporal structure is proposed. It incorporates the structural assumptions based on a hierarchical Gaussian process prior for spike and slab coefficients. We design an inference algorithm based on Expectation Propagation and evaluate the model over the real data. △ Less

Submitted 27 April, 2017; originally announced April 2017.

Comments: SPARS 2017

Learning Methods for Dynamic Topic Modeling in Automated Behaviour Analysis

Authors: Olga Isupova, Danil Kuzin, Lyudmila Mihaylova

Abstract: Semi-supervised and unsupervised systems provide operators with invaluable support and can tremendously reduce the operators load. In the light of the necessity to process large volumes of video data and provide autonomous decisions, this work proposes new learning algorithms for activity analysis in video. The activities and behaviours are described by a dynamic topic model. Two novel learning al… ▽ More Semi-supervised and unsupervised systems provide operators with invaluable support and can tremendously reduce the operators load. In the light of the necessity to process large volumes of video data and provide autonomous decisions, this work proposes new learning algorithms for activity analysis in video. The activities and behaviours are described by a dynamic topic model. Two novel learning algorithms based on the expectation maximisation approach and variational Bayes inference are proposed. Theoretical derivations of the posterior of model parameters are given. The designed learning algorithms are compared with the Gibbs sampling inference scheme introduced earlier in the literature. A detailed comparison of the learning algorithms is presented on real video data. We also propose an anomaly localisation procedure, elegantly embedded in the topic modeling framework. The proposed framework can be applied to a number of areas, including transportation systems, security and surveillance. △ Less

Submitted 18 September, 2017; v1 submitted 2 November, 2016; originally announced November 2016.

Comments: 15 pages

Dynamic Hierarchical Dirichlet Process for Abnormal Behaviour Detection in Video

Authors: Olga Isupova, Danil Kuzin, Lyudmila Mihaylova

Abstract: This paper proposes a novel dynamic Hierarchical Dirichlet Process topic model that considers the dependence between successive observations. Conventional posterior inference algorithms for this kind of models require processing of the whole data through several passes. It is computationally intractable for massive or sequential data. We design the batch and online inference algorithms, based on t… ▽ More This paper proposes a novel dynamic Hierarchical Dirichlet Process topic model that considers the dependence between successive observations. Conventional posterior inference algorithms for this kind of models require processing of the whole data through several passes. It is computationally intractable for massive or sequential data. We design the batch and online inference algorithms, based on the Gibbs sampling, for the proposed model. It allows to process sequential data, incrementally updating the model by a new observation. The model is applied to abnormal behaviour detection in video sequences. A new abnormality measure is proposed for decision making. The proposed method is compared with the method based on the non- dynamic Hierarchical Dirichlet Process, for which we also derive the online Gibbs sampler and the abnormality measure. The results with synthetic and real data show that the consideration of the dynamics in a topic model improves the classification performance for abnormal behaviour detection. △ Less

Submitted 27 June, 2016; originally announced June 2016.

Comments: 8 pages, International Conference on Information Fusion 2016

Anomaly detection in video with Bayesian nonparametrics

Authors: Olga Isupova, Danil Kuzin, Lyudmila Mihaylova

Abstract: A novel dynamic Bayesian nonparametric topic model for anomaly detection in video is proposed in this paper. Batch and online Gibbs samplers are developed for inference. The paper introduces a new abnormality measure for decision making. The proposed method is evaluated on both synthetic and real data. The comparison with a non-dynamic model shows the superiority of the proposed dynamic one in ter… ▽ More A novel dynamic Bayesian nonparametric topic model for anomaly detection in video is proposed in this paper. Batch and online Gibbs samplers are developed for inference. The paper introduces a new abnormality measure for decision making. The proposed method is evaluated on both synthetic and real data. The comparison with a non-dynamic model shows the superiority of the proposed dynamic one in terms of the classification performance for anomaly detection. △ Less

Submitted 27 June, 2016; originally announced June 2016.

Comments: 5 pages

Autonomous Crowds Tracking with Box Particle Filtering and Convolution Particle Filtering

Authors: Allan De Freitas, Lyudmila Mihaylova, Amadou Gning, Donka Angelova, Visakan Kadirkamanathan

Abstract: Autonomous systems such as Unmanned Aerial Vehicles (UAVs) need to be able to recognise and track crowds of people, e.g. for rescuing and surveillance purposes. Large groups generate multiple measurements with uncertain origin. Additionally, often the sensor noise characteristics are unknown but measurements are bounded within certain intervals. In this work we propose two solutions to the crowds… ▽ More Autonomous systems such as Unmanned Aerial Vehicles (UAVs) need to be able to recognise and track crowds of people, e.g. for rescuing and surveillance purposes. Large groups generate multiple measurements with uncertain origin. Additionally, often the sensor noise characteristics are unknown but measurements are bounded within certain intervals. In this work we propose two solutions to the crowds tracking problem - with a box particle filtering approach and with a convolution particle filtering approach. The developed filters can cope with the measurement origin uncertainty in an elegant way, i.e. resolve the data association problem. For the box particle filter (PF) we derive a theoretical expression of the generalised likelihood function in the presence of clutter. An adaptive convolution particle filter (CPF) is also developed and the performance of the two filters is compared with the standard sequential importance resampling (SIR) PF. The pros and cons of the two filters are illustrated over a realistic scenario (representing a crowd motion in a stadium) for a large crowd of pedestrians. Accurate estimation results are achieved. △ Less

Submitted 11 January, 2016; originally announced January 2016.

Sequential Markov Chain Monte Carlo for Bayesian Filtering with Massive Data

Authors: Allan De Freitas, François Septier, Lyudmila Mihaylova

Abstract: Advances in digital sensors, digital data storage and communications have resulted in systems being capable of accumulating large collections of data. In the light of dealing with the challenges that massive data present, this work proposes solutions to inference and filtering problems within the Bayesian framework. Two novel Bayesian inference algorithms are developed for non-linear and non-Gauss… ▽ More Advances in digital sensors, digital data storage and communications have resulted in systems being capable of accumulating large collections of data. In the light of dealing with the challenges that massive data present, this work proposes solutions to inference and filtering problems within the Bayesian framework. Two novel Bayesian inference algorithms are developed for non-linear and non-Gaussian state space models, able to deal with large volumes of data (or observations). These are sequential Markov chain Monte Carlo (MCMC) approaches relying on two key ideas: 1) subsample the massive data and utilise a smaller subset for filtering and inference, and 2) a divide and conquer type approach computing local filtering distributions each using a subset of the measurements. Simulation results highlight the accuracy and the large computational savings, that can reach 90% by the proposed algorithms when compared with standard techniques. △ Less

Submitted 8 December, 2015; originally announced December 2015.

Sequential Monte Carlo Methods for State and Parameter Estimation in Abruptly Changing Environments

Authors: Christopher Nemeth, Paul Fearnhead, Lyudmila Mihaylova

Abstract: This paper develops a novel sequential Monte Carlo (SMC) approach for joint state and parameter estimation that can deal efficiently with abruptly changing parameters which is a common case when tracking maneuvering targets. The approach combines Bayesian methods for dealing with changepoints with methods for estimating static parameters within the SMC framework. The result is an approach which ad… ▽ More This paper develops a novel sequential Monte Carlo (SMC) approach for joint state and parameter estimation that can deal efficiently with abruptly changing parameters which is a common case when tracking maneuvering targets. The approach combines Bayesian methods for dealing with changepoints with methods for estimating static parameters within the SMC framework. The result is an approach which adaptively estimates the model parameters in accordance with changes to the target's trajectory. The developed approach is compared against the Interacting Multiple Model (IMM) filter for tracking a maneuvering target over a complex maneuvering scenario with nonlinear observations. In the IMM filter a large combination of models is required to account for unknown parameters. In contrast, the proposed approach circumvents the combinatorial complexity of applying multiple models in the IMM filter through Bayesian parameter estimation techniques. The developed approach is validated over complex maneuvering scenarios where both the system parameters and measurement noise parameters are unknown. Accurate estimation results are presented. △ Less

Submitted 9 October, 2015; originally announced October 2015.

Comments: 26 pages, 5 figures

Journal ref: IEEE Transactions on Signal Processing (2014), Volume:62, Issue: 5

A Bayesian Compressed Sensing Kalman Filter for Direction of Arrival Estimation

Authors: Matthew Hawes, Lyudmila Mihaylova, Francois Septier, Simon Godsill

Abstract: In this paper, we look to address the problem of estimating the dynamic direction of arrival (DOA) of a narrowband signal impinging on a sensor array from the far field. The initial estimate is made using a Bayesian compressive sensing (BCS) framework and then tracked using a Bayesian compressed sensing Kalman filter (BCSKF). The BCS framework splits the angular region into N potential DOAs and en… ▽ More In this paper, we look to address the problem of estimating the dynamic direction of arrival (DOA) of a narrowband signal impinging on a sensor array from the far field. The initial estimate is made using a Bayesian compressive sensing (BCS) framework and then tracked using a Bayesian compressed sensing Kalman filter (BCSKF). The BCS framework splits the angular region into N potential DOAs and enforces a belief that only a few of the DOAs will have a non-zero valued signal present. A BCSKF can then be used to track the change in the DOA using the same framework. There can be an issue when the DOA approaches the endfire of the array. In this angular region current methods can struggle to accurately estimate and track changes in the DOAs. To tackle this problem, we propose changing the traditional sparse belief associated with BCS to a belief that the estimated signals will match the predicted signals given a known DOA change. This is done by modelling the difference between the expected sparse received signals and the estimated sparse received signals as a Gaussian distribution. Example test scenarios are provided and comparisons made with the traditional BCS based estimation method. They show that an improvement in estimation accuracy is possible without a significant increase in computational complexity. △ Less

Submitted 21 September, 2015; originally announced September 2015.

Comments: Fusion 2015 paper

How Can Subsampling Reduce Complexity in Sequential MCMC Methods and Deal with Big Data in Target Tracking?

Authors: Allan De Freitas, François Septier, Lyudmila Mihaylova, Simon Godsill

Abstract: Target tracking faces the challenge in coping with large volumes of data which requires efficient methods for real time applications. The complexity considered in this paper is when there is a large number of measurements which are required to be processed at each time step. Sequential Markov chain Monte Carlo (MCMC) has been shown to be a promising approach to target tracking in complex environme… ▽ More Target tracking faces the challenge in coping with large volumes of data which requires efficient methods for real time applications. The complexity considered in this paper is when there is a large number of measurements which are required to be processed at each time step. Sequential Markov chain Monte Carlo (MCMC) has been shown to be a promising approach to target tracking in complex environments, especially when dealing with clutter. However, a large number of measurements usually results in large processing requirements. This paper goes beyond the current state-of-the-art and presents a novel Sequential MCMC approach that can overcome this challenge through adaptively subsampling the set of measurements. Instead of using the whole large volume of available data, the proposed algorithm performs a trade off between the number of measurements to be used and the desired accuracy of the estimates to be obtained in the presence of clutter. We show results with large improvements in processing time, more than 40% with a negligible loss in tracking performance, compared with the solution without subsampling. △ Less

Submitted 30 July, 2015; originally announced July 2015.

Comments: International Conference on Information Fusion, 2015

Particle approximations of the score and observed information matrix for parameter estimation in state space models with linear computational cost

Authors: Christopher Nemeth, Paul Fearnhead, Lyudmila Mihaylova

Abstract: Poyiadjis et al. (2011) show how particle methods can be used to estimate both the score and the observed information matrix for state space models. These methods either suffer from a computational cost that is quadratic in the number of particles, or produce estimates whose variance increases quadratically with the amount of data. This paper introduces an alternative approach for estimating these… ▽ More Poyiadjis et al. (2011) show how particle methods can be used to estimate both the score and the observed information matrix for state space models. These methods either suffer from a computational cost that is quadratic in the number of particles, or produce estimates whose variance increases quadratically with the amount of data. This paper introduces an alternative approach for estimating these terms at a computational cost that is linear in the number of particles. The method is derived using a combination of kernel density estimation, to avoid the particle degeneracy that causes the quadratically increasing variance, and Rao-Blackwellisation. Crucially, we show the method is robust to the choice of bandwidth within the kernel density estimation, as it has good asymptotic properties regardless of this choice. Our estimates of the score and observed information matrix can be used within both online and batch procedures for estimating parameters for state space models. Empirical results show improved parameter estimates compared to existing methods at a significantly reduced computational cost. Supplementary materials including code are available. △ Less

Submitted 4 September, 2015; v1 submitted 4 June, 2013; originally announced June 2013.

Comments: Accepted to Journal of Computational and Graphical Statistics

Connectivity of Random 1-Dimensional Networks

Authors: V. Kurlin, L. Mihaylova

Abstract: An important problem in wireless sensor networks is to find the minimal number of randomly deployed sensors making a network connected with a given probability. In practice sensors are often deployed one by one along a trajectory of a vehicle, so it is natural to assume that arbitrary probability density functions of distances between successive sensors in a segment are given. The paper computes… ▽ More An important problem in wireless sensor networks is to find the minimal number of randomly deployed sensors making a network connected with a given probability. In practice sensors are often deployed one by one along a trajectory of a vehicle, so it is natural to assume that arbitrary probability density functions of distances between successive sensors in a segment are given. The paper computes the probability of connectivity and coverage of 1-dimensional networks and gives estimates for a minimal number of sensors for important distributions. △ Less

Submitted 9 October, 2009; v1 submitted 4 October, 2007; originally announced October 2007.

Comments: 12 pages, 10 figures

ACM Class: C.2.1