-
Efficient Interpretable Nonlinear Modeling for Multiple Time Series
Authors:
Kevin Roy,
Luis Miguel Lopez-Ramos,
Baltasar Beferull-Lozano
Abstract:
Predictive linear and nonlinear models based on kernel machines or deep neural networks have been used to discover dependencies among time series. This paper proposes an efficient nonlinear modeling approach for multiple time series, with a complexity comparable to linear vector autoregressive (VAR) models while still incorporating nonlinear interactions among different time-series variables. The…
▽ More
Predictive linear and nonlinear models based on kernel machines or deep neural networks have been used to discover dependencies among time series. This paper proposes an efficient nonlinear modeling approach for multiple time series, with a complexity comparable to linear vector autoregressive (VAR) models while still incorporating nonlinear interactions among different time-series variables. The modeling assumption is that the set of time series is generated in two steps: first, a linear VAR process in a latent space, and second, a set of invertible and Lipschitz continuous nonlinear mappings that are applied per sensor, that is, a component-wise mapping from each latent variable to a variable in the measurement space. The VAR coefficient identification provides a topology representation of the dependencies among the aforementioned variables. The proposed approach models each component-wise nonlinearity using an invertible neural network and imposes sparsity on the VAR coefficients to reflect the parsimonious dependencies usually found in real applications. To efficiently solve the formulated optimization problems, a custom algorithm is devised combining proximal gradient descent, stochastic primal-dual updates, and projection to enforce the corresponding constraints. Experimental results on both synthetic and real data sets show that the proposed algorithm improves the identification of the support of the VAR coefficients in a parsimonious manner while also improving the time-series prediction, as compared to the current state-of-the-art methods.
△ Less
Submitted 29 September, 2023;
originally announced September 2023.
-
Consistent Signal Reconstruction from Streaming Multivariate Time Series
Authors:
Emilio Ruiz-Moreno,
Luis Miguel López-Ramos,
Baltasar Beferull-Lozano
Abstract:
Digitalizing real-world analog signals typically involves sampling in time and discretizing in amplitude. Subsequent signal reconstructions inevitably incur an error that depends on the amplitude resolution and the temporal density of the acquired samples. From an implementation viewpoint, consistent signal reconstruction methods have proven a profitable error-rate decay as the sampling rate incre…
▽ More
Digitalizing real-world analog signals typically involves sampling in time and discretizing in amplitude. Subsequent signal reconstructions inevitably incur an error that depends on the amplitude resolution and the temporal density of the acquired samples. From an implementation viewpoint, consistent signal reconstruction methods have proven a profitable error-rate decay as the sampling rate increases. Despite that, these results are obtained under offline settings. Therefore, a research gap exists regarding methods for consistent signal reconstruction from data streams. Solving this problem is of great importance because such methods could run at a lower computational cost than the existing offline ones or be used under real-time requirements without losing the benefits of ensuring consistency. In this paper, we formalize for the first time the concept of consistent signal reconstruction from streaming time-series data. Then, we present a signal reconstruction method able to enforce consistency and also exploit the spatiotemporal dependencies of streaming multivariate time-series data to further reduce the signal reconstruction error. Our experiments show that our proposed method achieves a favorable error-rate decay with the sampling rate compared to a similar but non-consistent reconstruction.
△ Less
Submitted 31 January, 2024; v1 submitted 23 August, 2023;
originally announced August 2023.
-
An Online Multiple Kernel Parallelizable Learning Scheme
Authors:
Emilio Ruiz-Moreno,
Baltasar Beferull-Lozano
Abstract:
The performance of reproducing kernel Hilbert space-based methods is known to be sensitive to the choice of the reproducing kernel. Choosing an adequate reproducing kernel can be challenging and computationally demanding, especially in data-rich tasks without prior information about the solution domain. In this paper, we propose a learning scheme that scalably combines several single kernel-based…
▽ More
The performance of reproducing kernel Hilbert space-based methods is known to be sensitive to the choice of the reproducing kernel. Choosing an adequate reproducing kernel can be challenging and computationally demanding, especially in data-rich tasks without prior information about the solution domain. In this paper, we propose a learning scheme that scalably combines several single kernel-based online methods to reduce the kernel-selection bias. The proposed learning scheme applies to any task formulated as a regularized empirical risk minimization convex problem. More specifically, our learning scheme is based on a multi-kernel learning formulation that can be applied to widen any single-kernel solution space, thus increasing the possibility of finding higher-performance solutions. In addition, it is parallelizable, allowing for the distribution of the computational load across different computing units. We show experimentally that the proposed learning scheme outperforms the combined single-kernel online methods separately in terms of the cumulative regularized least squares cost metric.
△ Less
Submitted 6 November, 2023; v1 submitted 19 August, 2023;
originally announced August 2023.
-
A Trainable Approach to Zero-delay Smoothing Spline Interpolation
Authors:
Emilio Ruiz-Moreno,
Luis Miguel López-Ramos,
Baltasar Beferull-Lozano
Abstract:
The task of reconstructing smooth signals from streamed data in the form of signal samples arises in various applications. This work addresses such a task subject to a zero-delay response; that is, the smooth signal must be reconstructed sequentially as soon as a data sample is available and without having access to subsequent data. State-of-the-art approaches solve this problem by interpolating c…
▽ More
The task of reconstructing smooth signals from streamed data in the form of signal samples arises in various applications. This work addresses such a task subject to a zero-delay response; that is, the smooth signal must be reconstructed sequentially as soon as a data sample is available and without having access to subsequent data. State-of-the-art approaches solve this problem by interpolating consecutive data samples using splines. Here, each interpolation step yields a piece that ensures a smooth signal reconstruction while minimizing a cost metric, typically a weighted sum between the squared residual and a derivative-based measure of smoothness. As a result, a zero-delay interpolation is achieved in exchange for an almost certainly higher cumulative cost as compared to interpolating all data samples together. This paper presents a novel approach to further reduce this cumulative cost on average. First, we formulate a zero-delay smoothing spline interpolation problem from a sequential decision-making perspective, allowing us to model the future impact of each interpolated piece on the average cumulative cost. Then, an interpolation method is proposed to exploit the temporal dependencies between the streamed data samples. Our method is assisted by a recurrent neural network and accordingly trained to reduce the accumulated cost on average over a set of example data samples collected from the same signal source generating the signal to be reconstructed. Finally, we present extensive experimental results for synthetic and real data showing how our approach outperforms the abovementioned state-of-the-art.
△ Less
Submitted 20 August, 2023; v1 submitted 7 March, 2022;
originally announced March 2022.
-
Random Feature Approximation for Online Nonlinear Graph Topology Identification
Authors:
Rohan Money,
Joshin Krishnan,
Baltasar Beferull-Lozano
Abstract:
Online topology estimation of graph-connected time series is challenging, especially since the causal dependencies in many real-world networks are nonlinear. In this paper, we propose a kernel-based algorithm for graph topology estimation. The algorithm uses a Fourier-based Random feature approximation to tackle the curse of dimensionality associated with the kernel representations. Exploiting the…
▽ More
Online topology estimation of graph-connected time series is challenging, especially since the causal dependencies in many real-world networks are nonlinear. In this paper, we propose a kernel-based algorithm for graph topology estimation. The algorithm uses a Fourier-based Random feature approximation to tackle the curse of dimensionality associated with the kernel representations. Exploiting the fact that the real-world networks often exhibit sparse topologies, we propose a group lasso based optimization framework, which is solve using an iterative composite objective mirror descent method, yielding an online algorithm with fixed computational complexity per iteration. The experiments conducted on real and synthetic data show that the proposed method outperforms its competitors.
△ Less
Submitted 19 October, 2021;
originally announced October 2021.
-
Explainable nonlinear modelling of multiple time series with invertible neural networks
Authors:
Luis Miguel Lopez-Ramos,
Kevin Roy,
Baltasar Beferull-Lozano
Abstract:
A method for nonlinear topology identification is proposed, based on the assumption that a collection of time series are generated in two steps: i) a vector autoregressive process in a latent space, and ii) a nonlinear, component-wise, monotonically increasing observation mapping. The latter mappings are assumed invertible, and are modelled as shallow neural networks, so that their inverse can be…
▽ More
A method for nonlinear topology identification is proposed, based on the assumption that a collection of time series are generated in two steps: i) a vector autoregressive process in a latent space, and ii) a nonlinear, component-wise, monotonically increasing observation mapping. The latter mappings are assumed invertible, and are modelled as shallow neural networks, so that their inverse can be numerically evaluated, and their parameters can be learned using a technique inspired in deep learning. Due to the function inversion, the back-propagation step is not straightforward, and this paper explains the steps needed to calculate the gradients applying implicit differentiation. Whereas the model explainability is the same as that for linear VAR processes, preliminary numerical tests show that the prediction error becomes smaller.
△ Less
Submitted 1 July, 2021;
originally announced July 2021.
-
Channel Gain Cartography via Mixture of Experts
Authors:
Luis M. Lopez-Ramos,
Yves Teganya,
Baltasar Beferull-Lozano,
Seung-Jun Kim
Abstract:
In order to estimate the channel gain (CG) between the locations of an arbitrary transceiver pair across a geographic area of interest, CG maps can be constructed from spatially distributed sensor measurements. Most approaches to build such spectrum maps are location-based, meaning that the input variable to the estimating function is a pair of spatial locations. The performance of such maps depen…
▽ More
In order to estimate the channel gain (CG) between the locations of an arbitrary transceiver pair across a geographic area of interest, CG maps can be constructed from spatially distributed sensor measurements. Most approaches to build such spectrum maps are location-based, meaning that the input variable to the estimating function is a pair of spatial locations. The performance of such maps depends critically on the ability of the sensors to determine their positions, which may be drastically impaired if the positioning pilot signals are affected by multi-path channels. An alternative location-free approach was recently proposed for spectrum power maps, where the input variable to the maps consists of features extracted from the positioning signals, instead of location estimates. The location-based and the location-free approaches have complementary merits. In this work, apart from adapting the location-free features for the CG maps, a method that can combine both approaches is proposed in a mixture-of-experts framework.
△ Less
Submitted 8 December, 2020;
originally announced December 2020.
-
Accurate Graph Filtering in Wireless Sensor Networks
Authors:
Leila Ben Saad,
Baltasar Beferull-Lozano
Abstract:
Wireless sensor networks (WSNs) are considered as a major technology enabling the Internet of Things (IoT) paradigm. The recent emerging Graph Signal Processing field can also contribute to enabling the IoT by providing key tools, such as graph filters, for processing the data associated with the sensor devices. Graph filters can be performed over WSNs in a distributed manner by means of a certain…
▽ More
Wireless sensor networks (WSNs) are considered as a major technology enabling the Internet of Things (IoT) paradigm. The recent emerging Graph Signal Processing field can also contribute to enabling the IoT by providing key tools, such as graph filters, for processing the data associated with the sensor devices. Graph filters can be performed over WSNs in a distributed manner by means of a certain number of communication exchanges among the nodes. But, WSNs are often affected by interferences and noise, which leads to view these networks as directed, random and time-varying graph topologies. Most of existing works neglect this problem by considering an unrealistic assumption that claims the same probability of link activation in both directions when sending a packet between two neighboring nodes. This work focuses on the problem of operating graph filtering in random asymmetric WSNs. We show first that graph filtering with finite impulse response graph filters (node-invariant and node-variant) requires having equal connectivity probabilities for all the links in order to have an unbiased filtering, which can not be achieved in practice in random WSNs. After this, we characterize the graph filtering error and present an efficient strategy to conduct graph filtering tasks over random WSNs with node-variant graph filters by maximizing accuracy, that is, ensuring a small bias-variance trade-off. In order to enforce the desired accuracy, we optimize the filter coefficients and design a cross-layer distributed scheduling algorithm at the MAC layer. Extensive numerical experiments are presented to show the efficiency of the proposed solution as well as the cross-layer distributed scheduling algorithm for the denoising application.
△ Less
Submitted 15 July, 2020; v1 submitted 24 April, 2020;
originally announced April 2020.
-
Quantization Analysis and Robust Design for Distributed Graph Filters
Authors:
Leila Ben Saad,
Baltasar Beferull-Lozano,
Elvin Isufi
Abstract:
Distributed graph filters have found applications in wireless sensor networks (WSNs) to solve distributed tasks such as consensus, signal denoising, and reconstruction. However, when employed over WSN, the graph filters should deal with the network limited energy, processing, and communication capabilities. Quantization plays a fundamental role to improve the latter but its effects on distributed…
▽ More
Distributed graph filters have found applications in wireless sensor networks (WSNs) to solve distributed tasks such as consensus, signal denoising, and reconstruction. However, when employed over WSN, the graph filters should deal with the network limited energy, processing, and communication capabilities. Quantization plays a fundamental role to improve the latter but its effects on distributed graph filtering are little understood. WSNs are also prone to random link losses due to noise and interference. The filter output is affected by both the quantization error and the topological randomness error, which, if it is not properly accounted in the filter design phase, may lead to an accumulated error through the filtering iterations and significantly degrade the performance. In this paper, we analyze how quantization affects distributed graph filtering over both time-invariant and time-varying graphs. We bring insights on the quantization effects for the two most common graph filters: the finite impulse response (FIR) and autoregressive moving average (ARMA) graph filter. We devise theoretical performance guarantees on the filter performance when the quantization stepsize is fixed or changes dynamically over the filtering iterations. For FIR filters, we show that a dynamic quantization stepsize leads to more control on the quantization noise than the fixed-stepsize quantization. For ARMA graph filters, we show that decreasing the quantization stepsize over the iterations reduces the quantization noise to zero at the steady-state. In addition, we propose robust filter design strategies that minimize the quantization noise for both time-invariant and time-varying networks. Numerical experiments on synthetic and two real data sets corroborate our findings and show the different trade-offs between quantization bits, filter order, and robustness to topological randomness.
△ Less
Submitted 14 April, 2020;
originally announced April 2020.
-
Online Hyperparameter Search Interleaved with Proximal Parameter Updates
Authors:
Luis Miguel Lopez-Ramos,
Baltasar Beferull-Lozano
Abstract:
There is a clear need for efficient algorithms to tune hyperparameters for statistical learning schemes, since the commonly applied search methods (such as grid search with N-fold cross-validation) are inefficient and/or approximate. Previously existing algorithms that efficiently search for hyperparameters relying on the smoothness of the cost function cannot be applied in problems such as Lasso…
▽ More
There is a clear need for efficient algorithms to tune hyperparameters for statistical learning schemes, since the commonly applied search methods (such as grid search with N-fold cross-validation) are inefficient and/or approximate. Previously existing algorithms that efficiently search for hyperparameters relying on the smoothness of the cost function cannot be applied in problems such as Lasso regression.
In this contribution, we develop a hyperparameter optimization method that relies on the structure of proximal gradient methods and does not require a smooth cost function. Such a method is applied to Leave-one-out (LOO)-validated Lasso and Group Lasso to yield efficient, data-driven, hyperparameter optimization algorithms.
Numerical experiments corroborate the convergence of the proposed method to a local optimum of the LOO validation error curve, and the efficiency of its approximations.
△ Less
Submitted 6 April, 2020;
originally announced April 2020.
-
Design of SCMA Codebooks Using Differential Evolution
Authors:
Kuntal Deka,
Minerva Priyadarsini,
Sanjeev Sharma,
Baltasar Beferull-Lozano
Abstract:
Non-orthogonal multiple access (NOMA) is a promising technology which meets the demands of massive connectivity in future wireless networks. Sparse code multiple access (SCMA) is a popular code-domain NOMA technique. The effectiveness of SCMA comes from: (1) the multi-dimensional sparse codebooks offering high shaping gain and (2) sophisticated multi-user detection based on message passing algorit…
▽ More
Non-orthogonal multiple access (NOMA) is a promising technology which meets the demands of massive connectivity in future wireless networks. Sparse code multiple access (SCMA) is a popular code-domain NOMA technique. The effectiveness of SCMA comes from: (1) the multi-dimensional sparse codebooks offering high shaping gain and (2) sophisticated multi-user detection based on message passing algorithm (MPA). The codebooks of the users play the main role in determining the performance of SCMA system. This paper presents a framework to design the codebooks by taking into account the entire system including the SCMA encoder and the MPA-based detector. The symbol-error rate (SER) is considered as the design criterion which needs to be minimized. Differential evolution (DE) is used to carry out the minimization of the SER over the codebooks. The simulation results are presented for various channel models.
△ Less
Submitted 15 February, 2021; v1 submitted 7 March, 2020;
originally announced March 2020.
-
Robust Underlay Device-to-Device Communications on Multiple Channels
Authors:
Mohamed Elnourani,
Siddharth Deshmukh,
Baltasar Beferull-Lozano,
Daniel Romero
Abstract:
Most recent works in device-to-device (D2D) underlay communications focus on the optimization of either power or channel allocation to improve the spectral efficiency, and typically consider uplink and downlink separately. Further, several of them also assume perfect knowledge of channel-stateinformation (CSI). In this paper, we formulate a joint uplink and downlink resource allocation scheme, whi…
▽ More
Most recent works in device-to-device (D2D) underlay communications focus on the optimization of either power or channel allocation to improve the spectral efficiency, and typically consider uplink and downlink separately. Further, several of them also assume perfect knowledge of channel-stateinformation (CSI). In this paper, we formulate a joint uplink and downlink resource allocation scheme, which assigns both power and channel resources to D2D pairs and cellular users in an underlay network scenario. The objective is to maximize the overall network rate while maintaining fairness among the D2D pairs. In addition, we also consider imperfect CSI, where we guarantee a certain outage probability to maintain the desired quality-of-service (QoS). The resulting problem is a mixed integer non-convex optimization problem and we propose both centralized and decentralized algorithms to solve it, using convex relaxation, fractional programming, and alternating optimization. In the decentralized setting, the computational load is distributed among the D2D pairs and the base station, keeping also a low communication overhead. Moreover, we also provide a theoretical convergence analysis, including also the rate of convergence to stationary points. The proposed algorithms have been experimentally tested in a simulation environment, showing their favorable performance, as compared with the state-of-the-art alternatives.
△ Less
Submitted 26 February, 2020;
originally announced February 2020.
-
Quantizing Using Lattice Intersections
Authors:
N. J. A. Sloane,
B. Beferull-Lozano
Abstract:
The usual quantizer based on an n-dimensional lattice L maps a point x in R^n to a closest lattice point. Suppose L is the intersection of lattices L_1, ..., L_r. Then one may instead combine the information obtained by simultaneously quantizing x with respect to each of the L_i. This corresponds to decomposing R^n into a honeycomb of cells which are the intersections of the Voronoi cells for th…
▽ More
The usual quantizer based on an n-dimensional lattice L maps a point x in R^n to a closest lattice point. Suppose L is the intersection of lattices L_1, ..., L_r. Then one may instead combine the information obtained by simultaneously quantizing x with respect to each of the L_i. This corresponds to decomposing R^n into a honeycomb of cells which are the intersections of the Voronoi cells for the L_i, and identifying the cell to which x belongs. This paper shows how to write several standard lattices (the face-centered and body-centered cubic lattices, the root lattices D_4, E_6*, E_8, the Coxeter-Todd, Barnes-Wall and Leech lattices, etc.) in a canonical way as intersections of a small number of simpler, decomposable, lattices. The cells of the honeycombs are given explicitly and the mean squared quantizing error calculated in the cases when the intersection lattice is the face-centered or body-centered cubic lattice or the lattice D_4.
△ Less
Submitted 17 July, 2002;
originally announced July 2002.
