Showing 1–11 of 11 results for author: Babadi, B

Non-Asymptotic Guarantees for Reliable Identification of Granger Causality via the LASSO

Authors: Proloy Das, Behtash Babadi

Abstract: Granger causality is among the widely used data-driven approaches for causal analysis of time series data with applications in various areas including economics, molecular biology, and neuroscience. Two of the main challenges of this methodology are: 1) over-fitting as a result of limited data duration, and 2) correlated process noise as a confounding factor, both leading to errors in identifying… ▽ More Granger causality is among the widely used data-driven approaches for causal analysis of time series data with applications in various areas including economics, molecular biology, and neuroscience. Two of the main challenges of this methodology are: 1) over-fitting as a result of limited data duration, and 2) correlated process noise as a confounding factor, both leading to errors in identifying the causal influences. Sparse estimation via the LASSO has successfully addressed these challenges for parameter estimation. However, the classical statistical tests for Granger causality resort to asymptotic analysis of ordinary least squares, which require long data duration to be useful and are not immune to confounding effects. In this work, we address this disconnect by introducing a LASSO-based statistic and studying its non-asymptotic properties under the assumption that the true models admit sparse autoregressive representations. We establish fundamental limits for reliable identification of Granger causal influences using the proposed LASSO-based statistic. We further characterize the false positive error probability and test power of a simple thresholding rule for identifying Granger causal effects and provide two methods to set the threshold in a data-driven fashion. We present simulation studies and application to real data to compare the performance of our proposed method to ordinary least squares and existing LASSO-based methods in detecting Granger causal influences, which corroborate our theoretical results. △ Less

Submitted 14 July, 2023; v1 submitted 3 March, 2021; originally announced March 2021.

Multitaper Analysis of Evolutionary Spectra from Multivariate Spiking Observations

Authors: Anuththara Rupasinghe, Behtash Babadi

Abstract: Extracting the spectral representations of the neural processes that underlie spiking activity is key to understanding how the brain rhythms mediate cognitive functions. While spectral estimation of continuous time-series is well studied, inferring the spectral representation of latent non-stationary processes based on spiking observations is a challenging problem. In this paper, we address this i… ▽ More Extracting the spectral representations of the neural processes that underlie spiking activity is key to understanding how the brain rhythms mediate cognitive functions. While spectral estimation of continuous time-series is well studied, inferring the spectral representation of latent non-stationary processes based on spiking observations is a challenging problem. In this paper, we address this issue by developing a multitaper spectral estimation methodology that can be directly applied to multivariate spiking observations in order to extract the evolutionary spectral density of the latent non-stationary processes that drive spiking activity, based on point process theory. We establish theoretical bounds on the bias-variance trade-off of the proposed estimator. Finally, we compare the performance of our proposed technique with existing methods using simulation studies and application to real data, which reveal significant gains in terms of the bias-variance trade-off. △ Less

Submitted 21 June, 2019; originally announced June 2019.

Multitaper Spectral Analysis of Neuronal Spiking Activity Driven by Latent Stationary Processes

Authors: Proloy Das, Behtash Babadi

Abstract: Investigating the spectral properties of the neural covariates that underlie spiking activity is an important problem in systems neuroscience, as it allows to study the role of brain rhythms in cognitive functions. While the spectral estimation of continuous time-series is a well-established domain, computing the spectral representation of these neural covariates from spiking data sets forth vario… ▽ More Investigating the spectral properties of the neural covariates that underlie spiking activity is an important problem in systems neuroscience, as it allows to study the role of brain rhythms in cognitive functions. While the spectral estimation of continuous time-series is a well-established domain, computing the spectral representation of these neural covariates from spiking data sets forth various challenges due to the intrinsic non-linearities involved. In this paper, we address this problem by proposing a variant of the multitaper method specifically tailored for point process data. To this end, we construct auxiliary spiking statistics from which the eigen-spectra of the underlying latent process can be directly inferred using maximum likelihood estimation, and thereby the multitaper estimate can be efficiently computed. Comparison of our proposed technique to existing methods using simulated data reveals significant gains in terms of the bias-variance trade-off. △ Less

Submitted 20 June, 2019; originally announced June 2019.

Dynamic Bayesian Multitaper Spectral Analysis

Authors: Proloy Das, Behtash Babadi

Abstract: Spectral analysis using overlapping sliding windows is among the most widely used techniques in analyzing non-stationary time series. Although sliding window analysis is convenient to implement, the resulting estimates are sensitive to the window length and overlap size. In addition, it undermines the dynamics of the time series as the estimate associated to each window uses only the data within.… ▽ More Spectral analysis using overlapping sliding windows is among the most widely used techniques in analyzing non-stationary time series. Although sliding window analysis is convenient to implement, the resulting estimates are sensitive to the window length and overlap size. In addition, it undermines the dynamics of the time series as the estimate associated to each window uses only the data within. Finally, the overlap between consecutive windows hinders a precise statistical assessment. In this paper, we address these shortcomings by explicitly modeling the spectral dynamics through integrating the multitaper method with state-space models in a Bayesian estimation framework. The underlying states pertaining to the eigen-spectral quantities arising in multitaper analysis are estimated using instances of the Expectation-Maximization algorithm, and are used to construct spectrograms and their respective confidence intervals. We propose two spectral estimators that are robust to noise and are able to capture spectral dynamics at high spectrotemporal resolution. We provide theoretical analysis of the bias-variance trade-off, which establishes performance gains over the standard overlapping multitaper method. We apply our algorithms to synthetic data as well as real data from human EEG and electric network frequency recordings, the results of which validate our theoretical analysis. △ Less

Submitted 15 December, 2017; v1 submitted 5 June, 2017; originally announced June 2017.

Efficient Estimation of Compressible State-Space Models with Application to Calcium Signal Deconvolution

Authors: Abbas Kazemipour, Ji Liu, Patrick Kanold, Min Wu, Behtash Babadi

Abstract: In this paper, we consider linear state-space models with compressible innovations and convergent transition matrices in order to model spatiotemporally sparse transient events. We perform parameter and state estimation using a dynamic compressed sensing framework and develop an efficient solution consisting of two nested Expectation-Maximization (EM) algorithms. Under suitable sparsity assumption… ▽ More In this paper, we consider linear state-space models with compressible innovations and convergent transition matrices in order to model spatiotemporally sparse transient events. We perform parameter and state estimation using a dynamic compressed sensing framework and develop an efficient solution consisting of two nested Expectation-Maximization (EM) algorithms. Under suitable sparsity assumptions on the innovations, we prove recovery guarantees and derive confidence bounds for the state estimates. We provide simulation studies as well as application to spike deconvolution from calcium imaging data which verify our theoretical results and show significant improvement over existing algorithms. △ Less

Submitted 20 October, 2016; originally announced October 2016.

Comments: 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Dec. 7-9, 2016, Washington D.C

Sampling Requirements for Stable Autoregressive Estimation

Authors: Abbas Kazemipour, Sina Miran, Piya Pal, Behtash Babadi, Min Wu

Abstract: We consider the problem of estimating the parameters of a linear univariate autoregressive model with sub-Gaussian innovations from a limited sequence of consecutive observations. Assuming that the parameters are compressible, we analyze the performance of the $\ell_1$-regularized least squares as well as a greedy estimator of the parameters and characterize the sampling trade-offs required for st… ▽ More We consider the problem of estimating the parameters of a linear univariate autoregressive model with sub-Gaussian innovations from a limited sequence of consecutive observations. Assuming that the parameters are compressible, we analyze the performance of the $\ell_1$-regularized least squares as well as a greedy estimator of the parameters and characterize the sampling trade-offs required for stable recovery in the non-asymptotic regime. In particular, we show that for a fixed sparsity level, stable recovery of AR parameters is possible when the number of samples scale sub-linearly with the AR order. Our results improve over existing sampling complexity requirements in AR estimation using the LASSO, when the sparsity level scales faster than the square root of the model order. We further derive sufficient conditions on the sparsity level that guarantee the minimax optimality of the $\ell_1$-regularized least squares estimate. Applying these techniques to simulated data as well as real-world datasets from crude oil prices and traffic speed data confirm our predicted theoretical performance gains in terms of estimation accuracy and model selection. △ Less

Submitted 17 January, 2017; v1 submitted 4 May, 2016; originally announced May 2016.

Comment on "Asymptotic Achievability of the Cramér-Rao Bound for Noisy Compressive Sampling"

Authors: Behtash Babadi, Nicholas Kalouptsidis, Vahid Tarokh

Abstract: In [1], we proved the asymptotic achievability of the Cramér-Rao bound in the compressive sensing setting in the linear sparsity regime. In the proof, we used an erroneous closed-form expression of $ασ^2$ for the genie-aided Cramér-Rao bound $σ^2 \textrm{Tr} (\mathbf{A}^*_\mathcal{I} \mathbf{A}_\mathcal{I})^{-1}$ from Lemma 3.5, which appears in Eqs. (20) and (29). The proof, however, holds if one… ▽ More In [1], we proved the asymptotic achievability of the Cramér-Rao bound in the compressive sensing setting in the linear sparsity regime. In the proof, we used an erroneous closed-form expression of $ασ^2$ for the genie-aided Cramér-Rao bound $σ^2 \textrm{Tr} (\mathbf{A}^*_\mathcal{I} \mathbf{A}_\mathcal{I})^{-1}$ from Lemma 3.5, which appears in Eqs. (20) and (29). The proof, however, holds if one avoids replacing $σ^2 \textrm{Tr} (\mathbf{A}^*_\mathcal{I} \mathbf{A}_\mathcal{I})^{-1}$ by the expression of Lemma 3.5, and hence the claim of the Main Theorem stands true. In Chapter 2 of the Ph. D. dissertation by Behtash Babadi [2], this error was fixed and a more detailed proof in the non-asymptotic regime was presented. A draft of Chapter 2 of [2] is included in this note, verbatim. We would like to refer the interested reader to the full dissertation, which is electronically archived in the ProQuest database [2], and a draft of which can be accessed through the author's homepage under: http://ece.umd.edu/~behtash/babadi_thesis_2011.pdf. △ Less

Submitted 14 September, 2015; originally announced September 2015.

Recursive Sparse Point Process Regression with Application to Spectrotemporal Receptive Field Plasticity Analysis

Authors: Alireza Sheikhattar, Jonathan B. Fritz, Shihab A. Shamma, Behtash Babadi

Abstract: We consider the problem of estimating the sparse time-varying parameter vectors of a point process model in an online fashion, where the observations and inputs respectively consist of binary and continuous time series. We construct a novel objective function by incorporating a forgetting factor mechanism into the point process log-likelihood to enforce adaptivity and employ $\ell_1$-regularizatio… ▽ More We consider the problem of estimating the sparse time-varying parameter vectors of a point process model in an online fashion, where the observations and inputs respectively consist of binary and continuous time series. We construct a novel objective function by incorporating a forgetting factor mechanism into the point process log-likelihood to enforce adaptivity and employ $\ell_1$-regularization to capture the sparsity. We provide a rigorous analysis of the maximizers of the objective function, which extends the guarantees of compressed sensing to our setting. We construct two recursive filters for online estimation of the parameter vectors based on proximal optimization techniques, as well as a novel filter for recursive computation of statistical confidence regions. Simulation studies reveal that our algorithms outperform several existing point process filters in terms of trackability, goodness-of-fit and mean square error. We finally apply our filtering algorithms to experimentally recorded spiking data from the ferret primary auditory cortex during attentive behavior in a click rate discrimination task. Our analysis provides new insights into the time-course of the spectrotemporal receptive field plasticity of the auditory neurons. △ Less

Submitted 16 July, 2015; originally announced July 2015.

Robust Estimation of Self-Exciting Generalized Linear Models with Application to Neuronal Modeling

Authors: Abbas Kazemipour, Min Wu, Behtash Babadi

Abstract: We consider the problem of estimating self-exciting generalized linear models from limited binary observations, where the history of the process serves as the covariate. We analyze the performance of two classes of estimators, namely the $\ell_1$-regularized maximum likelihood and greedy estimators, for a canonical self-exciting process and characterize the sampling tradeoffs required for stable r… ▽ More We consider the problem of estimating self-exciting generalized linear models from limited binary observations, where the history of the process serves as the covariate. We analyze the performance of two classes of estimators, namely the $\ell_1$-regularized maximum likelihood and greedy estimators, for a canonical self-exciting process and characterize the sampling tradeoffs required for stable recovery in the non-asymptotic regime. Our results extend those of compressed sensing for linear and generalized linear models with i.i.d. covariates to those with highly inter-dependent covariates. We further provide simulation studies as well as application to real spiking data from the mouse's lateral geniculate nucleus and the ferret's retinal ganglion cells which agree with our theoretical predictions. △ Less

Submitted 22 March, 2017; v1 submitted 14 July, 2015; originally announced July 2015.

SPARLS: A Low Complexity Recursive $\mathcal{L}_1$-Regularized Least Squares Algorithm

Authors: Behtash Babadi, Nicholas Kalouptsidis, Vahid Tarokh

Abstract: We develop a Recursive $\mathcal{L}_1$-Regularized Least Squares (SPARLS) algorithm for the estimation of a sparse tap-weight vector in the adaptive filtering setting. The SPARLS algorithm exploits noisy observations of the tap-weight vector output stream and produces its estimate using an Expectation-Maximization type algorithm. Simulation studies in the context of channel estimation, employing… ▽ More We develop a Recursive $\mathcal{L}_1$-Regularized Least Squares (SPARLS) algorithm for the estimation of a sparse tap-weight vector in the adaptive filtering setting. The SPARLS algorithm exploits noisy observations of the tap-weight vector output stream and produces its estimate using an Expectation-Maximization type algorithm. Simulation studies in the context of channel estimation, employing multi-path wireless channels, show that the SPARLS algorithm has significant improvement over the conventional widely-used Recursive Least Squares (RLS) algorithm, in terms of both mean squared error (MSE) and computational complexity. △ Less

Submitted 6 January, 2009; originally announced January 2009.

A Distributed Dynamic Frequency Allocation Algorithm

Authors: Behtash Babadi, Vahid Tarokh

Abstract: We consider a network model where the nodes are grouped into a number of clusters and propose a distributed dynamic frequency allocation algorithm that achieves performance close to that of a centralized optimal algorithm. Each cluster chooses its transmission frequency band based on its knowledge of the interference that it experiences. The convergence of the proposed distributed algorithm to a… ▽ More We consider a network model where the nodes are grouped into a number of clusters and propose a distributed dynamic frequency allocation algorithm that achieves performance close to that of a centralized optimal algorithm. Each cluster chooses its transmission frequency band based on its knowledge of the interference that it experiences. The convergence of the proposed distributed algorithm to a sub-optimal frequency allocation pattern is proved. For some specific cases of spatial distributions of the clusters in the network, asymptotic bounds on the performance of the algorithm are derived and comparisons to the performance of optimal centralized solutions are made. These analytic results and additional simulation studies verify performance close to that of an optimum centralized frequency allocation algorithm. It is demonstrated that the algorithm achieves about 90% of the Shannon capacities corresponding to the optimum/near-optimum centralized frequency band assignments. Furthermore, we consider the scenario where each cluster can be in active or inactive mode according to a two-state Markov model. We derive conditions to guarantee finite steady state variance for the output of the algorithm using stochastic analysis. Further simulation studies confirm the results of stochastic modeling and the performance of the algorithm in the time-varying setup. △ Less

Submitted 13 March, 2008; v1 submitted 20 November, 2007; originally announced November 2007.