-
Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signals
Authors:
Christian Holberg,
Cristopher Salvi
Abstract:
We introduce a mathematically rigorous framework based on rough path theory to model stochastic spiking neural networks (SSNNs) as stochastic differential equations with event discontinuities (Event SDEs) and driven by càdlàg rough paths. Our formalism is general enough to allow for potential jumps to be present both in the solution trajectories as well as in the driving noise. We then identify a…
▽ More
We introduce a mathematically rigorous framework based on rough path theory to model stochastic spiking neural networks (SSNNs) as stochastic differential equations with event discontinuities (Event SDEs) and driven by càdlàg rough paths. Our formalism is general enough to allow for potential jumps to be present both in the solution trajectories as well as in the driving noise. We then identify a set of sufficient conditions ensuring the existence of pathwise gradients of solution trajectories and event times with respect to the network's parameters and show how these gradients satisfy a recursive relation. Furthermore, we introduce a general-purpose loss function defined by means of a new class of signature kernels indexed on càdlàg rough paths and use it to train SSNNs as generative models. We provide an end-to-end autodifferentiable solver for Event SDEs and make its implementation available as part of the $\texttt{diffrax}$ library. Our framework is, to our knowledge, the first enabling gradient-based training of SSNNs with noise affecting both the spike timing and the network's dynamics.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Uniform Inference for Cointegrated Vector Autoregressive Processes
Authors:
Christian Holberg,
Susanne Ditlevsen
Abstract:
Uniformly valid inference for cointegrated vector autoregressive processes has so far proven difficult due to certain discontinuities arising in the asymptotic distribution of the least squares estimator. We extend asymptotic results from the univariate case to multiple dimensions and show how inference can be based on these results. Furthermore, we show that lag augmentation and a recent instrume…
▽ More
Uniformly valid inference for cointegrated vector autoregressive processes has so far proven difficult due to certain discontinuities arising in the asymptotic distribution of the least squares estimator. We extend asymptotic results from the univariate case to multiple dimensions and show how inference can be based on these results. Furthermore, we show that lag augmentation and a recent instrumental variable procedure can also yield uniformly valid tests and confidence regions. We verify the theoretical findings and investigate finite sample properties in simulation experiments for two specific examples.
△ Less
Submitted 7 December, 2023; v1 submitted 6 June, 2023;
originally announced June 2023.
-
Weighted Reduced Rank Estimators Under Cointegration Rank Uncertainty
Authors:
Christian Holberg,
Susanne Ditlevsen
Abstract:
Cointegration analysis was developed for non-stationary linear processes that exhibit stationary relationships between coordinates. Estimation of the cointegration relationships in a multi-dimensional cointegrated process typically proceeds in two steps. First the rank is estimated, then the cointegration matrix is estimated, conditionally on the estimated rank (reduced rank regression). The asymp…
▽ More
Cointegration analysis was developed for non-stationary linear processes that exhibit stationary relationships between coordinates. Estimation of the cointegration relationships in a multi-dimensional cointegrated process typically proceeds in two steps. First the rank is estimated, then the cointegration matrix is estimated, conditionally on the estimated rank (reduced rank regression). The asymptotics of the estimator is usually derived under the assumption of knowing the true rank. In this paper, we quantify the asymptotic bias and find the asymptotic distributions of the cointegration estimator in case of misspecified rank. Furthermore, we suggest a new class of weighted reduced rank estimators that allow for more flexibility in settings where rank selection is hard. We show empirically that a proper choice of weights can lead to increased predictive performance when there is rank uncertainty. Finally, we illustrate the estimators on empirical EEG data from a psychological experiment on visual processing.
△ Less
Submitted 18 September, 2023; v1 submitted 9 August, 2022;
originally announced August 2022.
