-
Hybrid Real- and Complex-valued Neural Network Architecture
Authors:
Alex Young,
Luan Vinícius Fiorio,
Bo Yang,
Boris Karanov,
Wim van Houtum,
Ronald M. Aarts
Abstract:
We propose a \emph{hybrid} real- and complex-valued \emph{neural network} (HNN) architecture, designed to combine the computational efficiency of real-valued processing with the ability to effectively handle complex-valued data. We illustrate the limitations of using real-valued neural networks (RVNNs) for inherently complex-valued problems by showing how it learnt to perform complex-valued convol…
▽ More
We propose a \emph{hybrid} real- and complex-valued \emph{neural network} (HNN) architecture, designed to combine the computational efficiency of real-valued processing with the ability to effectively handle complex-valued data. We illustrate the limitations of using real-valued neural networks (RVNNs) for inherently complex-valued problems by showing how it learnt to perform complex-valued convolution, but with notable inefficiencies stemming from its real-valued constraints. To create the HNN, we propose to use building blocks containing both real- and complex-valued paths, where information between domains is exchanged through domain conversion functions. We also introduce novel complex-valued activation functions, with higher generalisation and parameterisation efficiency. HNN-specific architecture search techniques are described to navigate the larger solution space. Experiments with the AudioMNIST dataset demonstrate that the HNN reduces cross-entropy loss and consumes less parameters compared to an RVNN for all considered cases. Such results highlight the potential for the use of partially complex-valued processing in neural networks and applications for HNNs in many signal processing domains.
△ Less
Submitted 4 April, 2025;
originally announced April 2025.
-
Spectral Masking with Explicit Time-Context Windowing for Neural Network-Based Monaural Speech Enhancement
Authors:
Luan Vinícius Fiorio,
Boris Karanov,
Bruno Defraene,
Johan David,
Wim van Houtum,
Frans Widdershoven,
Ronald M. Aarts
Abstract:
We propose and analyze the use of an explicit time-context window for neural network-based spectral masking speech enhancement to leverage signal context dependencies between neighboring frames. In particular, we concentrate on soft masking and loss computed on the time-frequency representation of the reconstructed speech. We show that the application of a time-context windowing function at both i…
▽ More
We propose and analyze the use of an explicit time-context window for neural network-based spectral masking speech enhancement to leverage signal context dependencies between neighboring frames. In particular, we concentrate on soft masking and loss computed on the time-frequency representation of the reconstructed speech. We show that the application of a time-context windowing function at both input and output of the neural network model improves the soft mask estimation process by combining multiple estimates taken from different contexts. The proposed approach is only applied as post-optimization in inference mode, not requiring additional layers or special training for the neural network model. Our results show that the method consistently increases both intelligibility and signal quality of the denoised speech, as demonstrated for two classes of convolutional-based speech enhancement models. Importantly, the proposed method requires only a negligible ($\leq1\%$) increase in the number of model parameters, making it suitable for hardware-constrained applications.
△ Less
Submitted 28 August, 2024;
originally announced August 2024.
-
Internal-Coordinate Density Modelling of Protein Structure: Covariance Matters
Authors:
Marloes Arts,
Jes Frellsen,
Wouter Boomsma
Abstract:
After the recent ground-breaking advances in protein structure prediction, one of the remaining challenges in protein machine learning is to reliably predict distributions of structural states. Parametric models of fluctuations are difficult to fit due to complex covariance structures between degrees of freedom in the protein chain, often causing models to either violate local or global structural…
▽ More
After the recent ground-breaking advances in protein structure prediction, one of the remaining challenges in protein machine learning is to reliably predict distributions of structural states. Parametric models of fluctuations are difficult to fit due to complex covariance structures between degrees of freedom in the protein chain, often causing models to either violate local or global structural constraints. In this paper, we present a new strategy for modelling protein densities in internal coordinates, which uses constraints in 3D space to induce covariance structure between the internal degrees of freedom. We illustrate the potential of the procedure by constructing a variational autoencoder with full covariance output induced by the constraints implied by the conditional mean in 3D, and demonstrate that our approach makes it possible to scale density models of internal coordinates to full protein backbones in two settings: 1) a unimodal setting for proteins exhibiting small fluctuations and limited amounts of available data, and 2) a multimodal setting for larger conformational changes in a high data regime.
△ Less
Submitted 24 January, 2024; v1 submitted 27 February, 2023;
originally announced February 2023.
-
Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics
Authors:
Marloes Arts,
Victor Garcia Satorras,
Chin-Wei Huang,
Daniel Zuegner,
Marco Federici,
Cecilia Clementi,
Frank Noé,
Robert Pinsler,
Rianne van den Berg
Abstract:
Coarse-grained (CG) molecular dynamics enables the study of biological processes at temporal and spatial scales that would be intractable at an atomistic resolution. However, accurately learning a CG force field remains a challenge. In this work, we leverage connections between score-based generative models, force fields and molecular dynamics to learn a CG force field without requiring any force…
▽ More
Coarse-grained (CG) molecular dynamics enables the study of biological processes at temporal and spatial scales that would be intractable at an atomistic resolution. However, accurately learning a CG force field remains a challenge. In this work, we leverage connections between score-based generative models, force fields and molecular dynamics to learn a CG force field without requiring any force inputs during training. Specifically, we train a diffusion generative model on protein structures from molecular dynamics simulations, and we show that its score function approximates a force field that can directly be used to simulate CG molecular dynamics. While having a vastly simplified training setup compared to previous work, we demonstrate that our approach leads to improved performance across several small- to medium-sized protein simulations, reproducing the CG equilibrium distribution, and preserving dynamics of all-atom simulations such as protein folding events.
△ Less
Submitted 22 September, 2023; v1 submitted 1 February, 2023;
originally announced February 2023.
-
Compressive Cyclostationary Spectrum Sensing with a Constant False Alarm Rate
Authors:
Andreas Bollig,
Martijn Arts,
Anastasia Lavrenko,
Rudolf Mathar
Abstract:
Spectrum sensing is a crucial component of opportunistic spectrum access schemes, which aim at improving spectrum utilization by allowing for the reuse of idle licensed spectrum. Sensing a spectral band before using it makes sure the legitimate users are not disturbed. Since information about these users' signals is not necessarily available, the sensor should be able to conduct so-called blind sp…
▽ More
Spectrum sensing is a crucial component of opportunistic spectrum access schemes, which aim at improving spectrum utilization by allowing for the reuse of idle licensed spectrum. Sensing a spectral band before using it makes sure the legitimate users are not disturbed. Since information about these users' signals is not necessarily available, the sensor should be able to conduct so-called blind spectrum sensing. Historically, this has not been a feature of cyclostationarity-based algorithms. Indeed, in many application scenarios the information required for traditional cyclostationarity detection might not be available, hindering its practical applicability. In this work we propose two new cyclostationary spectrum sensing algorithms that make use of the inherent sparsity of the cyclic autocorrelation to make blind operation possible. Along with utilizing sparse recovery methods for estimating the cyclic autocorrelation, we take further advantage of its structure by introducing joint sparsity as well as general structure dictionaries into the recovery process. Furthermore, we extend a statistical test for cyclostationarity to accommodate sparse cyclic spectra. Our numerical results demonstrate that the new methods achieve a near constant false alarm rate behavior in contrast to earlier approaches from the literature.
△ Less
Submitted 13 October, 2016;
originally announced October 2016.
-
SNR-Walls in Eigenvalue-based Spectrum Sensing
Authors:
Andreas Bollig,
Constantin Disch,
Martijn Arts,
Rudolf Mathar
Abstract:
Various spectrum sensing approaches have been shown to suffer from a so-called SNR-wall, an SNR value below which a detector cannot perform robustly no matter how many observations are used. Up to now, the eigenvalue-based maximum-minimum-eigenvalue (MME) detector has been a notable exception. For instance, the model uncertainty of imperfect knowledge of the receiver noise power, which is known to…
▽ More
Various spectrum sensing approaches have been shown to suffer from a so-called SNR-wall, an SNR value below which a detector cannot perform robustly no matter how many observations are used. Up to now, the eigenvalue-based maximum-minimum-eigenvalue (MME) detector has been a notable exception. For instance, the model uncertainty of imperfect knowledge of the receiver noise power, which is known to be responsible for the energy detector's fundamental limits, does not adversely affect the MME detector's performance. While additive white Gaussian noise (AWGN) is a standard assumption in wireless communications, it is not a reasonable one for the MME detector. In fact, in this work we prove that uncertainty in the amount of noise coloring does lead to an SNR-wall for the MME detector. We derive a lower bound on this SNR-wall and evaluate it for example scenarios. The findings are supported by numerical simulations.
△ Less
Submitted 12 October, 2016;
originally announced October 2016.
-
A Discontinuous Neural Network for Non-Negative Sparse Approximation
Authors:
Martijn Arts,
Marius Cordts,
Monika Gorin,
Marc Spehr,
Rudolf Mathar
Abstract:
This paper investigates a discontinuous neural network which is used as a model of the mammalian olfactory system and can more generally be applied to solve non-negative sparse approximation problems. By inherently limiting the systems integrators to having non-negative outputs, the system function becomes discontinuous since the integrators switch between being inactive and being active. It is sh…
▽ More
This paper investigates a discontinuous neural network which is used as a model of the mammalian olfactory system and can more generally be applied to solve non-negative sparse approximation problems. By inherently limiting the systems integrators to having non-negative outputs, the system function becomes discontinuous since the integrators switch between being inactive and being active. It is shown that the presented network converges to equilibrium points which are solutions to general non-negative least squares optimization problems. We specify a Caratheodory solution and prove that the network is stable, provided that the system matrix has full column-rank. Under a mild condition on the equilibrium point, we show that the network converges to its equilibrium within a finite number of switches. Two applications of the neural network are shown. Firstly, we apply the network as a model of the olfactory system and show that in principle it may be capable of performing complex sparse signal recovery tasks. Secondly, we generalize the application to include non-negative sparse approximation problems and compare the recovery performance to a classical non-negative basis pursuit denoising algorithm. We conclude that the recovery performance differs only marginally from the classical algorithm, while the neural network has the advantage that no performance critical regularization parameter has to be chosen prior to recovery.
△ Less
Submitted 21 March, 2016;
originally announced March 2016.
-
Quickest Eigenvalue-Based Spectrum Sensing using Random Matrix Theory
Authors:
Martijn Arts,
Andreas Bollig,
Rudolf Mathar
Abstract:
We investigate the potential of quickest detection based on the eigenvalues of the sample covariance matrix for spectrum sensing applications. A simple phase shift keying (PSK) model with additive white Gaussian noise (AWGN), with $1$ primary user (PU) and $K$ secondary users (SUs) is considered. Under both detection hypotheses $\mathcal{H}_0$ (noise only) and $\mathcal{H}_1$ (signal + noise) the…
▽ More
We investigate the potential of quickest detection based on the eigenvalues of the sample covariance matrix for spectrum sensing applications. A simple phase shift keying (PSK) model with additive white Gaussian noise (AWGN), with $1$ primary user (PU) and $K$ secondary users (SUs) is considered. Under both detection hypotheses $\mathcal{H}_0$ (noise only) and $\mathcal{H}_1$ (signal + noise) the eigenvalues of the sample covariance matrix follow Wishart distributions. For the case of $K = 2$ SUs, we derive an analytical formulation of the probability density function (PDF) of the maximum-minimum eigenvalue (MME) detector under $\mathcal{H}_1$. Utilizing results from the literature under $\mathcal{H}_0$, we investigate two detection schemes. First, we calculate the receiver operator characteristic (ROC) for MME block detector based on analytical results. Second, we introduce two eigenvalue-based quickest detection algorithms: a cumulative sum (CUSUM) algorithm, when the signal-to-noise ratio (SNR) of the PU signal is known and an algorithm using the generalized likelihood ratio, in case the SNR is unknown. Bounds on the mean time to false-alarm $τ_\text{fa}$ and the mean time to detection $τ_\text{d}$ are given for the CUSUM algorithm. Numerical simulations illustrate the potential advantages of the quickest detection approach over the block detection scheme.
△ Less
Submitted 13 October, 2015; v1 submitted 7 April, 2015;
originally announced April 2015.
