-
Biodenoising: Animal Vocalization Denoising without Access to Clean Data
Authors:
Marius Miron,
Sara Keen,
Jen-Yu Liu,
Benjamin Hoffman,
Masato Hagiwara,
Olivier Pietquin,
Felix Effenberger,
Maddie Cusimano
Abstract:
Animal vocalization denoising is a task similar to human speech enhancement, which is relatively well-studied. In contrast to the latter, it comprises a higher diversity of sound production mechanisms and recording environments, and this higher diversity is a challenge for existing models. Adding to the challenge and in contrast to speech, we lack large and diverse datasets comprising clean vocali…
▽ More
Animal vocalization denoising is a task similar to human speech enhancement, which is relatively well-studied. In contrast to the latter, it comprises a higher diversity of sound production mechanisms and recording environments, and this higher diversity is a challenge for existing models. Adding to the challenge and in contrast to speech, we lack large and diverse datasets comprising clean vocalizations. As a solution we use as training data pseudo-clean targets, i.e. pre-denoised vocalizations, and segments of background noise without a vocalization. We propose a train set derived from bioacoustics datasets and repositories representing diverse species, acoustic environments, geographic regions. Additionally, we introduce a non-overlapping benchmark set comprising clean vocalizations from different taxa and noise samples. We show that that denoising models (demucs, CleanUNet) trained on pseudo-clean targets obtained with speech enhancement models achieve competitive results on the benchmarking set. We publish data, code, libraries, and demos at https://earthspecies.github.io/biodenoising/.
△ Less
Submitted 10 March, 2025; v1 submitted 4 October, 2024;
originally announced October 2024.
-
Fading memory as inductive bias in residual recurrent networks
Authors:
Igor Dubinin,
Felix Effenberger
Abstract:
Residual connections have been proposed as an architecture-based inductive bias to mitigate the problem of exploding and vanishing gradients and increased task performance in both feed-forward and recurrent networks (RNNs) when trained with the backpropagation algorithm. Yet, little is known about how residual connections in RNNs influence their dynamics and fading memory properties. Here, we intr…
▽ More
Residual connections have been proposed as an architecture-based inductive bias to mitigate the problem of exploding and vanishing gradients and increased task performance in both feed-forward and recurrent networks (RNNs) when trained with the backpropagation algorithm. Yet, little is known about how residual connections in RNNs influence their dynamics and fading memory properties. Here, we introduce weakly coupled residual recurrent networks (WCRNNs) in which residual connections result in well-defined Lyapunov exponents and allow for studying properties of fading memory. We investigate how the residual connections of WCRNNs influence their performance, network dynamics, and memory properties on a set of benchmark tasks. We show that several distinct forms of residual connections yield effective inductive biases that result in increased network expressivity. In particular, those are residual connections that (i) result in network dynamics at the proximity of the edge of chaos, (ii) allow networks to capitalize on characteristic spectral properties of the data, and (iii) result in heterogeneous memory properties. In addition, we demonstrate how our results can be extended to non-linear residuals and introduce a weakly coupled residual initialization scheme that can be used for Elman RNNs.
△ Less
Submitted 3 January, 2024; v1 submitted 27 July, 2023;
originally announced July 2023.
-
A Comparative Study on Generative Models for High Resolution Solar Observation Imaging
Authors:
Mehdi Cherti,
Alexander Czernik,
Stefan Kesselheim,
Frederic Effenberger,
Jenia Jitsev
Abstract:
Solar activity is one of the main drivers of variability in our solar system and the key source of space weather phenomena that affect Earth and near Earth space. The extensive record of high resolution extreme ultraviolet (EUV) observations from the Solar Dynamics Observatory (SDO) offers an unprecedented, very large dataset of solar images. In this work, we make use of this comprehensive dataset…
▽ More
Solar activity is one of the main drivers of variability in our solar system and the key source of space weather phenomena that affect Earth and near Earth space. The extensive record of high resolution extreme ultraviolet (EUV) observations from the Solar Dynamics Observatory (SDO) offers an unprecedented, very large dataset of solar images. In this work, we make use of this comprehensive dataset to investigate capabilities of current state-of-the-art generative models to accurately capture the data distribution behind the observed solar activity states. Starting from StyleGAN-based methods, we uncover severe deficits of this model family in handling fine-scale details of solar images when training on high resolution samples, contrary to training on natural face images. When switching to the diffusion based generative model family, we observe strong improvements of fine-scale detail generation. For the GAN family, we are able to achieve similar improvements in fine-scale generation when turning to ProjectedGANs, which uses multi-scale discriminators with a pre-trained frozen feature extractor. We conduct ablation studies to clarify mechanisms responsible for proper fine-scale handling. Using distributed training on supercomputers, we are able to train generative models for up to 1024x1024 resolution that produce high quality samples indistinguishable to human experts, as suggested by the evaluation we conduct. We make all code, models and workflows used in this study publicly available at \url{https://github.com/SLAMPAI/generative-models-for-highres-solar-images}.
△ Less
Submitted 14 April, 2023;
originally announced April 2023.
-
A novel feature-scrambling approach reveals the capacity of convolutional neural networks to learn spatial relations
Authors:
Amr Farahat,
Felix Effenberger,
Martin Vinck
Abstract:
Convolutional neural networks (CNNs) are one of the most successful computer vision systems to solve object recognition. Furthermore, CNNs have major applications in understanding the nature of visual representations in the human brain. Yet it remains poorly understood how CNNs actually make their decisions, what the nature of their internal representations is, and how their recognition strategies…
▽ More
Convolutional neural networks (CNNs) are one of the most successful computer vision systems to solve object recognition. Furthermore, CNNs have major applications in understanding the nature of visual representations in the human brain. Yet it remains poorly understood how CNNs actually make their decisions, what the nature of their internal representations is, and how their recognition strategies differ from humans. Specifically, there is a major debate about the question of whether CNNs primarily rely on surface regularities of objects, or whether they are capable of exploiting the spatial arrangement of features, similar to humans. Here, we develop a novel feature-scrambling approach to explicitly test whether CNNs use the spatial arrangement of features (i.e. object parts) to classify objects. We combine this approach with a systematic manipulation of effective receptive field sizes of CNNs as well as minimal recognizable configurations (MIRCs) analysis. In contrast to much previous literature, we provide evidence that CNNs are in fact capable of using relatively long-range spatial relationships for object classification. Moreover, the extent to which CNNs use spatial relationships depends heavily on the dataset, e.g. texture vs. sketch. In fact, CNNs even use different strategies for different classes within heterogeneous datasets (ImageNet), suggesting CNNs have a continuous spectrum of classification strategies. Finally, we show that CNNs learn the spatial arrangement of features only up to an intermediate level of granularity, which suggests that intermediate rather than global shape features provide the optimal trade-off between sensitivity and specificity in object classification. These results provide novel insights into the nature of CNN representations and the extent to which they rely on the spatial arrangement of features for object classification.
△ Less
Submitted 12 December, 2022;
originally announced December 2022.
-
BEANS: The Benchmark of Animal Sounds
Authors:
Masato Hagiwara,
Benjamin Hoffman,
Jen-Yu Liu,
Maddie Cusimano,
Felix Effenberger,
Katie Zacarian
Abstract:
The use of machine learning (ML) based techniques has become increasingly popular in the field of bioacoustics over the last years. Fundamental requirements for the successful application of ML based techniques are curated, agreed upon, high-quality datasets and benchmark tasks to be learned on a given dataset. However, the field of bioacoustics so far lacks such public benchmarks which cover mult…
▽ More
The use of machine learning (ML) based techniques has become increasingly popular in the field of bioacoustics over the last years. Fundamental requirements for the successful application of ML based techniques are curated, agreed upon, high-quality datasets and benchmark tasks to be learned on a given dataset. However, the field of bioacoustics so far lacks such public benchmarks which cover multiple tasks and species to measure the performance of ML techniques in a controlled and standardized way and that allows for benchmarking newly proposed techniques to existing ones. Here, we propose BEANS (the BEnchmark of ANimal Sounds), a collection of bioacoustics tasks and public datasets, specifically designed to measure the performance of machine learning algorithms in the field of bioacoustics. The benchmark proposed here consists of two common tasks in bioacoustics: classification and detection. It includes 12 datasets covering various species, including birds, land and marine mammals, anurans, and insects. In addition to the datasets, we also present the performance of a set of standard ML methods as the baseline for task performance. The benchmark and baseline code is made publicly available at \url{https://github.com/earthspecies/beans} in the hope of establishing a new standard dataset for ML-based bioacoustic research.
△ Less
Submitted 21 October, 2022;
originally announced October 2022.
-
A primer on information theory, with applications to neuroscience
Authors:
Felix Effenberger
Abstract:
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the neurosciences. Such analyses can provide deep insights into the functionality of such systems, as well as a rigid mathematical theory and quantitative measures of infor…
▽ More
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the neurosciences. Such analyses can provide deep insights into the functionality of such systems, as well as a rigid mathematical theory and quantitative measures of information processing in both healthy and diseased states of neural systems. This chapter will present a short introduction to the fundamentals of information theory, especially suited for people having a less firm background in mathematics and probability theory. To begin, the fundamentals of probability theory such as the notion of probability, probability distributions, and random variables will be reviewed. Then, the concepts of information and entropy (in the sense of Shannon), mutual information, and transfer entropy (sometimes also referred to as conditional mutual information) will be outlined. As these quantities cannot be computed exactly from measured data in practice, estimation techniques for information-theoretic quantities will be presented. The chapter will conclude with the applications of information theory in the field of neuroscience, including questions of possible medical applications and a short review of software packages that can be used for information-theoretic analyses of neural data.
△ Less
Submitted 7 October, 2013; v1 submitted 8 April, 2013;
originally announced April 2013.
-
Simplicial blowups and discrete normal surfaces in simpcomp
Authors:
Felix Effenberger,
Jonathan Spreer
Abstract:
simpcomp is an extension to GAP, the well known system for computational discrete algebra. It allows the user to work with simplicial complexes. In the latest version, support for simplicial blowups and discrete normal surfaces was added, both features unique to simpcomp. Furthermore, new functions for constructing certain infinite series of triangulations have been implemented and interfaces to o…
▽ More
simpcomp is an extension to GAP, the well known system for computational discrete algebra. It allows the user to work with simplicial complexes. In the latest version, support for simplicial blowups and discrete normal surfaces was added, both features unique to simpcomp. Furthermore, new functions for constructing certain infinite series of triangulations have been implemented and interfaces to other software packages have been improved to previous versions.
△ Less
Submitted 26 May, 2011;
originally announced May 2011.
-
Finding and Classifying Critical Points of 2D Vector Fields: A Cell-Oriented Approach Using Group Theory
Authors:
Felix Effenberger,
Daniel Weiskopf
Abstract:
We present a novel approach to finding critical points in cell-wise barycentrically or bilinearly interpolated vector fields on surfaces. The Poincar\e index of the critical points is determined by investigating the qualitative behavior of 0-level sets of the interpolants of the vector field components in parameter space using precomputed combinatorial results, thus avoiding the computation of the…
▽ More
We present a novel approach to finding critical points in cell-wise barycentrically or bilinearly interpolated vector fields on surfaces. The Poincar\e index of the critical points is determined by investigating the qualitative behavior of 0-level sets of the interpolants of the vector field components in parameter space using precomputed combinatorial results, thus avoiding the computation of the Jacobian of the vector field at the critical points in order to determine its index. The locations of the critical points within a cell are determined analytically to achieve accurate results. This approach leads to a correct treatment of cases with two first-order critical points or one second-order critical point of bilinearly interpolated vector fields within one cell, which would be missed by examining the linearized field only. We show that for the considered interpolation schemes determining the index of a critical point can be seen as a coloring problem of cell edges. A complete classification of all possible colorings in terms of the types and number of critical points yielded by each coloring is given using computational group theory. We present an efficient algorithm that makes use of these precomputed classifications in order to find and classify critical points in a cell-by-cell fashion. Issues of numerical stability, construction of the topological skeleton, topological simplification, and the statistics of the different types of critical points are also discussed.
△ Less
Submitted 26 April, 2010;
originally announced April 2010.
-
simpcomp -- A GAP toolbox for simplicial complexes
Authors:
Felix Effenberger,
Jonathan Spreer
Abstract:
simpcomp is an extension (a so called package) to GAP, the well known system for computational discrete algebra. The package enables the user to compute numerous properties of (abstract) simplicial complexes, provides functions to construct new complexes from existing ones and an extensive library of triangulations of manifolds.
simpcomp is an extension (a so called package) to GAP, the well known system for computational discrete algebra. The package enables the user to compute numerous properties of (abstract) simplicial complexes, provides functions to construct new complexes from existing ones and an extensive library of triangulations of manifolds.
△ Less
Submitted 14 September, 2010; v1 submitted 8 April, 2010;
originally announced April 2010.
