Showing 1–28 of 28 results for author: Imbiriba, T

Robust Recursive Fusion of Multiresolution Multispectral Images with Location-Aware Neural Networks

Authors: Haoqing Li, Ricardo Borsoi, Tales Imbiriba, Pau Closas

Abstract: Multiresolution image fusion is a key problem for real-time satellite imaging and plays a central role in detecting and monitoring natural phenomena such as floods. It aims to solve the trade-off between temporal and spatial resolution in remote sensing instruments. Although several algorithms have been proposed for this problem, the presence of outliers such as clouds downgrades their performance… ▽ More Multiresolution image fusion is a key problem for real-time satellite imaging and plays a central role in detecting and monitoring natural phenomena such as floods. It aims to solve the trade-off between temporal and spatial resolution in remote sensing instruments. Although several algorithms have been proposed for this problem, the presence of outliers such as clouds downgrades their performance. Moreover, strategies that integrate robustness, recursive operation and learned models are missing. In this paper, a robust recursive image fusion framework leveraging location-aware neural networks (NN) to model the image dynamics is proposed. Outliers are modeled by representing the probability of contamination of a given pixel and band. A NN model trained on a small dataset provides accurate predictions of the stochastic image time evolution, which improves both the accuracy and robustness of the method. A recursive solution is proposed to estimate the high-resolution images using a Bayesian variational inference framework. Experiments fusing images from the Landsat 8 and MODIS instruments show that the proposed approach is significantly more robust against cloud cover, without losing performance when no clouds are present. △ Less

Submitted 16 June, 2025; originally announced June 2025.

Interpretable Augmented Physics-Based Model for Estimation and Tracking

Authors: Ondřej Straka, Jindřich Duník, Pau Closas, Tales Imbiriba

Abstract: State-space estimation and tracking rely on accurate dynamical models to perform well. However, obtaining an vaccurate dynamical model for complex scenarios or adapting to changes in the system poses challenges to the estimation process. Recently, augmented physics-based models (APBMs) appear as an appealing strategy to cope with these challenges where the composition of a small and adaptive neura… ▽ More State-space estimation and tracking rely on accurate dynamical models to perform well. However, obtaining an vaccurate dynamical model for complex scenarios or adapting to changes in the system poses challenges to the estimation process. Recently, augmented physics-based models (APBMs) appear as an appealing strategy to cope with these challenges where the composition of a small and adaptive neural network with known physics-based models (PBM) is learned on the fly following an augmented state-space estimation approach. A major issue when introducing data-driven components in such a scenario is the danger of compromising the meaning (or interpretability) of estimated states. In this work, we propose a novel constrained estimation strategy that constrains the APBM dynamics close to the PBM. The novel state-space constrained approach leads to more flexible ways to impose constraints than the traditional APBM approach. Our experiments with a radar-tracking scenario demonstrate different aspects of the proposed approach and the trade-offs inherent in the imposed constraints. △ Less

Submitted 9 June, 2025; v1 submitted 26 May, 2025; originally announced May 2025.

Comments: Accepted for presentation at the FUSION 2025 conference

AI-Aided Kalman Filters

Authors: Nir Shlezinger, Guy Revach, Anubhab Ghosh, Saikat Chatterjee, Shuo Tang, Tales Imbiriba, Jindrich Dunik, Ondrej Straka, Pau Closas, Yonina C. Eldar

Abstract: The Kalman filter (KF) and its variants are among the most celebrated algorithms in signal processing. These methods are used for state estimation of dynamic systems by relying on mathematical representations in the form of simple state-space (SS) models, which may be crude and inaccurate descriptions of the underlying dynamics. Emerging data-centric artificial intelligence (AI) techniques tackle… ▽ More The Kalman filter (KF) and its variants are among the most celebrated algorithms in signal processing. These methods are used for state estimation of dynamic systems by relying on mathematical representations in the form of simple state-space (SS) models, which may be crude and inaccurate descriptions of the underlying dynamics. Emerging data-centric artificial intelligence (AI) techniques tackle these tasks using deep neural networks (DNNs), which are model-agnostic. Recent developments illustrate the possibility of fusing DNNs with classic Kalman-type filtering, obtaining systems that learn to track in partially known dynamics. This article provides a tutorial-style overview of design approaches for incorporating AI in aiding KF-type algorithms. We review both generic and dedicated DNN architectures suitable for state estimation, and provide a systematic presentation of techniques for fusing AI tools with KFs and for leveraging partial SS modeling and data, categorizing design approaches into task-oriented and SS model-oriented. The usefulness of each approach in preserving the individual strengths of model-based KFs and data-driven DNNs is investigated in a qualitative and quantitative study, whose code is publicly available, illustrating the gains of hybrid model-based/data-driven designs. We also discuss existing challenges and future research directions that arise from fusing AI and Kalman-type algorithms. △ Less

Submitted 6 May, 2025; v1 submitted 16 October, 2024; originally announced October 2024.

Comments: Submitted to the IEEE Signal Processing Magazine

Mitigation of Radar Range Deception Jamming Using Random Finite Sets

Authors: Helena Calatrava, Aanjhan Ranganathan, Tales Imbiriba, Gunar Schirner, Murat Akcakaya, Pau Closas

Abstract: This paper presents a radar target tracking framework for addressing main-beam range deception jamming attacks using random finite sets (RFSs). Our system handles false alarms and detections with false range information through multiple hypothesis tracking (MHT) to resolve data association uncertainties. We focus on range gate pull-off (RGPO) attacks, where the attacker adds positive delays to the… ▽ More This paper presents a radar target tracking framework for addressing main-beam range deception jamming attacks using random finite sets (RFSs). Our system handles false alarms and detections with false range information through multiple hypothesis tracking (MHT) to resolve data association uncertainties. We focus on range gate pull-off (RGPO) attacks, where the attacker adds positive delays to the radar pulse, thereby mimicking the target trajectory while appearing at a larger distance from the radar. The proposed framework incorporates knowledge about the spatial behavior of the attack into the assumed RFS clutter model and uses only position information without relying on additional signal features. We present an adaptive solution that estimates the jammer-induced biases to improve tracking accuracy as well as a simpler non-adaptive version that performs well when accurate priors on the jamming range are available. Furthermore, an expression for RGPO attack detection is derived, where the adaptive solution offers superior performance. The presented strategies provide tracking resilience against multiple RGPO attacks in terms of position estimation accuracy and jamming detection without degrading tracking performance in the absence of jamming. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: 12 pages, 7 figures

Multistatic-Radar RCS-Signature Recognition of Aerial Vehicles: A Bayesian Fusion Approach

Authors: Michael Potter, Murat Akcakaya, Marius Necsoiu, Gunar Schirner, Deniz Erdogmus, Tales Imbiriba

Abstract: Radar Automated Target Recognition (RATR) for Unmanned Aerial Vehicles (UAVs) involves transmitting Electromagnetic Waves (EMWs) and performing target type recognition on the received radar echo, crucial for defense and aerospace applications. Previous studies highlighted the advantages of multistatic radar configurations over monostatic ones in RATR. However, fusion methods in multistatic radar c… ▽ More Radar Automated Target Recognition (RATR) for Unmanned Aerial Vehicles (UAVs) involves transmitting Electromagnetic Waves (EMWs) and performing target type recognition on the received radar echo, crucial for defense and aerospace applications. Previous studies highlighted the advantages of multistatic radar configurations over monostatic ones in RATR. However, fusion methods in multistatic radar configurations often suboptimally combine classification vectors from individual radars probabilistically. To address this, we propose a fully Bayesian RATR framework employing Optimal Bayesian Fusion (OBF) to aggregate classification probability vectors from multiple radars. OBF, based on expected 0-1 loss, updates a Recursive Bayesian Classification (RBC) posterior distribution for target UAV type, conditioned on historical observations across multiple time steps. We evaluate the approach using simulated random walk trajectories for seven drones, correlating target aspect angles to Radar Cross Section (RCS) measurements in an anechoic chamber. Comparing against single radar Automated Target Recognition (ATR) systems and suboptimal fusion methods, our empirical results demonstrate that the OBF method integrated with RBC significantly enhances classification accuracy compared to other fusion methods and single radar configurations. △ Less

Submitted 15 August, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

Comments: Accepted to IEEE Transactions on Aerospace and Electronic Systems

On the Impact of Sampling on Deep Sequential State Estimation

Authors: Helena Calatrava, Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas

Abstract: State inference and parameter learning in sequential models can be successfully performed with approximation techniques that maximize the evidence lower bound to the marginal log-likelihood of the data distribution. These methods may be referred to as Dynamical Variational Autoencoders, and our specific focus lies on the deep Kalman filter. It has been shown that the ELBO objective can oversimplif… ▽ More State inference and parameter learning in sequential models can be successfully performed with approximation techniques that maximize the evidence lower bound to the marginal log-likelihood of the data distribution. These methods may be referred to as Dynamical Variational Autoencoders, and our specific focus lies on the deep Kalman filter. It has been shown that the ELBO objective can oversimplify data representations, potentially compromising estimation quality. Tighter Monte Carlo objectives have been proposed in the literature to enhance generative modeling performance. For instance, the IWAE objective uses importance weights to reduce the variance of marginal log-likelihood estimates. In this paper, importance sampling is applied to the DKF framework for learning deep Markov models, resulting in the IW-DKF, which shows an improvement in terms of log-likelihood estimates and KL divergence between the variational distribution and the transition model. The framework using the sampled DKF update rule is also accommodated to address sequential state and parameter estimation when working with highly non-linear physics-based models. An experiment with the 3-space Lorenz attractor shows an enhanced generative modeling performance and also a decrease in RMSE when estimating the model parameters and latent states, indicating that tighter MCOs lead to improved state inference performance. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Comments: To appear in the Proceedings of the Asilomar Conference on Signals, Systems, and Computers, October 2023, 5 pages, 3 figures, 3 tables

Fast and Expressive Gesture Recognition using a Combination-Homomorphic Electromyogram Encoder

Authors: Niklas Smedemark-Margulies, Yunus Bicer, Elifnur Sunger, Tales Imbiriba, Eugene Tunik, Deniz Erdogmus, Mathew Yarossi, Robin Walters

Abstract: We study the task of gesture recognition from electromyography (EMG), with the goal of enabling expressive human-computer interaction at high accuracy, while minimizing the time required for new subjects to provide calibration data. To fulfill these goals, we define combination gestures consisting of a direction component and a modifier component. New subjects only demonstrate the single component… ▽ More We study the task of gesture recognition from electromyography (EMG), with the goal of enabling expressive human-computer interaction at high accuracy, while minimizing the time required for new subjects to provide calibration data. To fulfill these goals, we define combination gestures consisting of a direction component and a modifier component. New subjects only demonstrate the single component gestures and we seek to extrapolate from these to all possible single or combination gestures. We extrapolate to unseen combination gestures by combining the feature vectors of real single gestures to produce synthetic training data. This strategy allows us to provide a large and flexible gesture vocabulary, while not requiring new subjects to demonstrate combinatorially many example gestures. We pre-train an encoder and a combination operator using self-supervision, so that we can produce useful synthetic training data for unseen test subjects. To evaluate the proposed method, we collect a real-world EMG dataset, and measure the effect of augmented supervision against two baselines: a partially-supervised model trained with only single gesture data from the unseen subject, and a fully-supervised model trained with real single and real combination gesture data from the unseen subject. We find that the proposed method provides a dramatic improvement over the partially-supervised model, and achieves a useful classification accuracy that in some cases approaches the performance of the fully-supervised model. △ Less

Submitted 29 November, 2023; v1 submitted 30 October, 2023; originally announced November 2023.

Comments: 24 pages, 7 figures, 6 tables V2: add link to code, fix bibliography

Tubular Curvature Filter: Implicit Pointwise Curvature Calculation Method for Tubular Objects

Authors: Elifnur Sunger, Beyza Kalkanli, Veysi Yildiz, Tales Imbiriba, Peter Campbell, Deniz Erdogmus

Abstract: Curvature estimation methods are important as they capture salient features for various applications in image processing, especially within medical domains where tortuosity of vascular structures is of significant interest. Existing methods based on centerline or skeleton curvature fail to capture curvature gradients across a rotating tubular structure. This paper presents a Tubular Curvature Filt… ▽ More Curvature estimation methods are important as they capture salient features for various applications in image processing, especially within medical domains where tortuosity of vascular structures is of significant interest. Existing methods based on centerline or skeleton curvature fail to capture curvature gradients across a rotating tubular structure. This paper presents a Tubular Curvature Filter method that locally calculates the acceleration of bundles of curves that traverse along the tubular object parallel to the centerline. This is achieved by examining the directional rate of change in the eigenvectors of the Hessian matrix of a tubular intensity function in space. This method implicitly calculates the local tubular curvature without the need to explicitly segment the tubular object. Experimental results demonstrate that the Tubular Curvature Filter method provides accurate estimates of local curvature at any point inside tubular structures. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Comments: 11 pages, 5 figures

Learning Interpretable Deep Disentangled Neural Networks for Hyperspectral Unmixing

Authors: Ricardo Augusto Borsoi, Deniz Erdoğmuş, Tales Imbiriba

Abstract: Although considerable effort has been dedicated to improving the solution to the hyperspectral unmixing problem, non-idealities such as complex radiation scattering and endmember variability negatively impact the performance of most existing algorithms and can be very challenging to address. Recently, deep learning-based frameworks have been explored for hyperspectral umixing due to their flexibil… ▽ More Although considerable effort has been dedicated to improving the solution to the hyperspectral unmixing problem, non-idealities such as complex radiation scattering and endmember variability negatively impact the performance of most existing algorithms and can be very challenging to address. Recently, deep learning-based frameworks have been explored for hyperspectral umixing due to their flexibility and powerful representation capabilities. However, such techniques either do not address the non-idealities of the unmixing problem, or rely on black-box models which are not interpretable. In this paper, we propose a new interpretable deep learning method for hyperspectral unmixing that accounts for nonlinearity and endmember variability. The proposed method leverages a probabilistic variational deep-learning framework, where disentanglement learning is employed to properly separate the abundances and endmembers. The model is learned end-to-end using stochastic backpropagation, and trained using a self-supervised strategy which leverages benefits from semi-supervised learning techniques. Furthermore, the model is carefully designed to provide a high degree of interpretability. This includes modeling the abundances as a Dirichlet distribution, the endmembers using low-dimensional deep latent variable representations, and using two-stream neural networks composed of additive piecewise-linear/nonlinear components. Experimental results on synthetic and real datasets illustrate the performance of the proposed method compared to state-of-the-art algorithms. △ Less

Submitted 3 October, 2023; originally announced October 2023.

A Multi-label Classification Approach to Increase Expressivity of EMG-based Gesture Recognition

Authors: Niklas Smedemark-Margulies, Yunus Bicer, Elifnur Sunger, Stephanie Naufel, Tales Imbiriba, Eugene Tunik, Deniz Erdoğmuş, Mathew Yarossi

Abstract: Objective: The objective of the study is to efficiently increase the expressivity of surface electromyography-based (sEMG) gesture recognition systems. Approach: We use a problem transformation approach, in which actions were subset into two biomechanically independent components - a set of wrist directions and a set of finger modifiers. To maintain fast calibration time, we train models for each… ▽ More Objective: The objective of the study is to efficiently increase the expressivity of surface electromyography-based (sEMG) gesture recognition systems. Approach: We use a problem transformation approach, in which actions were subset into two biomechanically independent components - a set of wrist directions and a set of finger modifiers. To maintain fast calibration time, we train models for each component using only individual gestures, and extrapolate to the full product space of combination gestures by generating synthetic data. We collected a supervised dataset with high-confidence ground truth labels in which subjects performed combination gestures while holding a joystick, and conducted experiments to analyze the impact of model architectures, classifier algorithms, and synthetic data generation strategies on the performance of the proposed approach. Main Results: We found that a problem transformation approach using a parallel model architecture in combination with a non-linear classifier, along with restricted synthetic data generation, shows promise in increasing the expressivity of sEMG-based gestures with a short calibration time. Significance: sEMG-based gesture recognition has applications in human-computer interaction, virtual reality, and the control of robotic and prosthetic devices. Existing approaches require exhaustive model calibration. The proposed approach increases expressivity without requiring users to demonstrate all combination gesture classes. Our results may be extended to larger gesture vocabularies and more complicated model architectures. △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: 14 pages, 12 figures

User Training with Error Augmentation for Electromyogram-based Gesture Classification

Authors: Yunus Bicer, Niklas Smedemark-Margulies, Basak Celik, Elifnur Sunger, Ryan Orendorff, Stephanie Naufel, Tales Imbiriba, Deniz Erdoğmuş, Eugene Tunik, Mathew Yarossi

Abstract: We designed and tested a system for real-time control of a user interface by extracting surface electromyographic (sEMG) activity from eight electrodes in a wrist-band configuration. sEMG data were streamed into a machine-learning algorithm that classified hand gestures in real-time. After an initial model calibration, participants were presented with one of three types of feedback during a human-… ▽ More We designed and tested a system for real-time control of a user interface by extracting surface electromyographic (sEMG) activity from eight electrodes in a wrist-band configuration. sEMG data were streamed into a machine-learning algorithm that classified hand gestures in real-time. After an initial model calibration, participants were presented with one of three types of feedback during a human-learning stage: veridical feedback, in which predicted probabilities from the gesture classification algorithm were displayed without alteration, modified feedback, in which we applied a hidden augmentation of error to these probabilities, and no feedback. User performance was then evaluated in a series of minigames, in which subjects were required to use eight gestures to manipulate their game avatar to complete a task. Experimental results indicated that, relative to baseline, the modified feedback condition led to significantly improved accuracy and improved gesture class separation. These findings suggest that real-time feedback in a gamified user interface with manipulation of feedback may enable intuitive, rapid, and accurate task acquisition for sEMG-based gesture recognition applications. △ Less

Submitted 22 March, 2024; v1 submitted 13 September, 2023; originally announced September 2023.

Comments: 10 pages, 10 figures. V2: Fix latex characters in author name. V3: Add published DOI and Copyright notice

Journal ref: in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 32, pp. 1187-1197, 2024

Dynamical Hyperspectral Unmixing with Variational Recurrent Neural Networks

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas

Abstract: Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. It reveals the dynamical evolution of the materials (endmembers) and of their proportions (abundances) in a given scene. However, adequately accounting for the spatial and temporal variability of the endmembers in MTHU is challenging, and has not been fully addressed so far in unsupe… ▽ More Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. It reveals the dynamical evolution of the materials (endmembers) and of their proportions (abundances) in a given scene. However, adequately accounting for the spatial and temporal variability of the endmembers in MTHU is challenging, and has not been fully addressed so far in unsupervised frameworks. In this work, we propose an unsupervised MTHU algorithm based on variational recurrent neural networks. First, a stochastic model is proposed to represent both the dynamical evolution of the endmembers and their abundances, as well as the mixing process. Moreover, a new model based on a low-dimensional parametrization is used to represent spatial and temporal endmember variability, significantly reducing the amount of variables to be estimated. We propose to formulate MTHU as a Bayesian inference problem. However, the solution to this problem does not have an analytical solution due to the nonlinearity and non-Gaussianity of the model. Thus, we propose a solution based on deep variational inference, in which the posterior distribution of the estimated abundances and endmembers is represented by using a combination of recurrent neural networks and a physically motivated model. The parameters of the model are learned using stochastic backpropagation. Experimental results show that the proposed method outperforms state of the art MTHU algorithms. △ Less

Submitted 19 March, 2023; originally announced March 2023.

On Parametric Misspecified Bayesian Cramér-Rao bound: An application to linear Gaussian systems

Authors: Shuo Tang, Gerald LaMountain, Tales Imbiriba, Pau Closas

Abstract: A lower bound is an important tool for predicting the performance that an estimator can achieve under a particular statistical model. Bayesian bounds are a kind of such bounds which not only utilizes the observation statistics but also includes the prior model information. In reality, however, the true model generating the data is either unknown or simplified when deriving estimators, which motiva… ▽ More A lower bound is an important tool for predicting the performance that an estimator can achieve under a particular statistical model. Bayesian bounds are a kind of such bounds which not only utilizes the observation statistics but also includes the prior model information. In reality, however, the true model generating the data is either unknown or simplified when deriving estimators, which motivates the works to derive estimation bounds under modeling mismatch situations. This paper provides a derivation of a Bayesian Cramér-Rao bound under model misspecification, defining important concepts such as pseudotrue parameter that were not clearly identified in previous works. The general result is particularized in linear and Gaussian problems, where closed-forms are available and results are used to validate the results. △ Less

Submitted 28 February, 2023; originally announced March 2023.

Comments: Accepted by ICASSP 2023

Online Fusion of Multi-resolution Multispectral Images with Weakly Supervised Temporal Dynamics

Authors: Haoqing Li, Bhavya Duvvuri, Ricardo Borsoi, Tales Imbiriba, Edward Beighley, Deniz Erdogmus, Pau Closas

Abstract: Real-time satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena such as floods, earthquakes, etc. One important constraint of satellite imaging is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In… ▽ More Real-time satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena such as floods, earthquakes, etc. One important constraint of satellite imaging is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In this paper, we focus on fusing multi-temporal, multi-spectral images where data acquired from different instruments with different spatial resolutions is used. We leverage the spatial relationship between images at multiple modalities to generate high-resolution image sequences at higher revisiting rates. To achieve this goal, we formulate the fusion method as a recursive state estimation problem and study its performance in filtering and smoothing contexts. Furthermore, a calibration strategy is proposed to estimate the time-varying temporal dynamics of the image sequence using only a small amount of historical image data. Differently from the training process in traditional machine learning algorithms, which usually require large datasets and computation times, the parameters of the temporal dynamical model are calibrated based on an analytical expression that uses only two of the images in the historical dataset. A distributed version of the Bayesian filtering and smoothing strategies is also proposed to reduce its computational complexity. To evaluate the proposed methodology we consider a water mapping task where real data acquired by the Landsat and MODIS instruments are fused generating high spatial-temporal resolution image estimates. Our experiments show that the proposed methodology outperforms the competing methods in both estimation accuracy and water mapping tasks. △ Less

Submitted 6 January, 2023; originally announced January 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2204.12566

Recursive classification of satellite imaging time-series: An application to land cover mapping

Authors: Helena Calatrava, Bhavya Duvvuri, Haoqing Li, Ricardo Borsoi, Edward Beighley, Deniz Erdogmus, Pau Closas, Tales Imbiriba

Abstract: Despite the extensive body of literature focused on remote sensing applications for land cover mapping and the availability of high-resolution satellite imagery, methods for continuously updating classification maps in real-time remain limited, especially when training data is scarce. This paper introduces the Recursive Bayesian Classifier (RBC), which converts any instantaneous classifier into a… ▽ More Despite the extensive body of literature focused on remote sensing applications for land cover mapping and the availability of high-resolution satellite imagery, methods for continuously updating classification maps in real-time remain limited, especially when training data is scarce. This paper introduces the Recursive Bayesian Classifier (RBC), which converts any instantaneous classifier into a robust online method through a probabilistic framework that is resilient to non-informative image variations. Three experiments are conducted using Sentinel-2 data: water mapping of the Oroville Dam in California and the Charles River basin in Massachusetts, and deforestation detection in the Amazon. RBC is applied to a Gaussian Mixture Model (GMM), Logistic Regression (LR), and our proposed Spectral Index Classifier (SIC). Results show that RBC significantly enhances classifier robustness in multitemporal settings under challenging conditions, such as cloud cover and cyanobacterial blooms. Specifically, balanced classification accuracy improves by up to 26.95% for SIC, 12.4% for GMM, and 13.81% for LR in water mapping, and by 15.25%, 14.17%, and 14.7% in deforestation detection. Moreover, without additional training data, RBC improves the performance of the state-of-the-art DeepWaterMap and WatNet algorithms by up to 9.62% and 11.03%. These benefits are provided by RBC while requiring minimal supervision and maintaining a low computational cost that remains constant for each time step regardless of the time-series length. △ Less

Submitted 9 September, 2024; v1 submitted 4 January, 2023; originally announced January 2023.

Comments: Without supplemental results: 31 pages, 16 figures, 5 tables. With supplemental results: 35 pages, 20 figures, 5 tables

Jamming Source Localization Using Augmented Physics-Based Model

Authors: Andrea Nardin, Tales Imbiriba, Pau Closas

Abstract: Monitoring interferences to satellite-based navigation systems is of paramount importance in order to reliably operate critical infrastructures, navigation systems, and a variety of applications relying on satellite-based positioning. This paper investigates the use of crowdsourced data to achieve such detection and monitoring at a central node that receives the data from an arbitrary number of ag… ▽ More Monitoring interferences to satellite-based navigation systems is of paramount importance in order to reliably operate critical infrastructures, navigation systems, and a variety of applications relying on satellite-based positioning. This paper investigates the use of crowdsourced data to achieve such detection and monitoring at a central node that receives the data from an arbitrary number of agents in an area of interest. Under ideal conditions, the pathloss model is used to compute the Cramér-Rao Bound of accuracy as well as the corresponding maximum likelihood estimator. However, in real scenarios where obstructions and reflections are common, the signal propagation is far more complex than the pathloss model can explain. We propose to augment the pathloss model with a data-driven component, able to explain the complexities of the propagation channel. The paper shows a general methodology to jointly estimate the interference location and the parameters of the augmented model, showing superior performances in complex scenarios such as those encountered in urban environments. △ Less

Submitted 15 December, 2022; originally announced December 2022.

Comments: 5 pages, 6 figures

Recursive Estimation of User Intent from Noninvasive Electroencephalography using Discriminative Models

Authors: Niklas Smedemark-Margulies, Basak Celik, Tales Imbiriba, Aziz Kocanaogullari, Deniz Erdogmus

Abstract: We study the problem of inferring user intent from noninvasive electroencephalography (EEG) to restore communication for people with severe speech and physical impairments (SSPI). The focus of this work is improving the estimation of posterior symbol probabilities in a typing task. At each iteration of the typing procedure, a subset of symbols is chosen for the next query based on the current prob… ▽ More We study the problem of inferring user intent from noninvasive electroencephalography (EEG) to restore communication for people with severe speech and physical impairments (SSPI). The focus of this work is improving the estimation of posterior symbol probabilities in a typing task. At each iteration of the typing procedure, a subset of symbols is chosen for the next query based on the current probability estimate. Evidence about the user's response is collected from event-related potentials (ERP) in order to update symbol probabilities, until one symbol exceeds a predefined confidence threshold. We provide a graphical model describing this task, and derive a recursive Bayesian update rule based on a discriminative probability over label vectors for each query, which we approximate using a neural network classifier. We evaluate the proposed method in a simulated typing task and show that it outperforms previous approaches based on generative modeling. △ Less

Submitted 29 October, 2022; originally announced November 2022.

Comments: 5 pages, 2 figures

Neural Network-based OFDM Receiver for Resource Constrained IoT Devices

Authors: Nasim Soltani, Hai Cheng, Mauro Belgiovine, Yanyu Li, Haoqing Li, Bahar Azari, Salvatore D'Oro, Tales Imbiriba, Tommaso Melodia, Pau Closas, Yanzhi Wang, Deniz Erdogmus, Kaushik Chowdhury

Abstract: Orthogonal Frequency Division Multiplexing (OFDM)-based waveforms are used for communication links in many current and emerging Internet of Things (IoT) applications, including the latest WiFi standards. For such OFDM-based transceivers, many core physical layer functions related to channel estimation, demapping, and decoding are implemented for specific choices of channel types and modulation sch… ▽ More Orthogonal Frequency Division Multiplexing (OFDM)-based waveforms are used for communication links in many current and emerging Internet of Things (IoT) applications, including the latest WiFi standards. For such OFDM-based transceivers, many core physical layer functions related to channel estimation, demapping, and decoding are implemented for specific choices of channel types and modulation schemes, among others. To decouple hard-wired choices from the receiver chain and thereby enhance the flexibility of IoT deployment in many novel scenarios without changing the underlying hardware, we explore a novel, modular Machine Learning (ML)-based receiver chain design. Here, ML blocks replace the individual processing blocks of an OFDM receiver, and we specifically describe this swapping for the legacy channel estimation, symbol demapping, and decoding blocks with Neural Networks (NNs). A unique aspect of this modular design is providing flexible allocation of processing functions to the legacy or ML blocks, allowing them to interchangeably coexist. Furthermore, we study the implementation cost-benefits of the proposed NNs in resource-constrained IoT devices through pruning and quantization, as well as emulation of these compressed NNs within Field Programmable Gate Arrays (FPGAs). Our evaluations demonstrate that the proposed modular NN-based receiver improves bit error rate of the traditional non-ML receiver by averagely 61% and 10% for the simulated and over-the-air datasets, respectively. We further show complexity-performance tradeoffs by presenting computational complexity comparisons between the traditional algorithms and the proposed compressed NNs. △ Less

Submitted 12 May, 2022; originally announced May 2022.

Online multi-resolution fusion of space-borne multispectral images

Authors: Haoqing Li, Bhavia Duvviri, Ricardo Borsoi, Tales Imbiriba, Edward Beighley, Deniz Erdogmus, Pau Closas

Abstract: Satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena. One important feature of satellite images is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In this paper, we focus on fusing multi-temporal,… ▽ More Satellite imaging has a central role in monitoring, detecting and estimating the intensity of key natural phenomena. One important feature of satellite images is the trade-off between spatial/spectral resolution and their revisiting time, a consequence of design and physical constraints imposed by satellite orbit among other technical limitations. In this paper, we focus on fusing multi-temporal, multi-spectral images where data acquired from different instruments with different spatial resolutions is used. We leverage the spatial relationship between images at multiple modalities to generate high-resolution image sequences at higher revisiting rates. To achieve this goal, we formulate the fusion method as a recursive state estimation problem and study its performance in filtering and smoothing contexts. The proposed strategy clearly outperforms competing methodologies, which is shown in the paper for real data acquired by the Landsat and MODIS instruments. △ Less

Submitted 26 April, 2022; originally announced April 2022.

Comments: Published in IEEE AeroConf 2022

Hybrid Neural Network Augmented Physics-based Models for Nonlinear Filtering

Authors: Tales Imbiriba, Ahmet Demirkaya, Jindřich Duník, Ondřej Straka, Deniz Erdoğmuş, Pau Closas

Abstract: In this paper we present a hybrid neural network augmented physics-based modeling (APBM) framework for Bayesian nonlinear latent space estimation. The proposed APBM strategy allows for model adaptation when new operation conditions come into play or the physics-based model is insufficient (or incomplete) to properly describe the latent phenomenon. One advantage of the APBMs and our estimation proc… ▽ More In this paper we present a hybrid neural network augmented physics-based modeling (APBM) framework for Bayesian nonlinear latent space estimation. The proposed APBM strategy allows for model adaptation when new operation conditions come into play or the physics-based model is insufficient (or incomplete) to properly describe the latent phenomenon. One advantage of the APBMs and our estimation procedure is the capability of maintaining the physical interpretability of estimated states. Furthermore, we propose a constraint filtering approach to control the neural network contributions to the overall model. We also exploit assumed density filtering techniques and cubature integration rules to present a flexible estimation strategy that can easily deal with nonlinear models and high-dimensional latent spaces. Finally, we demonstrate the efficacy of our methodology by leveraging a target tracking scenario with nonlinear and incomplete measurement and acceleration models, respectively. △ Less

Submitted 14 September, 2022; v1 submitted 13 April, 2022; originally announced April 2022.

Cubature Kalman Filter Based Training of Hybrid Differential Equation Recurrent Neural Network Physiological Dynamic Models

Authors: Ahmet Demirkaya, Tales Imbiriba, Kyle Lockwood, Sumientra Rampersad, Elie Alhajjar, Giovanna Guidoboni, Zachary Danziger, Deniz Erdogmus

Abstract: Modeling biological dynamical systems is challenging due to the interdependence of different system components, some of which are not fully understood. To fill existing gaps in our ability to mechanistically model physiological systems, we propose to combine neural networks with physics-based models. Specifically, we demonstrate how we can approximate missing ordinary differential equations (ODEs)… ▽ More Modeling biological dynamical systems is challenging due to the interdependence of different system components, some of which are not fully understood. To fill existing gaps in our ability to mechanistically model physiological systems, we propose to combine neural networks with physics-based models. Specifically, we demonstrate how we can approximate missing ordinary differential equations (ODEs) coupled with known ODEs using Bayesian filtering techniques to train the model parameters and simultaneously estimate dynamic state variables. As a study case we leverage a well-understood model for blood circulation in the human retina and replace one of its core ODEs with a neural network approximation, representing the case where we have incomplete knowledge of the physiological state dynamics. Results demonstrate that state dynamics corresponding to the missing ODEs can be approximated well using a neural network trained using a recursive Bayesian filtering approach in a fashion coupled with the known state dynamic differential equations. This demonstrates that dynamics and impact of missing state variables can be captured through joint state estimation and model parameter estimation within a recursive Bayesian state estimation (RBSE) framework. Results also indicate that this RBSE approach to training the NN parameters yields better outcomes (measurement/state estimation accuracy) than training the neural network with backpropagation through time in the same setting. △ Less

Submitted 12 October, 2021; originally announced October 2021.

Personalized Federated Learning over non-IID Data for Indoor Localization

Authors: Peng Wu, Tales Imbiriba, Junha Park, Sunwoo Kim, Pau Closas

Abstract: Localization and tracking of objects using data-driven methods is a popular topic due to the complexity in characterizing the physics of wireless channel propagation models. In these modeling approaches, data needs to be gathered to accurately train models, at the same time that user's privacy is maintained. An appealing scheme to cooperatively achieve these goals is known as Federated Learning (F… ▽ More Localization and tracking of objects using data-driven methods is a popular topic due to the complexity in characterizing the physics of wireless channel propagation models. In these modeling approaches, data needs to be gathered to accurately train models, at the same time that user's privacy is maintained. An appealing scheme to cooperatively achieve these goals is known as Federated Learning (FL). A challenge in FL schemes is the presence of non-independent and identically distributed (non-IID) data, caused by unevenly exploration of different areas. In this paper, we consider the use of recent FL schemes to train a set of personalized models that are then optimally fused through Bayesian rules, which makes it appropriate in the context of indoor localization. △ Less

Submitted 21 July, 2021; v1 submitted 8 July, 2021; originally announced July 2021.

Model-Based Deep Autoencoder Networks for Nonlinear Hyperspectral Unmixing

Authors: Haoqing Li, Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas, José Carlos Moreira Bermudez, Deniz Erdoğmuş

Abstract: Autoencoder (AEC) networks have recently emerged as a promising approach to perform unsupervised hyperspectral unmixing (HU) by associating the latent representations with the abundances, the decoder with the mixing model and the encoder with its inverse. AECs are especially appealing for nonlinear HU since they lead to unsupervised and model-free algorithms. However, existing approaches fail to e… ▽ More Autoencoder (AEC) networks have recently emerged as a promising approach to perform unsupervised hyperspectral unmixing (HU) by associating the latent representations with the abundances, the decoder with the mixing model and the encoder with its inverse. AECs are especially appealing for nonlinear HU since they lead to unsupervised and model-free algorithms. However, existing approaches fail to explore the fact that the encoder should invert the mixing process, which might reduce their robustness. In this paper, we propose a model-based AEC for nonlinear HU by considering the mixing model a nonlinear fluctuation over a linear mixture. Differently from previous works, we show that this restriction naturally imposes a particular structure to both the encoder and to the decoder networks. This introduces prior information in the AEC without reducing the flexibility of the mixing model. Simulations with synthetic and real data indicate that the proposed strategy improves nonlinear HU. △ Less

Submitted 16 April, 2021; originally announced April 2021.

Fast Unmixing and Change Detection in Multitemporal Hyperspectral Data

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard

Abstract: Multitemporal spectral unmixing (SU) is a powerful tool to process hyperspectral image (HI) sequences due to its ability to reveal the evolution of materials over time and space in a scene. However, significant spectral variability is often observed between collection of images due to variations in acquisition or seasonal conditions. This characteristic has to be considered in the design of SU alg… ▽ More Multitemporal spectral unmixing (SU) is a powerful tool to process hyperspectral image (HI) sequences due to its ability to reveal the evolution of materials over time and space in a scene. However, significant spectral variability is often observed between collection of images due to variations in acquisition or seasonal conditions. This characteristic has to be considered in the design of SU algorithms. Because of its good performance, the multiple endmember spectral mixture analysis algorithm (MESMA) has been recently used to perform SU in multitemporal scenarios arising in several practical applications. However, MESMA does not consider the relationship between the different HIs, and its computational complexity is extremely high for large spectral libraries. In this work, we propose an efficient multitemporal SU method that exploits the high temporal correlation between the abundances to provide more accurate results at a lower computational complexity. We propose to solve the complex general multitemporal SU problem by separately addressing the endmember selection and the abundance estimation problems. This leads to a simpler solution without sacrificing the accuracy of the results. We also propose a strategy to detect and address abrupt abundance variations in time. Theoretical results demonstrate how the proposed method compares to MESMA in terms of quality, and how effective it is in detecting abundance changes. This analysis provides valuable insight into the conditions under which the algorithm succeeds. Simulation results show that the proposed method achieves state-of-the-art performance at a smaller computational cost. △ Less

Submitted 9 September, 2021; v1 submitted 6 April, 2021; originally announced April 2021.

Spectral Variability in Hyperspectral Data Unmixing: A Comprehensive Review

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard, Jocelyn Chanussot, Lucas Drumetz, Jean-Yves Tourneret, Alina Zare, Christian Jutten

Abstract: The spectral signatures of the materials contained in hyperspectral images, also called endmembers (EM), can be significantly affected by variations in atmospheric, illumination or environmental conditions typically occurring within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, what propagates significant mismodeling errors throughout t… ▽ More The spectral signatures of the materials contained in hyperspectral images, also called endmembers (EM), can be significantly affected by variations in atmospheric, illumination or environmental conditions typically occurring within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the endmembers, what propagates significant mismodeling errors throughout the whole unmixing process and compromises the quality of its results. Therefore, large efforts have been recently dedicated to mitigate the effects of spectral variability in SU. This resulted in the development of algorithms that incorporate different strategies to allow the EMs to vary within a hyperspectral image, using, for instance, sets of spectral signatures known a priori, Bayesian, parametric, or local EM models. Each of these approaches has different characteristics and underlying motivations. This paper presents a comprehensive literature review contextualizing both classic and recent approaches to solve this problem. We give a detailed evaluation of the sources of spectral variability and their effect in image spectra. Furthermore, we propose a new taxonomy that organizes existing works according to a practitioner's point of view, based on the necessary amount of supervision and on the computational cost they require. We also review methods used to construct spectral libraries (which are required by many SU techniques) based on the observed hyperspectral image, as well as algorithms for library augmentation and reduction. Finally, we conclude the paper with some discussions and an outline of possible future directions for the field. △ Less

Submitted 6 April, 2021; v1 submitted 20 January, 2020; originally announced January 2020.

Comments: Published at the IEEE Geoscience and Remote Sensing Magazine

Kalman Filtering and Expectation Maximization for Multitemporal Spectral Unmixing

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas, José Carlos Moreira Bermudez, Cédric Richard

Abstract: The recent evolution of hyperspectral imaging technology and the proliferation of new emerging applications presses for the processing of multiple temporal hyperspectral images. In this work, we propose a novel spectral unmixing (SU) strategy using physically motivated parametric endmember representations to account for temporal spectral variability. By representing the multitemporal mixing proces… ▽ More The recent evolution of hyperspectral imaging technology and the proliferation of new emerging applications presses for the processing of multiple temporal hyperspectral images. In this work, we propose a novel spectral unmixing (SU) strategy using physically motivated parametric endmember representations to account for temporal spectral variability. By representing the multitemporal mixing process using a state-space formulation, we are able to exploit the Bayesian filtering machinery to estimate the endmember variability coefficients. Moreover, by assuming that the temporal variability of the abundances is small over short intervals, an efficient implementation of the expectation maximization (EM) algorithm is employed to estimate the abundances and the other model parameters. Simulation results indicate that the proposed strategy outperforms state-of-the-art multitemporal SU algorithms. △ Less

Submitted 12 August, 2020; v1 submitted 2 January, 2020; originally announced January 2020.

A Blind Multiscale Spatial Regularization Framework for Kernel-based Spectral Unmixing

Authors: Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez, Cédric Richard

Abstract: Introducing spatial prior information in hyperspectral imaging (HSI) analysis has led to an overall improvement of the performance of many HSI methods applied for denoising, classification, and unmixing. Extending such methodologies to nonlinear settings is not always straightforward, specially for unmixing problems where the consideration of spatial relationships between neighboring pixels might… ▽ More Introducing spatial prior information in hyperspectral imaging (HSI) analysis has led to an overall improvement of the performance of many HSI methods applied for denoising, classification, and unmixing. Extending such methodologies to nonlinear settings is not always straightforward, specially for unmixing problems where the consideration of spatial relationships between neighboring pixels might comprise intricate interactions between their fractional abundances and nonlinear contributions. In this paper, we consider a multiscale regularization strategy for nonlinear spectral unmixing with kernels. The proposed methodology splits the unmixing problem into two sub-problems at two different spatial scales: a coarse scale containing low-dimensional structures, and the original fine scale. The coarse spatial domain is defined using superpixels that result from a multiscale transformation. Spectral unmixing is then formulated as the solution of quadratically constrained optimization problems, which are solved efficiently by exploring their strong duality and a reformulation of their dual cost functions in the form of root-finding problems. Furthermore, we employ a theory-based statistical framework to devise a consistent strategy to estimate all required parameters, including both the regularization parameters of the algorithm and the number of superpixels of the transformation, resulting in a truly blind (from the parameters setting perspective) unmixing method. Experimental results attest the superior performance of the proposed method when comparing with other, state-of-the-art, related strategies. △ Less

Submitted 2 March, 2020; v1 submitted 19 August, 2019; originally announced August 2019.

Technical Report: A New Decision-Theory-Based Framework for Echo Canceler Control

Authors: Tales Imbiriba, José Carlos M. Bermudez, Jean-Yves Tourneret, Neil J. Bershad

Abstract: A control logic has a central role in many echo cancellation systems for optimizing the performance of adaptive filters while estimating the echo path. For reliable control, accurate double-talk (DT) and channel change (CC) detectors are usually incorporated to the echo canceler. This work expands the usual detection strategy to define a classification problem characterizing four possible states o… ▽ More A control logic has a central role in many echo cancellation systems for optimizing the performance of adaptive filters while estimating the echo path. For reliable control, accurate double-talk (DT) and channel change (CC) detectors are usually incorporated to the echo canceler. This work expands the usual detection strategy to define a classification problem characterizing four possible states of the echo canceler operation. The new formulation allow the use of decision theory to continuously control the transitions among the different modes of operation. The classification rule reduces to a low cost statistics for which it is possible to determine the probability of error under all hypotheses, allowing the classification performance to be accessed analytically. Monte Carlo simulations using synthetic and real data illustrate the reliability of the proposed method. △ Less

Submitted 29 November, 2017; originally announced November 2017.