-
A Bayesian Framework for Clustered Federated Learning
Authors:
Peng Wu,
Tales Imbiriba,
Pau Closas
Abstract:
One of the main challenges of federated learning (FL) is handling non-independent and identically distributed (non-IID) client data, which may occur in practice due to unbalanced datasets and use of different data sources across clients. Knowledge sharing and model personalization are key strategies for addressing this issue. Clustered federated learning is a class of FL methods that groups client…
▽ More
One of the main challenges of federated learning (FL) is handling non-independent and identically distributed (non-IID) client data, which may occur in practice due to unbalanced datasets and use of different data sources across clients. Knowledge sharing and model personalization are key strategies for addressing this issue. Clustered federated learning is a class of FL methods that groups clients that observe similarly distributed data into clusters, such that every client is typically associated with one data distribution and participates in training a model for that distribution along their cluster peers. In this paper, we present a unified Bayesian framework for clustered FL which associates clients to clusters. Then we propose several practical algorithms to handle the, otherwise growing, data associations in a way that trades off performance and computational complexity. This work provides insights on client-cluster associations and enables client knowledge sharing in new ways. The proposed framework circumvents the need for unique client-cluster associations, which is seen to increase the performance of the resulting models in a variety of experiments.
△ Less
Submitted 22 October, 2024; v1 submitted 20 October, 2024;
originally announced October 2024.
-
Continuously Optimizing Radar Placement with Model Predictive Path Integrals
Authors:
Michael Potter,
Shuo Tang,
Paul Ghanem,
Milica Stojanovic,
Pau Closas,
Murat Akcakaya,
Ben Wright,
Marius Necsoiu,
Deniz Erdogmus,
Michael Everett,
Tales Imbiriba
Abstract:
Continuously optimizing sensor placement is essential for precise target localization in various military and civilian applications. While information theory has shown promise in optimizing sensor placement, many studies oversimplify sensor measurement models or neglect dynamic constraints of mobile sensors. To address these challenges, we employ a range measurement model that incorporates radar p…
▽ More
Continuously optimizing sensor placement is essential for precise target localization in various military and civilian applications. While information theory has shown promise in optimizing sensor placement, many studies oversimplify sensor measurement models or neglect dynamic constraints of mobile sensors. To address these challenges, we employ a range measurement model that incorporates radar parameters and radar-target distance, coupled with Model Predictive Path Integral (MPPI) control to manage complex environmental obstacles and dynamic constraints. We compare the proposed approach against stationary radars or simplified range measurement models based on the root mean squared error (RMSE) of the Cubature Kalman Filter (CKF) estimator for the targets' state. Additionally, we visualize the evolving geometry of radars and targets over time, highlighting areas of highest measurement information gain, demonstrating the strengths of the approach. The proposed strategy outperforms stationary radars and simplified range measurement models in target localization, achieving a 38-74% reduction in mean RMSE and a 33-79% reduction in the upper tail of the 90% Highest Density Interval (HDI) over 500 Monte Carl (MC) trials across all time steps.
Code will be made publicly available upon acceptance.
△ Less
Submitted 18 May, 2025; v1 submitted 29 May, 2024;
originally announced May 2024.
-
Bayesian data fusion with shared priors
Authors:
Peng Wu,
Tales Imbiriba,
Victor Elvira,
Pau Closas
Abstract:
The integration of data and knowledge from several sources is known as data fusion. When data is only available in a distributed fashion or when different sensors are used to infer a quantity of interest, data fusion becomes essential. In Bayesian settings, a priori information of the unknown quantities is available and, possibly, present among the different distributed estimators. When the local…
▽ More
The integration of data and knowledge from several sources is known as data fusion. When data is only available in a distributed fashion or when different sensors are used to infer a quantity of interest, data fusion becomes essential. In Bayesian settings, a priori information of the unknown quantities is available and, possibly, present among the different distributed estimators. When the local estimates are fused, the prior knowledge used to construct several local posteriors might be overused unless the fusion node accounts for this and corrects it. In this paper, we analyze the effects of shared priors in Bayesian data fusion contexts. Depending on different common fusion rules, our analysis helps to understand the performance behavior as a function of the number of collaborative agents and as a consequence of different types of priors. The analysis is performed by using two divergences which are common in Bayesian inference, and the generality of the results allows to analyze very generic distributions. These theoretical results are corroborated through experiments in a variety of estimation and classification problems, including linear and nonlinear models, and federated learning schemes.
△ Less
Submitted 8 December, 2023; v1 submitted 14 December, 2022;
originally announced December 2022.
-
Importance Gaussian Quadrature
Authors:
Víctor Elvira,
Luca Martino,
Pau Closas
Abstract:
Importance sampling (IS) and numerical integration methods are usually employed for approximating moments of complicated target distributions. In its basic procedure, the IS methodology randomly draws samples from a proposal distribution and weights them accordingly, accounting for the mismatch between the target and proposal. In this work, we present a general framework of numerical integration t…
▽ More
Importance sampling (IS) and numerical integration methods are usually employed for approximating moments of complicated target distributions. In its basic procedure, the IS methodology randomly draws samples from a proposal distribution and weights them accordingly, accounting for the mismatch between the target and proposal. In this work, we present a general framework of numerical integration techniques inspired by the IS methodology. The framework can also be seen as an incorporation of deterministic rules into IS methods, reducing the error of the estimators by several orders of magnitude in several problems of interest. The proposed approach extends the range of applicability of the Gaussian quadrature rules. For instance, the IS perspective allows us to use Gauss-Hermite rules in problems where the integrand is not involving a Gaussian distribution, and even more, when the integrand can only be evaluated up to a normalizing constant, as it is usually the case in Bayesian inference. The novel perspective makes use of recent advances on the multiple IS (MIS) and adaptive (AIS) literatures, and incorporates it to a wider numerical integration framework that combines several numerical integration rules that can be iteratively adapted. We analyze the convergence of the algorithms and provide some representative examples showing the superiority of the proposed approach in terms of performance.
△ Less
Submitted 31 January, 2021; v1 submitted 9 January, 2020;
originally announced January 2020.
-
Mean Square Error bounds for parameter estimation under model misspecification
Authors:
Adrià Gusi-Amigó,
Pau Closas,
Luc Vandendorpe
Abstract:
In parameter estimation, assumptions about the model are typically considered which allow us to build optimal estimation methods under many statistical senses. However, it is usually the case where such models are inaccurately known or not capturing the complexity of the observed phenomenon. A natural question arises to whether we can find fundamental estimation bounds under model mismatches. This…
▽ More
In parameter estimation, assumptions about the model are typically considered which allow us to build optimal estimation methods under many statistical senses. However, it is usually the case where such models are inaccurately known or not capturing the complexity of the observed phenomenon. A natural question arises to whether we can find fundamental estimation bounds under model mismatches. This paper derives a general bound on the mean square error (MSE) following the Ziv-Zakai methodology for the widely used additive Gaussian model. The general result accounts for erroneous functionals, hyperparameters, and distributions differing from the Gaussian. The result is then particularized to gain some insight into specific problems and some illustrative examples demonstrate the predictive capabilities of the bound.
△ Less
Submitted 11 December, 2015; v1 submitted 12 November, 2015;
originally announced November 2015.
-
Sequential estimation of intrinsic activity and synaptic input in single neurons by particle filtering with optimal importance density
Authors:
Pau Closas,
Antoni Guillamon
Abstract:
This paper deals with the problem of inferring the signals and parameters that cause neural activity to occur. The ultimate challenge being to unveil brain's connectivity, here we focus on a microscopic vision of the problem, where single neurons (potentially connected to a network of peers) are at the core of our study. The sole observation available are noisy, sampled voltage traces obtained fro…
▽ More
This paper deals with the problem of inferring the signals and parameters that cause neural activity to occur. The ultimate challenge being to unveil brain's connectivity, here we focus on a microscopic vision of the problem, where single neurons (potentially connected to a network of peers) are at the core of our study. The sole observation available are noisy, sampled voltage traces obtained from intracellular recordings. We design algorithms and inference methods using the tools provided by stochastic filtering, that allow a probabilistic interpretation and treatment of the problem. Using particle filtering we are able to reconstruct traces of voltages and estimate the time course of auxiliary variables. By extending the algorithm, through PMCMC methodology, we are able to estimate hidden physiological parameters as well, like intrinsic conductances or reversal potentials. Last, but not least, the method is applied to estimate synaptic conductances arriving at a target cell, thus reconstructing the synaptic excitatory/inhibitory input traces. Notably, these estimations have a bound-achieving performance even in spiking regimes.
△ Less
Submitted 12 November, 2015;
originally announced November 2015.
