-
Joint Signal Recovery and Uncertainty Quantification via the Residual Prior Transform
Authors:
Yao Xiao,
Anne Gelb
Abstract:
Conventional priors used for signal recovery are often limited by the assumption that the type of a signal's variability, such as piecewise constant or linear behavior, is known and fixed. This assumption is problematic for complex signals that exhibit different behaviors across the domain. The recently developed {\em residual transform operator} effectively reduces such variability-dependent erro…
▽ More
Conventional priors used for signal recovery are often limited by the assumption that the type of a signal's variability, such as piecewise constant or linear behavior, is known and fixed. This assumption is problematic for complex signals that exhibit different behaviors across the domain. The recently developed {\em residual transform operator} effectively reduces such variability-dependent error within the LASSO regression framework. Importantly, it does not require prior information regarding structure of the underlying signal. This paper reformulates the residual transform operator into a new prior within a hierarchical Bayesian framework. In so doing, it unlocks two powerful new capabilities. First, it enables principled uncertainty quantification, providing robust credible intervals for the recovered signal, and second, it provides a natural framework for the joint recovery of signals from multimodal measurements by coherently fusing information from disparate data sources. Numerical experiments demonstrate that the residual prior yields high-fidelity signal and image recovery from multimodal data while providing robust uncertainty quantification.
△ Less
Submitted 22 October, 2025;
originally announced October 2025.
-
Non-intrusive structural-preserving sequential data assimilation
Authors:
Lizuo Liu,
Tongtong Li,
Anne Gelb
Abstract:
Data assimilation (DA) methods combine model predictions with observational data to improve state estimation in dynamical systems, inspiring their increasingly prominent role in geophysical and climate applications. Classical DA methods assume that the governing equations modeling the dynamics are known, which is unlikely for most real world applications. Machine learning (ML) provides a flexible…
▽ More
Data assimilation (DA) methods combine model predictions with observational data to improve state estimation in dynamical systems, inspiring their increasingly prominent role in geophysical and climate applications. Classical DA methods assume that the governing equations modeling the dynamics are known, which is unlikely for most real world applications. Machine learning (ML) provides a flexible alternative by learning surrogate models directly from data, but standard ML methods struggle in noisy and data-scarce environments, where meaningful extrapolation requires incorporating physical constraints. Recent advances in structure-preserving ML architectures, such as the development of the entropy-stable conservative flux form network (ESCFN), highlight the critical role of physical structure in improving learning stability and accuracy for unknown systems of conservation laws. Structural information has also been shown to improve DA performance. Gradient-based measures of spatial variability, in particular, can help refine ensemble updates in discontinuous systems. Motivated by both of these recent innovations, this investigation proposes a new non-intrusive, structure-preserving sequential data assimilation (NSSDA) framework that leverages structure at both the forecast and analysis stages. We use the ESCFN to construct a surrogate model to preserve physical laws during forecasting, and a structurally informed ensemble transform Kalman filter (SETKF) to embed local statistical structure into the assimilation step. Our method operates in a highly constrained environment, using only a single noisy trajectory for both training and assimilation. Numerical experiments where the unknown dynamics correspond respectively to the shallow water and Euler equations demonstrate significantly improved predictive accuracy.
△ Less
Submitted 22 October, 2025;
originally announced October 2025.
-
Structurally informed data assimilation in two dimensions
Authors:
Tongtong Li,
Anne Gelb,
Yoonsang Lee
Abstract:
Accurate data assimilation (DA) for systems with piecewise-smooth or discontinuous state variables remains a significant challenge, as conventional covariance-based ensemble Kalman filter approaches often fail to effectively balance observations and model information near sharp features. In this paper we develop a structurally informed DA framework using ensemble transform Kalman filtering (ETKF).…
▽ More
Accurate data assimilation (DA) for systems with piecewise-smooth or discontinuous state variables remains a significant challenge, as conventional covariance-based ensemble Kalman filter approaches often fail to effectively balance observations and model information near sharp features. In this paper we develop a structurally informed DA framework using ensemble transform Kalman filtering (ETKF). Our approach introduces gradient-based weighting matrices constructed from finite difference statistics of the forecast ensemble, thereby allowing the assimilation process to dynamically adjust the influence of observations and prior estimates according to local roughness. The design is intentionally flexible so that it can be suitably refined for sparse data environments. Numerical experiments demonstrate that our new structurally informed data assimilation framework consistently yields greater accuracy when compared to more conventional approaches.
△ Less
Submitted 7 October, 2025;
originally announced October 2025.
-
Neural Entropy-stable conservative flux form neural networks for learning hyperbolic conservation laws
Authors:
Lizuo Liu,
Lu Zhang,
Anne Gelb
Abstract:
We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior…
▽ More
We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior knowledge of the governing equations or assume a fixed discretization. Our approach removes this dependency by embedding entropy-stable design principles into the learning process itself, enabling the discovery of physically consistent dynamics in a fully data-driven setting. By jointly learning both the numerical flux function and a corresponding entropy, the proposed method ensures conservation and entropy dissipation, critical for long-term stability and fidelity in the system of hyperbolic conservation laws. Numerical results demonstrate that the method achieves stability and conservation over extended time horizons and accurately captures shock propagation speeds, even without oracle access to future-time solution profiles in the training data.
△ Less
Submitted 2 July, 2025;
originally announced July 2025.
-
A new sparsity promoting residual transform operator for Lasso regression
Authors:
Yao Xiao,
Anne Gelb,
Aditya Viswanathan
Abstract:
Lasso regression is a widely employed approach within the $\ell_1$ regularization framework used to promote sparsity and recover piecewise smooth signals $f:[a,b) \rightarrow \mathbb{R}$ when the given observations are obtained from noisy, blurred, and/or incomplete data environments. In choosing the regularizing sparsity-promoting operator, it is assumed that the particular type of variability of…
▽ More
Lasso regression is a widely employed approach within the $\ell_1$ regularization framework used to promote sparsity and recover piecewise smooth signals $f:[a,b) \rightarrow \mathbb{R}$ when the given observations are obtained from noisy, blurred, and/or incomplete data environments. In choosing the regularizing sparsity-promoting operator, it is assumed that the particular type of variability of the underlying signal, for example, piecewise constant or piecewise linear behavior across the entire domain, is both known and fixed. Such an assumption is problematic in more general cases, e.g.~when a signal exhibits piecewise oscillatory behavior with varying wavelengths and magnitudes. To address the limitations of assuming a fixed (and typically low order) variability when choosing a sparsity-promoting operator, this investigation proposes a novel residual transform operator that can be used within the Lasso regression formulation. In a nutshell, the idea is that for a general piecewise smooth signal $f$, it is possible to design two operators $\mathcal L_1$ and $\mathcal L_2$ such that $\mathcal L_1{\boldsymbol f} \approx \mathcal L_2{\boldsymbol f}$, where ${\boldsymbol f} \in \mathbb{R}^n$ is a discretized approximation of $f$, but $\mathcal L_1 \not\approx \mathcal L_2$. The corresponding residual transform operator, $\mathcal L = \mathcal L_1- \mathcal L_2$, yields a result that (1) effectively reduces the variability dependent error that occurs when applying either $\mathcal L_1$ or $\mathcal L_2$ to ${\boldsymbol f}$, a property that holds even when $\mathcal L_1{\boldsymbol f} \approx \mathcal L_2{\boldsymbol f}$ is not a good approximation to the true sparse domain vector of ${\boldsymbol f}$, and (2) does not require $\mathcal L_1$ or $\mathcal L_2$ to have prior information regarding the variability of the underlying signal.
△ Less
Submitted 27 June, 2025;
originally announced June 2025.
-
Entropy stable conservative flux form neural networks
Authors:
Lizuo Liu,
Tongtong Li,
Anne Gelb,
Yoonsang Lee
Abstract:
We propose an entropy-stable conservative flux form neural network (CFN) that integrates classical numerical conservation laws into a data-driven framework using the entropy-stable, second-order, and non-oscillatory Kurganov-Tadmor (KT) scheme. The proposed entropy-stable CFN uses slope limiting as a denoising mechanism, ensuring accurate predictions in both noisy and sparse observation environmen…
▽ More
We propose an entropy-stable conservative flux form neural network (CFN) that integrates classical numerical conservation laws into a data-driven framework using the entropy-stable, second-order, and non-oscillatory Kurganov-Tadmor (KT) scheme. The proposed entropy-stable CFN uses slope limiting as a denoising mechanism, ensuring accurate predictions in both noisy and sparse observation environments, as well as in both smooth and discontinuous regions. Numerical experiments demonstrate that the entropy-stable CFN achieves both stability and conservation while maintaining accuracy over extended time domains. Furthermore, it successfully predicts shock propagation speeds in long-term simulations, {\it without} oracle knowledge of later-time profiles in the training data.
△ Less
Submitted 8 July, 2025; v1 submitted 3 November, 2024;
originally announced November 2024.
-
A Bayesian framework for spectral reprojection
Authors:
Tongtong Li,
Anne Gelb
Abstract:
Fourier partial sum approximations yield exponential accuracy for smooth and periodic functions, but produce the infamous Gibbs phenomenon for non-periodic ones. Spectral reprojection resolves the Gibbs phenomenon by projecting the Fourier partial sum onto a Gibbs complementary basis, often prescribed as the Gegenbauer polynomials. Noise in the Fourier data and the Runge phenomenon both degrade th…
▽ More
Fourier partial sum approximations yield exponential accuracy for smooth and periodic functions, but produce the infamous Gibbs phenomenon for non-periodic ones. Spectral reprojection resolves the Gibbs phenomenon by projecting the Fourier partial sum onto a Gibbs complementary basis, often prescribed as the Gegenbauer polynomials. Noise in the Fourier data and the Runge phenomenon both degrade the quality of the Gegenbauer reconstruction solution, however. Motivated by its theoretical convergence properties, this paper proposes a new Bayesian framework for spectral reprojection, which allows a greater understanding of the impact of noise on the reprojection method from a statistical point of view. We are also able to improve the robustness with respect to the Gegenbauer polynomials parameters. Finally, the framework provides a mechanism to quantify the uncertainty of the solution estimate.
△ Less
Submitted 22 June, 2024;
originally announced June 2024.
-
Complex-Valued Signal Recovery using the Bayesian LASSO
Authors:
Dylan Green,
Jonathan Lindbloom,
Anne Gelb
Abstract:
Recovering complex-valued image recovery from noisy indirect data is important in applications such as ultrasound imaging and synthetic aperture radar. While there are many effective algorithms to recover point estimates of the magnitude, fewer are designed to recover the phase. Quantifying uncertainty in the estimate can also provide valuable information for real-time decision making. This invest…
▽ More
Recovering complex-valued image recovery from noisy indirect data is important in applications such as ultrasound imaging and synthetic aperture radar. While there are many effective algorithms to recover point estimates of the magnitude, fewer are designed to recover the phase. Quantifying uncertainty in the estimate can also provide valuable information for real-time decision making. This investigation therefore proposes a new Bayesian inference method that recovers point estimates while also quantifying the uncertainty for complex-valued signals or images given noisy and indirect observation data. Our method is motivated by the Bayesian LASSO approach for real-valued sparse signals, and here we demonstrate that the Bayesian LASSO can be effectively adapted to recover complex-valued images whose magnitude is sparse in some (e.g.~the gradient) domain. Numerical examples demonstrate our algorithm's robustness to noise as well as its computational efficiency.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Generalized sparsity-promoting solvers for Bayesian inverse problems: Versatile sparsifying transforms and unknown noise variances
Authors:
Jonathan Lindbloom,
Jan Glaubitz,
Anne Gelb
Abstract:
Bayesian hierarchical models can provide efficient algorithms for finding sparse solutions to ill-posed inverse problems. The models typically comprise a conditionally Gaussian prior model for the unknown which is augmented by a generalized gamma hyper-prior model for variance hyper-parameters. This investigation generalizes these models and their efficient maximum a posterior (MAP) estimation usi…
▽ More
Bayesian hierarchical models can provide efficient algorithms for finding sparse solutions to ill-posed inverse problems. The models typically comprise a conditionally Gaussian prior model for the unknown which is augmented by a generalized gamma hyper-prior model for variance hyper-parameters. This investigation generalizes these models and their efficient maximum a posterior (MAP) estimation using the iterative alternating sequential (IAS) algorithm in two ways: (1) General sparsifying transforms: Diverging from conventional methods, our approach permits the use of sparsifying transformations with nontrivial kernels; (2) Unknown noise variances: We treat the noise variance as a random variable that is estimated during the inference procedure. This is important in applications where the noise estimate cannot be accurately estimated a priori. Remarkably, these augmentations neither significantly burden the computational expense of the algorithm nor compromise its efficacy. We include convexity and convergence analysis for the method and demonstrate its efficacy in several numerical experiments.
△ Less
Submitted 7 January, 2025; v1 submitted 26 February, 2024;
originally announced February 2024.
-
Learning In-between Imagery Dynamics via Physical Latent Spaces
Authors:
Jihun Han,
Yoonsang Lee,
Anne Gelb
Abstract:
We present a framework designed to learn the underlying dynamics between two images observed at consecutive time steps. The complex nature of image data and the lack of temporal information pose significant challenges in capturing the unique evolving patterns. Our proposed method focuses on estimating the intermediary stages of image evolution, allowing for interpretability through latent dynamics…
▽ More
We present a framework designed to learn the underlying dynamics between two images observed at consecutive time steps. The complex nature of image data and the lack of temporal information pose significant challenges in capturing the unique evolving patterns. Our proposed method focuses on estimating the intermediary stages of image evolution, allowing for interpretability through latent dynamics while preserving spatial correlations with the image. By incorporating a latent variable that follows a physical model expressed in partial differential equations (PDEs), our approach ensures the interpretability of the learned model and provides insight into corresponding image dynamics. We demonstrate the robustness and effectiveness of our learning framework through a series of numerical tests using geoscientific imagery data.
△ Less
Submitted 14 October, 2023;
originally announced October 2023.
-
A Structurally Informed Data Assimilation Approach for Nonlinear Partial Differential Equations
Authors:
Tongtong Li,
Anne Gelb,
Yoonsang Lee
Abstract:
Ensemble transform Kalman filtering (ETKF) data assimilation is often used to combine available observations with numerical simulations to obtain statistically accurate and reliable state representations in dynamical systems. However, it is well known that the commonly used Gaussian distribution assumption introduces biases for state variables that admit discontinuous profiles, which are prevalent…
▽ More
Ensemble transform Kalman filtering (ETKF) data assimilation is often used to combine available observations with numerical simulations to obtain statistically accurate and reliable state representations in dynamical systems. However, it is well known that the commonly used Gaussian distribution assumption introduces biases for state variables that admit discontinuous profiles, which are prevalent in nonlinear partial differential equations. This investigation designs a new structurally informed non-Gaussian prior that exploits statistical information from the simulated state variables. In particular, we construct a new weighting matrix based on the second moment of the gradient information of the state variable to replace the prior covariance matrix used for model/data compromise in the ETKF data assimilation framework. We further adapt our weighting matrix to include information in discontinuity regions via a clustering technique. Our numerical experiments demonstrate that this new approach yields more accurate estimates than those obtained using ETKF on shallow water equations, even when ETKF is enhanced with inflation and localization techniques.
△ Less
Submitted 5 March, 2024; v1 submitted 5 September, 2023;
originally announced September 2023.
-
Leveraging joint sparsity in hierarchical Bayesian learning
Authors:
Jan Glaubitz,
Anne Gelb
Abstract:
We present a hierarchical Bayesian learning approach to infer jointly sparse parameter vectors from multiple measurement vectors. Our model uses separate conditionally Gaussian priors for each parameter vector and common gamma-distributed hyper-parameters to enforce joint sparsity. The resulting joint-sparsity-promoting priors are combined with existing Bayesian inference methods to generate a new…
▽ More
We present a hierarchical Bayesian learning approach to infer jointly sparse parameter vectors from multiple measurement vectors. Our model uses separate conditionally Gaussian priors for each parameter vector and common gamma-distributed hyper-parameters to enforce joint sparsity. The resulting joint-sparsity-promoting priors are combined with existing Bayesian inference methods to generate a new family of algorithms. Our numerical experiments, which include a multi-coil magnetic resonance imaging application, demonstrate that our new approach consistently outperforms commonly used hierarchical Bayesian methods.
△ Less
Submitted 24 May, 2024; v1 submitted 29 March, 2023;
originally announced March 2023.
-
Sequential edge detection using joint hierarchical Bayesian learning
Authors:
Yao Xiao,
Anne Gelb,
Guohui Song
Abstract:
This paper introduces a new sparse Bayesian learning (SBL) algorithm that jointly recovers a temporal sequence of edge maps from noisy and under-sampled Fourier data. The new method is cast in a Bayesian framework and uses a prior that simultaneously incorporates intra-image information to promote sparsity in each individual edge map with inter-image information to promote similarities in any unch…
▽ More
This paper introduces a new sparse Bayesian learning (SBL) algorithm that jointly recovers a temporal sequence of edge maps from noisy and under-sampled Fourier data. The new method is cast in a Bayesian framework and uses a prior that simultaneously incorporates intra-image information to promote sparsity in each individual edge map with inter-image information to promote similarities in any unchanged regions. By treating both the edges as well as the similarity between adjacent images as random variables, there is no need to separately form regions of change. Thus we avoid both additional computational cost as well as any information loss resulting from pre-processing the image. Our numerical examples demonstrate that our new method compares favorably with more standard SBL approaches.
△ Less
Submitted 27 February, 2023;
originally announced February 2023.
-
Designing Neural Networks for Hyperbolic Conservation Laws
Authors:
Zhen Chen,
Anne Gelb,
Yoonsang Lee
Abstract:
We propose a new data-driven method to learn the dynamics of an unknown hyperbolic system of conservation laws using deep neural networks. Inspired by classical methods in numerical conservation laws, we develop a new conservative form network (CFN) in which the network learns the flux function of the unknown system. Our numerical examples demonstrate that the CFN yields significantly better predi…
▽ More
We propose a new data-driven method to learn the dynamics of an unknown hyperbolic system of conservation laws using deep neural networks. Inspired by classical methods in numerical conservation laws, we develop a new conservative form network (CFN) in which the network learns the flux function of the unknown system. Our numerical examples demonstrate that the CFN yields significantly better prediction accuracy than what is obtained using a standard non-conservative form network, even when it is enhanced with constraints to promote conservation. In particular, solutions obtained using the CFN consistently capture the correct shock propagation speed without introducing non-physical oscillations into the solution. They are furthermore robust to noisy and sparse observation environments.
△ Less
Submitted 25 November, 2022;
originally announced November 2022.
-
Sub-aperture SAR Imaging with Uncertainty Quantification
Authors:
Victor Churchill,
Anne Gelb
Abstract:
In the problem of spotlight mode airborne synthetic aperture radar (SAR) image formation, it is well-known that data collected over a wide azimuthal angle violate the isotropic scattering property typically assumed. Many techniques have been proposed to account for this issue, including both full-aperture and sub-aperture methods based on filtering, regularized least squares, and Bayesian methods.…
▽ More
In the problem of spotlight mode airborne synthetic aperture radar (SAR) image formation, it is well-known that data collected over a wide azimuthal angle violate the isotropic scattering property typically assumed. Many techniques have been proposed to account for this issue, including both full-aperture and sub-aperture methods based on filtering, regularized least squares, and Bayesian methods. A full-aperture method that uses a hierarchical Bayesian prior to incorporate appropriate speckle modeling and reduction was recently introduced to produce samples of the posterior density rather than a single image estimate. This uncertainty quantification information is more robust as it can generate a variety of statistics for the scene. As proposed, the method was not well-suited for large problems, however, as the sampling was inefficient. Moreover, the method was not explicitly designed to mitigate the effects of the faulty isotropic scattering assumption. In this work we therefore propose a new sub-aperture SAR imaging method that uses a sparse Bayesian learning-type algorithm to more efficiently produce approximate posterior densities for each sub-aperture window. These estimates may be useful in and of themselves, or when of interest, the statistics from these distributions can be combined to form a composite image. Furthermore, unlike the often-employed lp-regularized least squares methods, no user-defined parameters are required. Application-specific adjustments are made to reduce the typically burdensome runtime and storage requirements so that appropriately large images can be generated. Finally, this paper focuses on incorporating these techniques into SAR image formation process. That is, for the problem starting with SAR phase history data, so that no additional processing errors are incurred.
△ Less
Submitted 25 August, 2022;
originally announced August 2022.
-
Improving numerical accuracy for the viscous-plastic formulation of sea ice
Authors:
Tongtong Li,
Anne Gelb,
Yoonsang Lee
Abstract:
Accurate modeling of sea ice dynamics is critical for predicting environmental variables and is important in applications such as navigating ice breaker ships. Research for both modeling and simulating sea ice dynamics is ongoing, with the most widely accepted model based on the viscous-plastic (VP) formulation introduced by Hibler in 1979. Due to its highly nonlinear features, this model is intri…
▽ More
Accurate modeling of sea ice dynamics is critical for predicting environmental variables and is important in applications such as navigating ice breaker ships. Research for both modeling and simulating sea ice dynamics is ongoing, with the most widely accepted model based on the viscous-plastic (VP) formulation introduced by Hibler in 1979. Due to its highly nonlinear features, this model is intrinsically challenging for computational solvers. In particular, sea ice simulations often significantly differ from satellite observations. This study therefore focuses on improving the numerical accuracy of the VP sea ice model. Since the poor convergence observed in existing numerical simulations stems from the nonlinear nature of the VP formulation, this investigation proposes using the celebrated weighted essentially non-oscillatory (WENO) scheme -- as opposed to the frequently employed centered difference (CD) scheme -- for the spatial derivatives in the VP sea ice model. We then proceed to numerically demonstrate that WENO yields higher-order convergence for smooth solutions, and that furthermore it is able to resolve the discontinuities in the sharp features of sea ice covers -- something that is not possible using CD methods. Finally, our proposed framework integrates a potential function method that utilizes the phase field method to naturally incorporates the physical restrictions of ice thickness and ice concentration in transport equations, resulting in a modified transport equations which includes additional forcing terms. Our method does not require post-processing, thereby avoiding the possible introduction of discontinuities and corresponding negative impact on the solution behavior. Numerical experiments are provided to demonstrate the efficacy of our new methodology.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
Sequential image recovery from noisy and under-sampled Fourier data
Authors:
Yao Xiao,
Jan Glaubitz,
Anne Gelb,
Guohui Song
Abstract:
A new algorithm is developed to jointly recover a temporal sequence of images from noisy and under-sampled Fourier data. Specifically, we consider the case where each data set is missing vital information that prevents its (individual) accurate recovery. Our new method is designed to restore the missing information in each individual image by "borrowing" it from the other images in the sequence. A…
▽ More
A new algorithm is developed to jointly recover a temporal sequence of images from noisy and under-sampled Fourier data. Specifically, we consider the case where each data set is missing vital information that prevents its (individual) accurate recovery. Our new method is designed to restore the missing information in each individual image by "borrowing" it from the other images in the sequence. As a result, {\em all} of the individual reconstructions yield improved accuracy. The use of high resolution Fourier edge detection methods is essential to our algorithm. In particular, edge information is obtained directly from the Fourier data which leads to an accurate coupling term between data sets. Moreover, data loss is largely avoided as coarse reconstructions are not required to process inter- and intra-image information. Numerical examples are provided to demonstrate the accuracy, efficiency and robustness of our new method.
△ Less
Submitted 11 May, 2022; v1 submitted 28 March, 2022;
originally announced March 2022.
-
Generalized sparse Bayesian learning and application to image reconstruction
Authors:
Jan Glaubitz,
Anne Gelb,
Guohui Song
Abstract:
Image reconstruction based on indirect, noisy, or incomplete data remains an important yet challenging task. While methods such as compressive sensing have demonstrated high-resolution image recovery in various settings, there remain issues of robustness due to parameter tuning. Moreover, since the recovery is limited to a point estimate, it is impossible to quantify the uncertainty, which is ofte…
▽ More
Image reconstruction based on indirect, noisy, or incomplete data remains an important yet challenging task. While methods such as compressive sensing have demonstrated high-resolution image recovery in various settings, there remain issues of robustness due to parameter tuning. Moreover, since the recovery is limited to a point estimate, it is impossible to quantify the uncertainty, which is often desirable. Due to these inherent limitations, a sparse Bayesian learning approach is sometimes adopted to recover a posterior distribution of the unknown. Sparse Bayesian learning assumes that some linear transformation of the unknown is sparse. However, most of the methods developed are tailored to specific problems, with particular forward models and priors. Here, we present a generalized approach to sparse Bayesian learning. It has the advantage that it can be used for various types of data acquisitions and prior information. Some preliminary results on image reconstruction/recovery indicate its potential use for denoising, deblurring, and magnetic resonance imaging.
△ Less
Submitted 27 January, 2023; v1 submitted 18 January, 2022;
originally announced January 2022.
-
Empirical Bayesian Inference using Joint Sparsity
Authors:
Jiahui Zhang,
Anne Gelb,
Theresa Scarnati
Abstract:
This paper develops a new empirical Bayesian inference algorithm for solving a linear inverse problem given multiple measurement vectors (MMV) of under-sampled and noisy observable data. Specifically, by exploiting the joint sparsity across the multiple measurements in the sparse domain of the underlying signal or image, we construct a new support informed sparsity promoting prior. Several applica…
▽ More
This paper develops a new empirical Bayesian inference algorithm for solving a linear inverse problem given multiple measurement vectors (MMV) of under-sampled and noisy observable data. Specifically, by exploiting the joint sparsity across the multiple measurements in the sparse domain of the underlying signal or image, we construct a new support informed sparsity promoting prior. Several applications can be modeled using this framework, and as a prototypical example we consider reconstructing an image from synthetic aperture radar (SAR) observations using nearby azimuth angles. Our numerical experiments demonstrate that using this new prior not only improves accuracy of the recovery, but also reduces the uncertainty in the posterior when compared to standard sparsity producing priors.
△ Less
Submitted 29 March, 2021;
originally announced March 2021.
-
Stabilizing Radial Basis Function Methods for Conservation Laws Using Weakly Enforced Boundary Conditions
Authors:
Jan Glaubitz,
Anne Gelb
Abstract:
It is well understood that boundary conditions (BCs) may cause global radial basis function (RBF) methods to become unstable for hyperbolic conservation laws (CLs). Here we investigate this phenomenon and identify the strong enforcement of BCs as the mechanism triggering such stability issues. Based on this observation we propose a technique to weakly enforce BCs in RBF methods. In the case of hyp…
▽ More
It is well understood that boundary conditions (BCs) may cause global radial basis function (RBF) methods to become unstable for hyperbolic conservation laws (CLs). Here we investigate this phenomenon and identify the strong enforcement of BCs as the mechanism triggering such stability issues. Based on this observation we propose a technique to weakly enforce BCs in RBF methods. In the case of hyperbolic CLs, this is achieved by carefully building RBF methods from the weak form of the CL, rather than the typically enforced strong form. Furthermore, we demonstrate that global RBF methods may violate conservation, yielding physically unreasonable solutions when the approximation does not take into account these considerations. Numerical experiments validate our theoretical results.
△ Less
Submitted 1 February, 2021;
originally announced February 2021.
-
Estimation and uncertainty quantification for piecewise smooth signal recovery
Authors:
Victor Churchill,
Anne Gelb
Abstract:
This paper presents a sparse Bayesian learning (SBL) algorithm for linear inverse problems with a high order total variation (HOTV) sparsity prior. For the problem of sparse signal recovery, SBL often produces more accurate estimates than maximum a posteriori estimates, including those that rely on l1 regularization. Moreover, rather than a single signal estimate, SBL yields a full posterior densi…
▽ More
This paper presents a sparse Bayesian learning (SBL) algorithm for linear inverse problems with a high order total variation (HOTV) sparsity prior. For the problem of sparse signal recovery, SBL often produces more accurate estimates than maximum a posteriori estimates, including those that rely on l1 regularization. Moreover, rather than a single signal estimate, SBL yields a full posterior density estimate which can be used for uncertainty quantification. However, SBL is only immediately applicable to problems having a direct sparsity prior, or to those that can be formed via synthesis. This paper demonstrates how a problem with an HOTV sparsity prior can be formulated via synthesis, and then develops a corresponding Bayesian learning method. This expands the class of problems available to Bayesian learning to include, e.g., inverse problems dealing with the recovery of piecewise smooth functions or signals from data. Numerical examples are provided to demonstrate how this new technique is effectively employed.
△ Less
Submitted 17 July, 2020;
originally announced July 2020.
-
Synthetic Aperture Radar Image Formation with Uncertainty Quantification
Authors:
Victor Churchill,
Anne Gelb
Abstract:
Synthetic aperture radar (SAR) is a day or night any-weather imaging modality that is an important tool in remote sensing. Most existing SAR image formation methods result in a maximum a posteriori image which approximates the reflectivity of an unknown ground scene. This single image provides no quantification of the certainty with which the features in the estimate should be trusted. In addition…
▽ More
Synthetic aperture radar (SAR) is a day or night any-weather imaging modality that is an important tool in remote sensing. Most existing SAR image formation methods result in a maximum a posteriori image which approximates the reflectivity of an unknown ground scene. This single image provides no quantification of the certainty with which the features in the estimate should be trusted. In addition, finding the mode is generally not the best way to interrogate a posterior. This paper addresses these issues by introducing a sampling framework to SAR image formation. A hierarchical Bayesian model is constructed using conjugate priors that directly incorporate coherent imaging and the problematic speckle phenomenon which is known to degrade image quality. Samples of the resulting posterior as well as parameters governing speckle and noise are obtained using a Gibbs sampler. These samples may then be used to compute estimates, and also to derive other statistics like variance which aid in uncertainty quantification. The latter information is particularly important in SAR, where ground truth images even for synthetically-created examples are typically unknown. An example result using real-world data shows that the sampling-based approach introduced here to SAR image formation provides parameter-free estimates with improved contrast and significantly reduced speckle, as well as unprecedented uncertainty quantification information.
△ Less
Submitted 13 July, 2020;
originally announced July 2020.
-
Thresholded Non-Uniform Fourier Frame-Based Reconstruction for Stripmap SAR
Authors:
John McKay,
Anne Gelb,
Suren Jayasuriya,
Vishal Monga
Abstract:
Fourier domain methods are fast algorithms for SAR imaging. They typically involve an interpolation in the frequency domain to re-grid non-uniform data so inverse fast Fourier transforms can be performed. In this paper, we apply a frame reconstruction algorithm, extending the non-uniform fast Fourier transform, to stripmap SAR data. Further, we present an improved thresholded frame reconstruction…
▽ More
Fourier domain methods are fast algorithms for SAR imaging. They typically involve an interpolation in the frequency domain to re-grid non-uniform data so inverse fast Fourier transforms can be performed. In this paper, we apply a frame reconstruction algorithm, extending the non-uniform fast Fourier transform, to stripmap SAR data. Further, we present an improved thresholded frame reconstruction algorithm for robust performance and improved computational efficiency. We demonstrate compelling results on real stripmap SAR data.
△ Less
Submitted 24 October, 2019;
originally announced October 2019.
-
Accelerated Variance Based Joint Sparsity Recovery of Images from Fourier Data
Authors:
Theresa Scarnati,
Anne Gelb
Abstract:
Several problems in imaging acquire multiple measurement vectors (MMVs) of Fourier samples for the same underlying scene. Image recovery techniques from MMVs aim to exploit the joint sparsity across the measurements in the sparse domain. This is typically accomplished by extending the use of $\ell_1$ regularization of the sparse domain in the single measurement vector (SMV) case to using…
▽ More
Several problems in imaging acquire multiple measurement vectors (MMVs) of Fourier samples for the same underlying scene. Image recovery techniques from MMVs aim to exploit the joint sparsity across the measurements in the sparse domain. This is typically accomplished by extending the use of $\ell_1$ regularization of the sparse domain in the single measurement vector (SMV) case to using $\ell_{2,1}$ regularization so that the "jointness" can be accounted for. Although effective, the approach is inherently coupled and therefore computationally inefficient. The method also does not consider current approaches in the SMV case that use spatially varying weighted $\ell_1$ regularization term. The recently introduced variance based joint sparsity (VBJS) recovery method uses the variance across the measurements in the sparse domain to produce a weighted MMV method that is more accurate and more efficient than the standard $\ell_{2,1}$ approach. The efficiency is due to the decoupling of the measurement vectors, with the increased accuracy resulting from the spatially varying weight. This paper introduces the accelerated VBJS which reduces computational cost even further by eliminating the requirement to first approximate the underlying image in order to construct the weights. Eliminating this preprocessing step moreover reduces the amount of information lost from the data, so that our method is more accurate. Numerical examples provided in the paper verify these benefits.
△ Less
Submitted 18 October, 2019;
originally announced October 2019.
-
Total Variation Bayesian Learning via Synthesis
Authors:
Victor Churchill,
Anne Gelb
Abstract:
This paper presents a sparse Bayesian learning algorithm for inverse problems in signal and image processing with a total variation (TV) sparsity prior. Because of the prior used, and the fact that the prior parameters are estimated directly from the data, sparse Bayesian learning often produces more accurate results than the typical maximum a posteriori Bayesian estimates for sparse signal recove…
▽ More
This paper presents a sparse Bayesian learning algorithm for inverse problems in signal and image processing with a total variation (TV) sparsity prior. Because of the prior used, and the fact that the prior parameters are estimated directly from the data, sparse Bayesian learning often produces more accurate results than the typical maximum a posteriori Bayesian estimates for sparse signal recovery. It also provides a full posterior distribution. However, sparse Bayesian learning is only available to problems with a direct sparsity prior or those formed via synthesis. This paper demonstrates how a problem with a TV sparsity prior can be formulated in a synthesis approach. We then develop a method that combines this synthesis-based TV with the sparse Bayesian learning algorithm and provide numerical examples to demonstrate how our new technique is effectively employed.
△ Less
Submitted 3 May, 2019;
originally announced May 2019.
-
High Order Edge Sensors with $\ell^1$ Regularization for Enhanced Discontinuous Galerkin Methods
Authors:
Jan Glaubitz,
Anne Gelb
Abstract:
This paper investigates the use of $\ell^1$ regularization for solving hyperbolic conservation laws based on high order discontinuous Galerkin (DG) approximations. We first use the polynomial annihilation method to construct a high order edge sensor which enables us to flag troubled elements. The DG approximation is enhanced in these troubled regions by activating $\ell^1$ regularization to promot…
▽ More
This paper investigates the use of $\ell^1$ regularization for solving hyperbolic conservation laws based on high order discontinuous Galerkin (DG) approximations. We first use the polynomial annihilation method to construct a high order edge sensor which enables us to flag troubled elements. The DG approximation is enhanced in these troubled regions by activating $\ell^1$ regularization to promote sparsity in the corresponding jump function of the numerical solution. The resulting $\ell^1$ optimization problem is efficiently implemented using the alternating direction method of multipliers. By enacting $\ell^1$ regularization only in troubled cells, our method remains accurate and efficient, as no additional regularization or expensive iterative procedures are needed in smooth regions. We present results for the inviscid Burgers' equation as well as a nonlinear system of conservation laws using a nodal collocation-type DG method as a solver.
△ Less
Submitted 24 May, 2019; v1 submitted 9 March, 2019;
originally announced March 2019.
-
Image reconstruction enhancement via masked regularization
Authors:
Victor Churchill,
Anne Gelb
Abstract:
Image reconstruction based on an edge-sparsity assumption has become popular in recent years. Many methods of this type are capable of reconstructing nearly perfect edge-sparse images using limited data. In this paper, we present a method to improve the accuracy of a suboptimal image resulting from an edge-sparsity image reconstruction method when compressed sensing or empirical data requirements…
▽ More
Image reconstruction based on an edge-sparsity assumption has become popular in recent years. Many methods of this type are capable of reconstructing nearly perfect edge-sparse images using limited data. In this paper, we present a method to improve the accuracy of a suboptimal image resulting from an edge-sparsity image reconstruction method when compressed sensing or empirical data requirements are not met. The method begins with an edge detection from an initial edge-sparsity based reconstruction. From this edge map, a mask matrix is created which allows us to regularize exclusively in regions away from edges. By accounting for the spatial distribution of the sparsity, our method preserves edge information and and furthermore enhances suboptimal reconstructions to be nearly perfect from fewer data than needed by the initial method. We present results for two phantom images using a variety of initial reconstruction methods.
△ Less
Submitted 31 January, 2019;
originally announced February 2019.
-
Edge-masked CT image reconstruction from limited data
Authors:
Victor Churchill,
Anne Gelb
Abstract:
This paper presents an iterative inversion algorithm for computed tomography image reconstruction that performs well in terms of accuracy and speed using limited data. The computational method combines an image domain technique and statistical reconstruction by using an initial filtered back projection reconstruction to create a binary edge mask, which is then used in an l2-regularized reconstruct…
▽ More
This paper presents an iterative inversion algorithm for computed tomography image reconstruction that performs well in terms of accuracy and speed using limited data. The computational method combines an image domain technique and statistical reconstruction by using an initial filtered back projection reconstruction to create a binary edge mask, which is then used in an l2-regularized reconstruction. Both theoretical and empirical results are offered to support the algorithm. While in this paper a simple forward model is used and physical edges are used as the sparse feature, the proposed method is flexible and can accommodate any forward model and sparsifying transform.
△ Less
Submitted 16 January, 2019;
originally announced January 2019.
-
Detecting edges from non-uniform Fourier data via sparse Bayesian learning
Authors:
Victor Churchill,
Anne Gelb
Abstract:
In recent investigations, the problem of detecting edges given non-uniform Fourier data was reformulated as a sparse signal recovery problem with an l1-regularized least squares cost function. This result can also be derived by employing a Bayesian formulation. Specifically, reconstruction of an edge map using l1 regularization corresponds to a so-called type-I (maximum a posteriori) Bayesian esti…
▽ More
In recent investigations, the problem of detecting edges given non-uniform Fourier data was reformulated as a sparse signal recovery problem with an l1-regularized least squares cost function. This result can also be derived by employing a Bayesian formulation. Specifically, reconstruction of an edge map using l1 regularization corresponds to a so-called type-I (maximum a posteriori) Bayesian estimate. In this paper, we use the Bayesian framework to design an improved algorithm for detecting edges from non-uniform Fourier data. In particular, we employ what is known as type-II Bayesian estimation, specifically a method called sparse Bayesian learning. We also show that our new edge detection method can be used to improve downstream processes that rely on accurate edge information like image reconstruction, especially with regards to compressed sensing techniques.
△ Less
Submitted 29 November, 2018;
originally announced November 2018.
-
Edge-adaptive l2 regularization image reconstruction from non-uniform Fourier data
Authors:
Victor Churchill,
Rick Archibald,
Anne Gelb
Abstract:
Total variation regularization based on the l1 norm is ubiquitous in image reconstruction. However, the resulting reconstructions are not always as sparse in the edge domain as desired. Iteratively reweighted methods provide some improvement in accuracy, but at the cost of extended runtime. In this paper we examine these methods for the case of data acquired as non-uniform Fourier samples. We then…
▽ More
Total variation regularization based on the l1 norm is ubiquitous in image reconstruction. However, the resulting reconstructions are not always as sparse in the edge domain as desired. Iteratively reweighted methods provide some improvement in accuracy, but at the cost of extended runtime. In this paper we examine these methods for the case of data acquired as non-uniform Fourier samples. We then develop a non-iterative weighted regularization method that uses a pre-processing edge detection to find exactly where the sparsity should be in the edge domain. We show that its performance in terms of both accuracy and speed has the potential to outperform reweighted TV regularization methods.
△ Less
Submitted 20 November, 2018;
originally announced November 2018.
-
Using Frame Theoretic Convolutional Gridding for Robust Synthetic Aperture Sonar Imaging
Authors:
John McKay,
Anne Gelb,
Vishal Monga,
Raghu Raj
Abstract:
Recent progress in synthetic aperture sonar (SAS) technology and processing has led to significant advances in underwater imaging, outperforming previously common approaches in both accuracy and efficiency. There are, however, inherent limitations to current SAS reconstruction methodology. In particular, popular and efficient Fourier domain SAS methods require a 2D interpolation which is often ill…
▽ More
Recent progress in synthetic aperture sonar (SAS) technology and processing has led to significant advances in underwater imaging, outperforming previously common approaches in both accuracy and efficiency. There are, however, inherent limitations to current SAS reconstruction methodology. In particular, popular and efficient Fourier domain SAS methods require a 2D interpolation which is often ill conditioned and inaccurate, inevitably reducing robustness with regard to speckle and inaccurate sound-speed estimation. To overcome these issues, we propose using the frame theoretic convolution gridding (FTCG) algorithm to handle the non-uniform Fourier data. FTCG extends upon non-uniform fast Fourier transform (NUFFT) algorithms by casting the NUFFT as an approximation problem given Fourier frame data. The FTCG has been show to yield improved accuracy at little more computational cost. Using simulated data, we outline how the FTCG can be used to enhance current SAS processing.
△ Less
Submitted 26 June, 2017;
originally announced June 2017.
-
Recovering Fine Details from Under-Resolved Electron Tomography Data using HOTV Regularization
Authors:
Toby Sanders,
Anne Gelb,
Rodrigo Platte,
Ilke Arslan,
Kai Landskron
Abstract:
Over the last decade or so, reconstruction methods using $\ell_1$ regularization, often categorized as compressed sensing (CS) algorithms, have significantly improved the capabilities of high fidelity imaging in electron tomography. The most popular $\ell_1$ regularization approach within electron tomography has been total variation (TV) regularization. In addition to reducing unwanted noise, TV r…
▽ More
Over the last decade or so, reconstruction methods using $\ell_1$ regularization, often categorized as compressed sensing (CS) algorithms, have significantly improved the capabilities of high fidelity imaging in electron tomography. The most popular $\ell_1$ regularization approach within electron tomography has been total variation (TV) regularization. In addition to reducing unwanted noise, TV regularization encourages a piecewise constant solution with sparse boundary regions. In this paper we propose an alternative $\ell_1$ regularization approach for electron tomography based on higher order total variation (HOTV). Like TV, the HOTV approach promotes solutions with sparse boundary regions. In smooth regions however, the solution is not limited to piecewise constant behavior. We demonstrate that this allows for more accurate reconstruction of a broader class of images -- even those for which TV was designed for -- particularly when dealing with pragmatic tomographic sampling patterns and very fine image features. We develop results for an electron tomography data set as well as a phantom example, and we also make comparisons with discrete tomography approaches.
△ Less
Submitted 5 March, 2017; v1 submitted 16 August, 2016;
originally announced August 2016.
-
A Two-dimensional Inverse Frame Operator Approximation Technique
Authors:
Guohui Song,
Jacqueline Davis,
Anne Gelb
Abstract:
The ability to efficiently and accurately construct an inverse frame operator is critical for establishing the utility of numerical frame approximations. Recently, the admissible frame method was developed to approximate inverse frame operators for one-dimensional problems. Using the admissible frame approach, it is possible to project the corresponding frame data onto a more suitable (admissible)…
▽ More
The ability to efficiently and accurately construct an inverse frame operator is critical for establishing the utility of numerical frame approximations. Recently, the admissible frame method was developed to approximate inverse frame operators for one-dimensional problems. Using the admissible frame approach, it is possible to project the corresponding frame data onto a more suitable (admissible) frame, even when the sampling frame is only weakly localized. As a result, a target function may be approximated as a finite frame expansion with its asymptotic convergence solely dependent on its smoothness. In this investigation, we seek to expand the admissible frame approach to two dimensions, which requires some additional constraints. We prove that the admissible frame technique converges in two dimensions and then demonstrate its usefulness with some numerical experiments that use sampling patterns inspired by applications that sample data non-uniformly in the Fourier domain.
△ Less
Submitted 9 November, 2015;
originally announced November 2015.
-
A frame theoretic approach to the Non-Uniform Fast Fourier Transform
Authors:
Anne Gelb,
Guohui Song
Abstract:
Nonuniform Fourier data are routinely collected in applications such as magnetic resonance imaging, synthetic aperture radar, and synthetic imaging in radio astronomy. To acquire a fast reconstruction that does not require an online inverse process, the non-uniform fast Fourier transform (NFFT), also called convolutional gridding, is frequently employed. While various investigations have led to im…
▽ More
Nonuniform Fourier data are routinely collected in applications such as magnetic resonance imaging, synthetic aperture radar, and synthetic imaging in radio astronomy. To acquire a fast reconstruction that does not require an online inverse process, the non-uniform fast Fourier transform (NFFT), also called convolutional gridding, is frequently employed. While various investigations have led to improvements in accuracy, efficiency, and robustness of the NFFT, not much attention has been paid to the fundamental analysis of the scheme, and in particular its convergence properties. This paper analyzes the convergence of the NFFT by casting it as a Fourier frame approximation. In so doing, we are able to design parameters for the method that satisfy conditions for numerical convergence. Our so called frame theoretic convolutional gridding algorithm can also be applied to detect features (such as edges) from non-uniform Fourier samples of piecewise smooth functions.
△ Less
Submitted 13 March, 2014;
originally announced March 2014.
-
Approximating the Inverse Frame Operator from Localized Frames
Authors:
Guohui Song,
Anne Gelb
Abstract:
This investigation seeks to establish the practicality of numerical frame approximations. Specifically, it develops a new method to approximate the inverse frame operator and analyzes its convergence properties. It is established that sampling with {\em well-localized frames} improves both the accuracy of the numerical frame approximation as well as the robustness and efficiency of the (finite) fr…
▽ More
This investigation seeks to establish the practicality of numerical frame approximations. Specifically, it develops a new method to approximate the inverse frame operator and analyzes its convergence properties. It is established that sampling with {\em well-localized frames} improves both the accuracy of the numerical frame approximation as well as the robustness and efficiency of the (finite) frame operator inversion. Moreover, in applications such as magnetic resonance imaging, where the given data often may not constitute a well-localized frame, a technique is devised to project the corresponding frame data onto a more suitable frame. As a result, the target function may be approximated as a finite expansion with its asymptotic convergence solely dependent on its smoothness. Numerical examples are provided.
△ Less
Submitted 29 March, 2012;
originally announced March 2012.
-
Detection of Edges in Spectral Data II. Nonlinear Enhancement
Authors:
Anne Gelb,
Eitan Tadmor
Abstract:
We discuss a general framework for recovering edges in piecewise smooth functions with finitely many jump discontinuities, where $[f](x):=f(x+)-f(x-) \neq 0$. Our approach is based on two main aspects--localization using appropriate concentration kernels and separation of scales by nonlinear enhancement.
To detect such edges, one employs concentration kernels, $K_ε(\cdot)$, depending on the sm…
▽ More
We discuss a general framework for recovering edges in piecewise smooth functions with finitely many jump discontinuities, where $[f](x):=f(x+)-f(x-) \neq 0$. Our approach is based on two main aspects--localization using appropriate concentration kernels and separation of scales by nonlinear enhancement.
To detect such edges, one employs concentration kernels, $K_ε(\cdot)$, depending on the small scale $ε$. It is shown that odd kernels, properly scaled, and admissible (in the sense of having small $W^{-1,\infty}$-moments of order ${\cal O}(ε)$) satisfy $K_ε*f(x) = [f](x) +{\cal O}(ε)$, thus recovering both the location and amplitudes of all edges.As an example we consider general concentration kernels of the form $K^σ_N(t)=\sumσ(k/N)\sin kt$ to detect edges from the first $1/ε=N$ spectral modes of piecewise smooth f's. Here we improve in generality and simplicity over our previous study in [A. Gelb and E. Tadmor, Appl. Comput. Harmon. Anal., 7 (1999), pp. 101-135]. Both periodic and nonperiodic spectral projections are considered. We identify, in particular, a new family of exponential factors, $σ^{exp}(\cdot)$, with superior localization properties.
The other aspect of our edge detection involves a nonlinear enhancement procedure which is based on separation of scales between the edges, where $K_ε*f(x)\sim [f](x) \neq 0$, and the smooth regions where $K_ε*f = {\cal O}(ε) \sim 0$. Numerical examples demonstrate that by coupling concentration kernels with nonlinear enhancement one arrives at effective edge detectors.
△ Less
Submitted 3 December, 2001;
originally announced December 2001.
