-
Upper-Bounding the Regularization Constant for Convex Sparse Signal Reconstruction
Authors:
Renliang Gu,
Aleksandar Dogandžić
Abstract:
Consider reconstructing a signal $x$ by minimizing a weighted sum of a convex differentiable negative log-likelihood (NLL) (data-fidelity) term and a convex regularization term that imposes a convex-set constraint on $x$ and enforces its sparsity using $\ell_1$-norm analysis regularization. We compute upper bounds on the regularization tuning constant beyond which the regularization term overwhelm…
▽ More
Consider reconstructing a signal $x$ by minimizing a weighted sum of a convex differentiable negative log-likelihood (NLL) (data-fidelity) term and a convex regularization term that imposes a convex-set constraint on $x$ and enforces its sparsity using $\ell_1$-norm analysis regularization. We compute upper bounds on the regularization tuning constant beyond which the regularization term overwhelmingly dominates the NLL term so that the set of minimum points of the objective function does not change. Necessary and sufficient conditions for irrelevance of sparse signal regularization and a condition for the existence of finite upper bounds are established. We formulate an optimization problem for finding these bounds when the regularization term can be globally minimized by a feasible $x$ and also develop an alternating direction method of multipliers (ADMM) type method for their computation. Simulation examples show that the derived and empirical bounds match.
△ Less
Submitted 25 February, 2017;
originally announced February 2017.
-
Polychromatic X-ray CT Image Reconstruction and Mass-Attenuation Spectrum Estimation
Authors:
Renliang Gu,
Aleksandar Dogandžić
Abstract:
We develop a method for sparse image reconstruction from polychromatic computed tomography (CT) measurements under the blind scenario where the material of the inspected object and the incident-energy spectrum are unknown. We obtain a parsimonious measurement-model parameterization by changing the integral variable from photon energy to mass attenuation, which allows us to combine the variations b…
▽ More
We develop a method for sparse image reconstruction from polychromatic computed tomography (CT) measurements under the blind scenario where the material of the inspected object and the incident-energy spectrum are unknown. We obtain a parsimonious measurement-model parameterization by changing the integral variable from photon energy to mass attenuation, which allows us to combine the variations brought by the unknown incident spectrum and mass attenuation into a single unknown mass-attenuation spectrum function; the resulting measurement equation has the Laplace integral form. The mass-attenuation spectrum is then expanded into first order B-spline basis functions. We derive a block coordinate-descent algorithm for constrained minimization of a penalized negative log-likelihood (NLL) cost function, where penalty terms ensure nonnegativity of the spline coefficients and nonnegativity and sparsity of the density map. The image sparsity is imposed using total-variation (TV) and $\ell_1$ norms, applied to the density-map image and its discrete wavelet transform (DWT) coefficients, respectively. This algorithm alternates between Nesterov's proximal-gradient (NPG) and limited-memory Broyden-Fletcher-Goldfarb-Shanno with box constraints (L-BFGS-B) steps for updating the image and mass-attenuation spectrum parameters. To accelerate convergence of the density-map NPG step, we apply a step-size selection scheme that accounts for varying local Lipschitz constant of the NLL. We consider lognormal and Poisson noise models and establish conditions for biconvexity of the corresponding NLLs. We also prove the Kurdyka-Łojasiewicz property of the objective function, which is important for establishing local convergence of the algorithm. Numerical experiments with simulated and real X-ray CT data demonstrate the performance of the proposed scheme.
△ Less
Submitted 7 September, 2015;
originally announced September 2015.
-
Projected Nesterov's Proximal-Gradient Algorithm for Sparse Signal Reconstruction with a Convex Constraint
Authors:
Renliang Gu,
Aleksandar Dogandžić
Abstract:
We develop a projected Nesterov's proximal-gradient (PNPG) approach for sparse signal reconstruction that combines adaptive step size with Nesterov's momentum acceleration. The objective function that we wish to minimize is the sum of a convex differentiable data-fidelity (negative log-likelihood (NLL)) term and a convex regularization term. We apply sparse signal regularization where the signal b…
▽ More
We develop a projected Nesterov's proximal-gradient (PNPG) approach for sparse signal reconstruction that combines adaptive step size with Nesterov's momentum acceleration. The objective function that we wish to minimize is the sum of a convex differentiable data-fidelity (negative log-likelihood (NLL)) term and a convex regularization term. We apply sparse signal regularization where the signal belongs to a closed convex set within the closure of the domain of the NLL; the convex-set constraint facilitates flexible NLL domains and accurate signal recovery. Signal sparsity is imposed using the $\ell_1$-norm penalty on the signal's linear transform coefficients or gradient map, respectively. The PNPG approach employs projected Nesterov's acceleration step with restart and an inner iteration to compute the proximal mapping. We propose an adaptive step-size selection scheme to obtain a good local majorizing function of the NLL and reduce the time spent backtracking. Thanks to step-size adaptation, PNPG does not require Lipschitz continuity of the gradient of the NLL. We present an integrated derivation of the momentum acceleration and its $\mathcal{O}(k^{-2})$ convergence-rate and iterate convergence proofs, which account for adaptive step-size selection, inexactness of the iterative proximal mapping, and the convex-set constraint. The tuning of PNPG is largely application-independent. Tomographic and compressed-sensing reconstruction experiments with Poisson generalized linear and Gaussian linear measurement models demonstrate the performance of the proposed approach.
△ Less
Submitted 5 October, 2016; v1 submitted 9 February, 2015;
originally announced February 2015.
-
A Max-Product EM Algorithm for Reconstructing Markov-tree Sparse Signals from Compressive Samples
Authors:
Zhao Song,
Aleksandar Dogandzic
Abstract:
We propose a Bayesian expectation-maximization (EM) algorithm for reconstructing Markov-tree sparse signals via belief propagation. The measurements follow an underdetermined linear model where the regression-coefficient vector is the sum of an unknown approximately sparse signal and a zero-mean white Gaussian noise with an unknown variance. The signal is composed of large- and small-magnitude com…
▽ More
We propose a Bayesian expectation-maximization (EM) algorithm for reconstructing Markov-tree sparse signals via belief propagation. The measurements follow an underdetermined linear model where the regression-coefficient vector is the sum of an unknown approximately sparse signal and a zero-mean white Gaussian noise with an unknown variance. The signal is composed of large- and small-magnitude components identified by binary state variables whose probabilistic dependence structure is described by a Markov tree. Gaussian priors are assigned to the signal coefficients given their state variables and the Jeffreys' noninformative prior is assigned to the noise variance. Our signal reconstruction scheme is based on an EM iteration that aims at maximizing the posterior distribution of the signal and its state variables given the noise variance. We construct the missing data for the EM iteration so that the complete-data posterior distribution corresponds to a hidden Markov tree (HMT) probabilistic graphical model that contains no loops and implement its maximization (M) step via a max-product algorithm. This EM algorithm estimates the vector of state variables as well as solves iteratively a linear system of equations to obtain the corresponding signal estimate. We select the noise variance so that the corresponding estimated signal and state variables obtained upon convergence of the EM iteration have the largest marginal posterior distribution. We compare the proposed and existing state-of-the-art reconstruction methods via signal and image reconstruction experiments.
△ Less
Submitted 25 August, 2013; v1 submitted 5 September, 2012;
originally announced September 2012.
-
Mask Iterative Hard Thresholding Algorithms for Sparse Image Reconstruction of Objects with Known Contour
Authors:
Aleksandar Dogandzic,
Renliang Gu,
Kun Qiu
Abstract:
We develop mask iterative hard thresholding algorithms (mask IHT and mask DORE) for sparse image reconstruction of objects with known contour. The measurements follow a noisy underdetermined linear model common in the compressive sampling literature. Assuming that the contour of the object that we wish to reconstruct is known and that the signal outside the contour is zero, we formulate a constrai…
▽ More
We develop mask iterative hard thresholding algorithms (mask IHT and mask DORE) for sparse image reconstruction of objects with known contour. The measurements follow a noisy underdetermined linear model common in the compressive sampling literature. Assuming that the contour of the object that we wish to reconstruct is known and that the signal outside the contour is zero, we formulate a constrained residual squared error minimization problem that incorporates both the geometric information (i.e. the knowledge of the object's contour) and the signal sparsity constraint. We first introduce a mask IHT method that aims at solving this minimization problem and guarantees monotonically non-increasing residual squared error for a given signal sparsity level. We then propose a double overrelaxation scheme for accelerating the convergence of the mask IHT algorithm. We also apply convex mask reconstruction approaches that employ a convex relaxation of the signal sparsity constraint. In X-ray computed tomography (CT), we propose an automatic scheme for extracting the convex hull of the inspected object from the measured sinograms; the obtained convex hull is used to capture the object contour information. We compare the proposed mask reconstruction schemes with the existing large-scale sparse signal reconstruction methods via numerical simulations and demonstrate that, by exploiting both the geometric contour information of the underlying image and sparsity of its wavelet coefficients, we can reconstruct this image using a significantly smaller number of measurements than the existing methods.
△ Less
Submitted 2 December, 2011;
originally announced December 2011.
