-
A differentiable forward model for the concurrent, multi-peak Bragg coherent x-ray diffraction imaging problem
Authors:
S. Maddali,
T. D. Frazer,
N. Delegan,
K. J. Harmon,
S. E. Sullivan,
M. Allain,
W. Cha,
A. Dibos,
I. Poudyal,
S. Kandel,
Y. S. G. Nashed,
F. J. Heremans,
H. You,
Y. Cao,
S. O. Hruszkewycz
Abstract:
We present a general analytic approach to spatially resolve the nano-scale lattice distortion field of strained and defected compact crystals with Bragg coherent x-ray diffraction imaging (BCDI). Our approach relies on fitting a differentiable forward model simultaneously to multiple BCDI datasets corresponding to independent Bragg reflections from the same single crystal. It is designed to be fai…
▽ More
We present a general analytic approach to spatially resolve the nano-scale lattice distortion field of strained and defected compact crystals with Bragg coherent x-ray diffraction imaging (BCDI). Our approach relies on fitting a differentiable forward model simultaneously to multiple BCDI datasets corresponding to independent Bragg reflections from the same single crystal. It is designed to be faithful to heterogeneities that potentially manifest as phase discontinuities in the coherently diffracted wave, such as lattice dislocations in an imperfect crystal. We retain fidelity to such small features in the reconstruction process through a Fourier transform -based resampling algorithm designed to largely avoid the point spread tendencies of commonly employed interpolation methods. The reconstruction model defined in this manner brings BCDI reconstruction into the scope of explicit optimization driven by automatic differentiation. With results from simulations and experimental diffraction data, we demonstrate significant improvement in the final image quality compared to conventional phase retrieval, enabled by explicitly coupling multiple BCDI datasets into the reconstruction loss function.
△ Less
Submitted 1 August, 2022;
originally announced August 2022.
-
End-to-End Simultaneous Learning of Single-particle Orientation and 3D Map Reconstruction from Cryo-electron Microscopy Data
Authors:
Youssef S. G. Nashed,
Frederic Poitevin,
Harshit Gupta,
Geoffrey Woollard,
Michael Kagan,
Chuck Yoon,
Daniel Ratner
Abstract:
Cryogenic electron microscopy (cryo-EM) provides images from different copies of the same biomolecule in arbitrary orientations. Here, we present an end-to-end unsupervised approach that learns individual particle orientations from cryo-EM data while reconstructing the average 3D map of the biomolecule, starting from a random initialization. The approach relies on an auto-encoder architecture wher…
▽ More
Cryogenic electron microscopy (cryo-EM) provides images from different copies of the same biomolecule in arbitrary orientations. Here, we present an end-to-end unsupervised approach that learns individual particle orientations from cryo-EM data while reconstructing the average 3D map of the biomolecule, starting from a random initialization. The approach relies on an auto-encoder architecture where the latent space is explicitly interpreted as orientations used by the decoder to form an image according to the linear projection model. We evaluate our method on simulated data and show that it is able to reconstruct 3D particle maps from noisy- and CTF-corrupted 2D projection images of unknown particle orientations.
△ Less
Submitted 6 July, 2021;
originally announced July 2021.
-
A matrix-free Levenberg-Marquardt algorithm for efficient ptychographic phase retrieval
Authors:
Saugat Kandel,
S. Maddali,
Youssef S G Nashed,
Stephan O Hruszkewycz,
Chris Jacobsen,
Marc Allain
Abstract:
The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second o…
▽ More
The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second order derivatives is even more laborious. Additionally, second order techniques often require the storage and inversion of large matrices of partial derivatives, with memory requirements that can be prohibitive for data-rich imaging modalities. We use a reverse-mode automatic differentiation (AD) framework to implement an efficient matrix-free version of the Levenberg-Marquardt (LM) algorithm, a longstanding method that finds popular use in nonlinear least-square minimization problems but which has seen little use in phase retrieval. Furthermore, we extend the basic LM algorithm so that it can be applied for general constrained optimization problems beyond just the least-square applications. Since we use AD, we only need to specify the physics-based forward model for a specific imaging application; the derivative terms are calculated automatically through matrix-vector products, without explicitly forming any large Jacobian or Gauss-Newton matrices. We demonstrate that this algorithm can be used to solve both the unconstrained ptychographic object retrieval problem and the constrained "blind" ptychographic object and probe retrieval problems, under both the Gaussian and Poisson noise models, and that this method outperforms best-in-class first-order ptychographic reconstruction methods: it provides excellent convergence guarantees with (in many cases) a superlinear rate of convergence, all with a computational cost comparable to, or lower than, the tested first-order algorithms.
△ Less
Submitted 26 February, 2021;
originally announced March 2021.
-
Real-time 3D Nanoscale Coherent Imaging via Physics-aware Deep Learning
Authors:
Henry Chan,
Youssef S. G. Nashed,
Saugat Kandel,
Stephan Hruszkewycz,
Subramanian Sankaranarayanan,
Ross J. Harder,
Mathew J. Cherukara
Abstract:
Phase retrieval, the problem of recovering lost phase information from measured intensity alone, is an inverse problem that is widely faced in various imaging modalities ranging from astronomy to nanoscale imaging. The current process of phase recovery is iterative in nature. As a result, the image formation is time-consuming and computationally expensive, precluding real-time imaging. Here, we us…
▽ More
Phase retrieval, the problem of recovering lost phase information from measured intensity alone, is an inverse problem that is widely faced in various imaging modalities ranging from astronomy to nanoscale imaging. The current process of phase recovery is iterative in nature. As a result, the image formation is time-consuming and computationally expensive, precluding real-time imaging. Here, we use 3D nanoscale X-ray imaging as a representative example to develop a deep learning model to address this phase retrieval problem. We introduce 3D-CDI-NN, a deep convolutional neural network and differential programming framework trained to predict 3D structure and strain solely from input 3D X-ray coherent scattering data. Our networks are designed to be "physics-aware" in multiple aspects; in that the physics of x-ray scattering process is explicitly enforced in the training of the network, and the training data are drawn from atomistic simulations that are representative of the physics of the material. We further refine the neural network prediction through a physics-based optimization procedure to enable maximum accuracy at lowest computational cost. 3D-CDI-NN can invert a 3D coherent diffraction pattern to real-space structure and strain hundreds of times faster than traditional iterative phase retrieval methods, with negligible loss in accuracy. Our integrated machine learning and differential programming solution to the phase retrieval problem is broadly applicable across inverse problems in other application areas.
△ Less
Submitted 16 June, 2020;
originally announced June 2020.
-
InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations
Authors:
Wenbin He,
Junpeng Wang,
Hanqi Guo,
Ko-Chih Wang,
Han-Wei Shen,
Mukund Raj,
Youssef S. G. Nashed,
Tom Peterka
Abstract:
We propose InSituNet, a deep learning based surrogate model to support parameter space exploration for ensemble simulations that are visualized in situ. In situ visualization, generating visualizations at simulation time, is becoming prevalent in handling large-scale simulations because of the I/O and storage constraints. However, in situ visualization approaches limit the flexibility of post-hoc…
▽ More
We propose InSituNet, a deep learning based surrogate model to support parameter space exploration for ensemble simulations that are visualized in situ. In situ visualization, generating visualizations at simulation time, is becoming prevalent in handling large-scale simulations because of the I/O and storage constraints. However, in situ visualization approaches limit the flexibility of post-hoc exploration because the raw simulation data are no longer available. Although multiple image-based approaches have been proposed to mitigate this limitation, those approaches lack the ability to explore the simulation parameters. Our approach allows flexible exploration of parameter space for large-scale ensemble simulations by taking advantage of the recent advances in deep learning. Specifically, we design InSituNet as a convolutional regression model to learn the mapping from the simulation and visualization parameters to the visualization results. With the trained model, users can generate new images for different simulation parameters under various visualization settings, which enables in-depth analysis of the underlying ensemble simulations. We demonstrate the effectiveness of InSituNet in combustion, cosmology, and ocean simulations through quantitative and qualitative evaluations.
△ Less
Submitted 16 October, 2019; v1 submitted 1 August, 2019;
originally announced August 2019.
-
Three dimensions, two microscopes, one code: automatic differentiation for x-ray nanotomography beyond the depth of focus limit
Authors:
Ming Du,
Youssef S. G. Nashed,
Saugat Kandel,
Doga Gursoy,
Chris Jacobsen
Abstract:
Conventional tomographic reconstruction algorithms assume that one has obtained pure projection images, involving no within-specimen diffraction effects nor multiple scattering. Advances in x-ray nanotomography are leading towards the violation of these assumptions, by combining the high penetration power of x-rays which enables thick specimens to be imaged, with improved spatial resolution which…
▽ More
Conventional tomographic reconstruction algorithms assume that one has obtained pure projection images, involving no within-specimen diffraction effects nor multiple scattering. Advances in x-ray nanotomography are leading towards the violation of these assumptions, by combining the high penetration power of x-rays which enables thick specimens to be imaged, with improved spatial resolution which decreases the depth of focus of the imaging system. We describe a reconstruction method where multiple scattering and diffraction effects in thick samples are modeled by multislice propagation, and the 3D object function is retrieved through iterative optimization. We show that the same proposed method works for both full-field microscopy, and for coherent scanning techniques like ptychography. Our implementation utilizes the optimization toolbox and the automatic differentiation capability of the open-source deep learning package TensorFlow, which demonstrates a much straightforward way to solve optimization problems in computational imaging, and endows our program great flexibility and portability.
△ Less
Submitted 24 May, 2019;
originally announced May 2019.
-
Using Automatic Differentiation as a General Framework for Ptychographic Reconstruction
Authors:
Saugat Kandel,
S. Maddali,
Marc Allain,
Stephan O. Hruszkewycz,
Chris Jacobsen,
Youssef S G Nashed
Abstract:
Coherent diffraction imaging methods enable imaging beyond lens-imposed resolution limits. In these methods, the object can be recovered by minimizing an error metric that quantifies the difference between diffraction patterns as observed, and those calculated from a present guess of the object. Efficient minimization methods require analytical calculation of the derivatives of the error metric, w…
▽ More
Coherent diffraction imaging methods enable imaging beyond lens-imposed resolution limits. In these methods, the object can be recovered by minimizing an error metric that quantifies the difference between diffraction patterns as observed, and those calculated from a present guess of the object. Efficient minimization methods require analytical calculation of the derivatives of the error metric, which is not always straightforward. This limits our ability to explore variations of basic imaging approaches. In this paper, we propose to substitute analytical derivative expressions with the automatic differentiation method, whereby we can achieve object reconstruction by specifying only the physics-based experimental forward model. We demonstrate the generality of the proposed method through straightforward object reconstruction for a variety of complex ptychographic experimental models.
△ Less
Submitted 21 May, 2019; v1 submitted 8 February, 2019;
originally announced February 2019.
-
Real-time coherent diffraction inversion using deep generative networks
Authors:
Mathew J. Cherukara,
Youssef S. G. Nashed,
Ross J. Harder
Abstract:
Phase retrieval, or the process of recovering phase information in reciprocal space to reconstruct images from measured intensity alone, is the underlying basis to a variety of imaging applications including coherent diffraction imaging (CDI). Typical phase retrieval algorithms are iterative in nature, and hence, are time-consuming and computationally expensive, precluding real-time imaging. Furth…
▽ More
Phase retrieval, or the process of recovering phase information in reciprocal space to reconstruct images from measured intensity alone, is the underlying basis to a variety of imaging applications including coherent diffraction imaging (CDI). Typical phase retrieval algorithms are iterative in nature, and hence, are time-consuming and computationally expensive, precluding real-time imaging. Furthermore, iterative phase retrieval algorithms struggle to converge to the correct solution especially in the presence of strong phase structures. In this work, we demonstrate the training and testing of CDI NN, a pair of deep deconvolutional networks trained to predict structure and phase in real space of a 2D object from its corresponding far-field diffraction intensities alone. Once trained, CDI NN can invert a diffraction pattern to an image within a few milliseconds of compute time on a standard desktop machine, opening the door to real-time imaging.
△ Less
Submitted 7 June, 2018;
originally announced June 2018.
