-
Ring deconvolution microscopy: exploiting symmetry for efficient spatially varying aberration correction
Authors:
Amit Kohli,
Anastasios N. Angelopoulos,
David McAllister,
Esther Whang,
Sixian You,
Kyrollos Yanny,
Federico M. Gasparoli,
Bo-Jui Chang,
Reto Fiolka,
Laura Waller
Abstract:
The most ubiquitous form of computational aberration correction for microscopy is deconvolution. However, deconvolution relies on the assumption that the point spread function is the same across the entire field-of-view. This assumption is often inadequate, but space-variant deblurring techniques generally require impractical amounts of calibration and computation. We present a new imaging pipelin…
▽ More
The most ubiquitous form of computational aberration correction for microscopy is deconvolution. However, deconvolution relies on the assumption that the point spread function is the same across the entire field-of-view. This assumption is often inadequate, but space-variant deblurring techniques generally require impractical amounts of calibration and computation. We present a new imaging pipeline that leverages symmetry to provide simple and fast spatially-varying aberration correction. Our ring deconvolution microscopy (RDM) method leverages the rotational symmetry of most microscopes and cameras, and naturally extends to sheet deconvolution in the case of lateral symmetry. We formally derive theory and algorithms for image recovery and additionally propose a neural network based on Seidel coefficients as a fast alternative, as well as extension of RDM to blind deconvolution. We demonstrate significant improvements in speed and image quality as compared to standard deconvolution and existing spatially-varying deconvolution across a diverse range of microscope modalities, including miniature microscopy, multicolor fluorescence microscopy, point-scanning multimode fiber micro-endoscopy, and light-sheet fluorescence microscopy. Our approach enables near-isotropic, subcellular resolution in each of these applications.
△ Less
Submitted 29 April, 2025; v1 submitted 17 June, 2022;
originally announced June 2022.
-
Miniscope3D: optimized single-shot miniature 3D fluorescence microscopy
Authors:
Kyrollos Yanny,
Nick Antipa,
William Liberti,
Sam Dehaeck,
Kristina Monakhova,
Fanglin Linda Liu,
Konlin Shen,
Ren Ng,
Laura Waller
Abstract:
Miniature fluorescence microscopes are a standard tool in systems biology. However, widefield miniature microscopes capture only 2D information, and modifications that enable 3D capabilities increase the size and weight and have poor resolution outside a narrow depth range. Here, we achieve the 3D capability by replacing the tube lens of a conventional 2D Miniscope with an optimized multifocal pha…
▽ More
Miniature fluorescence microscopes are a standard tool in systems biology. However, widefield miniature microscopes capture only 2D information, and modifications that enable 3D capabilities increase the size and weight and have poor resolution outside a narrow depth range. Here, we achieve the 3D capability by replacing the tube lens of a conventional 2D Miniscope with an optimized multifocal phase mask at the objective's aperture stop. Placing the phase mask at the aperture stop significantly reduces the size of the device, and varying the focal lengths enables a uniform resolution across a wide depth range. The phase mask encodes the 3D fluorescence intensity into a single 2D measurement, and the 3D volume is recovered by solving a sparsity-constrained inverse problem. We provide methods for designing and fabricating the phase mask and an efficient forward model that accounts for the field-varying aberrations in miniature objectives. We demonstrate a prototype that is 17 mm tall and weighs 2.5 grams, achieving 2.76 $μ$m lateral, and 15 $μ$m axial resolution across most of the 900x700x390 $μm^3$ volume at 40 volumes per second. The performance is validated experimentally on resolution targets, dynamic biological samples, and mouse brain tissue. Compared with existing miniature single-shot volume-capture implementations, our system is smaller and lighter and achieves a more than 2x better lateral and axial resolution throughout a 10x larger usable depth range. Our microscope design provides single-shot 3D imaging for applications where a compact platform matters, such as volumetric neural imaging in freely moving animals and 3D motion studies of dynamic samples in incubators and lab-on-a-chip devices.
△ Less
Submitted 11 October, 2020;
originally announced October 2020.
-
Fourier DiffuserScope: Single-shot 3D Fourier light field microscopy with a diffuser
Authors:
Fanglin Linda Liu,
Grace Kuo,
Nick Antipa,
Kyrollos Yanny,
Laura Waller
Abstract:
Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provi…
▽ More
Light field microscopy (LFM) uses a microlens array (MLA) near the sensor plane of a microscope to achieve single-shot 3D imaging of a sample without any moving parts. Unfortunately, the 3D capability of LFM comes with a significant loss of lateral resolution at the focal plane. Placing the MLA near the pupil plane of the microscope, instead of the image plane, can mitigate the artifacts and provide an efficient forward model, at the expense of field-of-view (FOV). Here, we demonstrate improved resolution across a large volume with Fourier DiffuserScope, which uses a diffuser in the pupil plane to encode 3D information, then computationally reconstructs the volume by solving a sparsity-constrained inverse problem. Our diffuser consists of randomly placed microlenses with varying focal lengths; the random positions provide a larger FOV compared to a conventional MLA, and the diverse focal lengths improve the axial depth range. To predict system performance based on diffuser parameters, we for the first time establish a theoretical framework and design guidelines, which are verified by numerical simulations, then build an experimental system that achieves $< 3$ um lateral and $4$ um axial resolution over a $1000 \times 1000 \times 280$ um$^3$ volume. Our diffuser design outperforms the MLA used in LFM, providing more uniform resolution over a larger volume, both laterally and axially.
△ Less
Submitted 16 September, 2020; v1 submitted 29 June, 2020;
originally announced June 2020.
-
Spectral DiffuserCam: lensless snapshot hyperspectral imaging with a spectral filter array
Authors:
Kristina Monakhova,
Kyrollos Yanny,
Neerja Aggarwal,
Laura Waller
Abstract:
Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring; however, traditional scanning hyperspectral imagers are prohibitively slow and expensive for widespread adoption. Snapshot techniques exist but are often confined to bulky benchtop setups or have low spatio-spectral resolution. In this paper, we propose a novel, compact, and inexpensi…
▽ More
Hyperspectral imaging is useful for applications ranging from medical diagnostics to agricultural crop monitoring; however, traditional scanning hyperspectral imagers are prohibitively slow and expensive for widespread adoption. Snapshot techniques exist but are often confined to bulky benchtop setups or have low spatio-spectral resolution. In this paper, we propose a novel, compact, and inexpensive computational camera for snapshot hyperspectral imaging. Our system consists of a tiled spectral filter array placed directly on the image sensor and a diffuser placed close to the sensor. Each point in the world maps to a unique pseudorandom pattern on the spectral filter array, which encodes multiplexed spatio-spectral information. By solving a sparsity-constrained inverse problem, we recover the hyperspectral volume with sub-super-pixel resolution. Our hyperspectral imaging framework is flexible and can be designed with contiguous or non-contiguous spectral filters that can be chosen for a given application. We provide theory for system design, demonstrate a prototype device, and present experimental results with high spatio-spectral resolution.
△ Less
Submitted 28 September, 2020; v1 submitted 15 June, 2020;
originally announced June 2020.
-
Learned reconstructions for practical mask-based lensless imaging
Authors:
Kristina Monakhova,
Joshua Yurtsever,
Grace Kuo,
Nick Antipa,
Kyrollos Yanny,
Laura Waller
Abstract:
Mask-based lensless imagers are smaller and lighter than traditional lensed cameras. In these imagers, the sensor does not directly record an image of the scene; rather, a computational algorithm reconstructs it. Typically, mask-based lensless imagers use a model-based reconstruction approach that suffers from long compute times and a heavy reliance on both system calibration and heuristically cho…
▽ More
Mask-based lensless imagers are smaller and lighter than traditional lensed cameras. In these imagers, the sensor does not directly record an image of the scene; rather, a computational algorithm reconstructs it. Typically, mask-based lensless imagers use a model-based reconstruction approach that suffers from long compute times and a heavy reliance on both system calibration and heuristically chosen denoisers. In this work, we address these limitations using a bounded-compute, trainable neural network to reconstruct the image. We leverage our knowledge of the physical system by unrolling a traditional model-based optimization algorithm, whose parameters we optimize using experimentally gathered ground-truth data. Optionally, images produced by the unrolled network are then fed into a jointly-trained denoiser. As compared to traditional methods, our architecture achieves better perceptual image quality and runs 20x faster, enabling interactive previewing of the scene. We explore a spectrum between model-based and deep learning methods, showing the benefits of using an intermediate approach. Finally, we test our network on images taken in the wild with a prototype mask-based camera, demonstrating that our network generalizes to natural images.
△ Less
Submitted 29 August, 2019;
originally announced August 2019.
