-
QuATON: Quantization Aware Training of Optical Neurons
Authors:
Hasindu Kariyawasam,
Ramith Hettiarachchi,
Quansan Yang,
Alex Matlock,
Takahiro Nambara,
Hiroyuki Kusaka,
Yuichiro Kunai,
Peter T C So,
Edward S Boyden,
Dushan Wadduwage
Abstract:
Optical processors, built with "optical neurons", can efficiently perform high-dimensional linear operations at the speed of light. Thus they are a promising avenue to accelerate large-scale linear computations. With the current advances in micro-fabrication, such optical processors can now be 3D fabricated, but with a limited precision. This limitation translates to quantization of learnable para…
▽ More
Optical processors, built with "optical neurons", can efficiently perform high-dimensional linear operations at the speed of light. Thus they are a promising avenue to accelerate large-scale linear computations. With the current advances in micro-fabrication, such optical processors can now be 3D fabricated, but with a limited precision. This limitation translates to quantization of learnable parameters in optical neurons, and should be handled during the design of the optical processor in order to avoid a model mismatch. Specifically, optical neurons should be trained or designed within the physical-constraints at a predefined quantized precision level. To address this critical issues we propose a physics-informed quantization-aware training framework. Our approach accounts for physical constraints during the training process, leading to robust designs. We demonstrate that our approach can design state of the art optical processors using diffractive networks for multiple physics based tasks despite quantized learnable parameters. We thus lay the foundation upon which improved optical processors may be 3D fabricated in the future.
△ Less
Submitted 21 March, 2024; v1 submitted 3 October, 2023;
originally announced October 2023.
-
Neural Lithography: Close the Design-to-Manufacturing Gap in Computational Optics with a 'Real2Sim' Learned Photolithography Simulator
Authors:
Cheng Zheng,
Guangyuan Zhao,
Peter T. C. So
Abstract:
We introduce neural lithography to address the 'design-to-manufacturing' gap in computational optics. Computational optics with large design degrees of freedom enable advanced functionalities and performance beyond traditional optics. However, the existing design approaches often overlook the numerical modeling of the manufacturing process, which can result in significant performance deviation bet…
▽ More
We introduce neural lithography to address the 'design-to-manufacturing' gap in computational optics. Computational optics with large design degrees of freedom enable advanced functionalities and performance beyond traditional optics. However, the existing design approaches often overlook the numerical modeling of the manufacturing process, which can result in significant performance deviation between the design and the fabricated optics. To bridge this gap, we, for the first time, propose a fully differentiable design framework that integrates a pre-trained photolithography simulator into the model-based optical design loop. Leveraging a blend of physics-informed modeling and data-driven training using experimentally collected datasets, our photolithography simulator serves as a regularizer on fabrication feasibility during design, compensating for structure discrepancies introduced in the lithography process. We demonstrate the effectiveness of our approach through two typical tasks in computational optics, where we design and fabricate a holographic optical element (HOE) and a multi-level diffractive lens (MDL) using a two-photon lithography system, showcasing improved optical performance on the task-specific metrics.
△ Less
Submitted 29 September, 2023;
originally announced September 2023.
-
Reflection-mode optical diffraction tomography for label-free imaging of thick biological specimens
Authors:
Sungsam Kang,
Renjie Zhou,
Marten Brelen,
Heather K. Mak,
Peter T. C. So,
Zahid Yaqoob
Abstract:
Optical diffraction tomography (ODT) has emerged as a powerful label-free three-dimensional (3D) bioimaging techniques for observing living cells and thin tissue layers. We report a new reflection-mode ODT (rODT) method for imaging thick biological specimens with 500 nm lateral resolution and 1 μm axial resolution. In rODT, multiple scattering background is rejected through spatio-temporal gating…
▽ More
Optical diffraction tomography (ODT) has emerged as a powerful label-free three-dimensional (3D) bioimaging techniques for observing living cells and thin tissue layers. We report a new reflection-mode ODT (rODT) method for imaging thick biological specimens with 500 nm lateral resolution and 1 μm axial resolution. In rODT, multiple scattering background is rejected through spatio-temporal gating provided by dynamic speckle-field interferometry, while depth-resolved refractive index maps are reconstructed by developing a comprehensive inverse scattering model that also considers specimen-induced aberration. Benefiting from the high-resolution and full-field quantitative imaging capabilities of rODT, we succeeded in imaging red blood cells and quantifying their membrane fluctuations behind a turbid sample with a thickness of 2.8 scattering mean-free-paths. We further realized volumetric imaging of cornea inside an ex vivo rat eye and quantified its optical properties, including mapping the topography of Dua's and Descemet's membrane surfaces on the nanometer scale.
△ Less
Submitted 28 February, 2022;
originally announced February 2022.
-
xSCYTE: Express Single-frame Cytometer through Tomographic Phase
Authors:
Baoliang Ge,
Yanping He,
Mo Deng,
Md Habibur Rahman,
Yijin Wang,
Ziling Wu,
Yongliang Yang,
Cuifang Kuang,
Chung Hong N. Wong,
Michael K. Chan,
Yi-Ping Ho,
Liting Duan,
Zahid Yaqoob,
Peter T. C. So,
George Barbastathis,
Renjie Zhou
Abstract:
Rapid, comprehensive, and accurate cell phenotyping without compromising viability, is crucial to many important biomedical applications, including stem-cell therapy, drug screening, and liquid biopsy. Typical image cytometry methods acquire two-dimensional (2D) fluorescence images, where the fluorescence labelling process may damage living cells, and the information from 2D images is not comprehe…
▽ More
Rapid, comprehensive, and accurate cell phenotyping without compromising viability, is crucial to many important biomedical applications, including stem-cell therapy, drug screening, and liquid biopsy. Typical image cytometry methods acquire two-dimensional (2D) fluorescence images, where the fluorescence labelling process may damage living cells, and the information from 2D images is not comprehensive enough for precise cell analysis. Although three-dimensional (3D) label-free image cytometry holds great promise, its high throughput development faces several technical challenges. Here, we report eXpress Single-frame CYtometer through Tomographic phasE (xSCYTE), which reconstructs 3D Refractive Index (RI) maps of cells with diffraction-limited resolution. With these high-speed and high-precision imaging capabilities empowered by artificial intelligence, we envision xSCYTE may open up many new avenues of biomedical investigations and industries, such as multi-omic assays and quality control during cellular therapeutic manufacturing.
△ Less
Submitted 5 November, 2024; v1 submitted 7 February, 2022;
originally announced February 2022.
-
Single-shot quantitative polarization imaging of complex birefringent structure dynamics
Authors:
Baoliang Ge,
Qing Zhang,
Rui Zhang,
Jing-Tang Lin,
Po-Hang Tseng,
Che-Wei Chang,
Chen-Yuan Dong,
Renjie Zhou,
Zahid Yaqoob,
Irmgard Bischofberger,
Peter T. C. So
Abstract:
Polarization light microscopes are powerful tools for probing molecular order and orientation in birefringent materials. While a multitude of polarization light microscopy techniques are often used to access steady-state properties of birefringent samples, quantitative measurements of the molecular orientation dynamics on the millisecond time scale have remained a challenge. We propose polarized s…
▽ More
Polarization light microscopes are powerful tools for probing molecular order and orientation in birefringent materials. While a multitude of polarization light microscopy techniques are often used to access steady-state properties of birefringent samples, quantitative measurements of the molecular orientation dynamics on the millisecond time scale have remained a challenge. We propose polarized shearing interference microscopy (PSIM), a single-shot quantitative polarization imaging method, for extracting the retardance and orientation angle of the laser beam transmitting through optically anisotropic specimens with complex structures. The measurement accuracy and imaging performances of PSIM are validated by imaging a rotating wave plate and a bovine tendon specimen. We demonstrate that PSIM can quantify the dynamics of a flowing lyotropic chromonic liquid crystal in a microfluidic channel at an imaging speed of 506 frames per second (only limited by the camera frame rate), with a field-of-view of up to $350\times350 μm^2$ and a diffraction-limit spatial resolution of $\sim 2μm$. We envision that PSIM will find a broad range of applications in quantitative material characterization under dynamical conditions.
△ Less
Submitted 10 June, 2021;
originally announced June 2021.
-
3D Deep Learning Enables Fast Imaging of Spines through Scattering Media by Temporal Focusing Microscopy
Authors:
Zhun Wei,
Josiah R. Boivin,
Yi Xue,
Xudong Chen,
Peter T. C. So,
Elly Nedivi,
Dushan N. Wadduwage
Abstract:
Today the gold standard for in vivo imaging through scattering tissue is the point-scanning two-photon microscope (PSTPM). Especially in neuroscience, PSTPM is widely used for deep-tissue imaging in the brain. However, due to sequential scanning, PSTPM is slow. Temporal focusing microscopy (TFM), on the other hand, focuses femtosecond pulsed laser light temporally, while keeping wide-field illumin…
▽ More
Today the gold standard for in vivo imaging through scattering tissue is the point-scanning two-photon microscope (PSTPM). Especially in neuroscience, PSTPM is widely used for deep-tissue imaging in the brain. However, due to sequential scanning, PSTPM is slow. Temporal focusing microscopy (TFM), on the other hand, focuses femtosecond pulsed laser light temporally, while keeping wide-field illumination, and is consequently much faster. However, due to the use of a camera detector, TFM suffers from the scattering of emission photons. As a result, TFM produces images of poor spatial resolution and signal-to-noise ratio (SNR), burying fluorescent signals from small structures such as dendritic spines. In this work, we present a data-driven deep learning approach to improve resolution and SNR of TFM images. Using a 3D convolutional neural network (CNN) we build a map from TFM to PSTPM modalities, to enable fast TFM imaging while maintaining high-resolution through scattering media. We demonstrate this approach for in vivo imaging of dendritic spines on pyramidal neurons in the mouse visual cortex. We show that our trained network rapidly outputs high-resolution images that recover biologically relevant features previously buried in the scattered fluorescence in the TFM images. In vivo imaging that combines TFM and the proposed 3D convolution neural network is one to two orders of magnitude faster than PSTPM but retains the high resolution and SNR necessary to analyze small fluorescent structures. The proposed 3D convolution deep network could also be potentially beneficial for improving the performance of many speed-demanding deep-tissue imaging applications such as in vivo voltage imaging.
△ Less
Submitted 24 December, 2019;
originally announced January 2020.
-
Multiline Orthogonal Scanning Temporal Focusing (mosTF) Microscopy for Scattering Reduction in High-speed in vivo Brain Imaging
Authors:
Yi Xue,
Josiah R. Boivin,
Dushan N. Wadduwage,
Jong Kang Park,
Elly Nedivi,
Peter T. C. So
Abstract:
Temporal focusing two-photon microscopy enables high resolution imaging of fine structures in vivo over a large volume. A limitation of temporal focusing is that signal-to-background ratio and resolution degrade rapidly with increasing imaging depth. This degradation originates from the scattered emission photons are widely distributed resulting in a strong background. We have developed Multiline…
▽ More
Temporal focusing two-photon microscopy enables high resolution imaging of fine structures in vivo over a large volume. A limitation of temporal focusing is that signal-to-background ratio and resolution degrade rapidly with increasing imaging depth. This degradation originates from the scattered emission photons are widely distributed resulting in a strong background. We have developed Multiline Orthogonal Scanning Temporal Focusing (mosTF) microscopy that overcomes this problem. mosTF captures a sequence of images at each scan location of the excitation line, followed by a reconstruction algorithm reassigns scattered photons back to the correct scan position. We demonstrate mosTF by acquiring mice neuronal images in vivo. Our results show remarkably improvements with mosTF for in vivo brain imaging while maintaining its speed advantage.
△ Less
Submitted 9 December, 2019; v1 submitted 27 May, 2019;
originally announced May 2019.
-
De-scattering with Excitation Patterning (DEEP) Enables Rapid Wide-field Imaging Through Scattering Media
Authors:
Dushan N. Wadduwage,
Jong Kang Park,
Josiah R. Boivin,
Yi Xue,
Peter T. C. So
Abstract:
From multi-photon imaging penetrating millimeters deep through scattering biological tissue, to super-resolution imaging conquering the diffraction limit, optical imaging techniques have greatly advanced in recent years. Notwithstanding, a key unmet challenge in all these imaging techniques is to perform rapid wide-field imaging through a turbid medium. Strategies such as active wave-front correct…
▽ More
From multi-photon imaging penetrating millimeters deep through scattering biological tissue, to super-resolution imaging conquering the diffraction limit, optical imaging techniques have greatly advanced in recent years. Notwithstanding, a key unmet challenge in all these imaging techniques is to perform rapid wide-field imaging through a turbid medium. Strategies such as active wave-front correction and multi-photon excitation, both used for deep tissue imaging; or wide-field total-internal-refection illumination, used for super-resolution imaging; can generate arbitrary excitation patterns over a large field-of-view through or under turbid media. In these cases, throughput advantage gained by wide-field excitation is lost due to the use of point detection. To address this challenge, here we introduce a novel technique called De-scattering with Excitation Patterning, or 'DEEP', which uses patterned excitation followed by wide-field detection with computational imaging. We use two-photon temporal focusing (TFM) to demonstrate our approach at multiple scattering lengths deep in tissue. Our results suggest that millions of point-scanning measurements could be substituted with tens to hundreds of DEEP measurements with no compromise in image quality.
△ Less
Submitted 27 February, 2019;
originally announced February 2019.
-
Chip-based Resonance Raman Spectroscopy Using Tantalum Pentoxide Waveguides
Authors:
David A. Coucheron,
Dushan N. Wadduwage,
G. Senthil Murugan,
Peter T. C. So,
Balpreet S. Ahluwalia
Abstract:
Blood analysis is an important diagnostic tool, as it provides a wealth of information about the patient health. Raman spectroscopy is a promising tool in blood analysis, but widespread clinical application is limited by its low signal strength, as well as complex and costly instrumentation. The growing field of waveguide-based Raman spectroscopy tries to solve these challenges by working towards…
▽ More
Blood analysis is an important diagnostic tool, as it provides a wealth of information about the patient health. Raman spectroscopy is a promising tool in blood analysis, but widespread clinical application is limited by its low signal strength, as well as complex and costly instrumentation. The growing field of waveguide-based Raman spectroscopy tries to solve these challenges by working towards fully integrated Raman sensors with increased interaction areas. In this work, we demonstrate on-chip resonance Raman measurements of hemoglobin, a crucial component of blood, at 532 nm excitation using a tantalum pentoxide (Ta2O5) waveguide platform. We have also characterized the background signal from Ta2O5 waveguide material when excited at 532 nm. In addition, we demonstrate spontaneous Raman measurements of iso-propanol and methanol using the same platform. Our results suggest that Ta2O5 is a promising waveguide platform for resonance Raman spectroscopy at 532 nm, and in particular for blood analysis.
△ Less
Submitted 11 November, 2018;
originally announced November 2018.
-
Reference-free polarization-sensitive quantitative phase imaging using singe-point optical phase conjugation
Authors:
Seungwoo Shin,
Kyeoreh Lee,
Zahid Yaqoob,
Peter T. C. So,
Yongkeun Park
Abstract:
We propose and experimentally demonstrate a method of polarization-sensitive quantitative phase imaging using two photo detectors. Instead of recording wide-field interference patterns, finding the modulation patterns maximizing focused intensities in terms of the polarization states enables polarization-dependent quantitative phase imaging without the need for a reference beam and an image sensor…
▽ More
We propose and experimentally demonstrate a method of polarization-sensitive quantitative phase imaging using two photo detectors. Instead of recording wide-field interference patterns, finding the modulation patterns maximizing focused intensities in terms of the polarization states enables polarization-dependent quantitative phase imaging without the need for a reference beam and an image sensor. The feasibility of the present method is experimentally validated by reconstructing Jones matrices of various samples including a polystyrene microsphere, a maize starch granule, and a rat retinal nerve fiber layer. Since the present method is simple and sufficiently general, we expect that it may offer solutions for quantitative phase imaging of birefringent materials.
△ Less
Submitted 16 July, 2018;
originally announced July 2018.
-
Low-coherent optical diffraction tomography by angle-scanning illumination
Authors:
KyeoReh Lee,
Seungwoo Shin,
Zahid Yaqoob,
Peter T. C. So,
YongKeun Park
Abstract:
We propose and experimentally demonstrate temporally low-coherent optical diffraction tomography (ODT) based on angle-scanning Mach-Zehnder interferometry. Using a digital micromirror device based on diffractive tilting, we successfully maintain full-field interference of incoherent light during every scan sequence. The ODT reconstruction principles for temporally incoherent illuminations are thor…
▽ More
We propose and experimentally demonstrate temporally low-coherent optical diffraction tomography (ODT) based on angle-scanning Mach-Zehnder interferometry. Using a digital micromirror device based on diffractive tilting, we successfully maintain full-field interference of incoherent light during every scan sequence. The ODT reconstruction principles for temporally incoherent illuminations are thoroughly reviewed and developed. Several limitations of incoherent illumination are also discussed, such as the nondispersive assumption, optical sectioning capacity, and illumination angle limitation. Using the proposed setup and reconstruction algorithms, we successfully demonstrate low-coherent ODT imaging of microspheres, human red blood cells, and eukaryotic cells.
△ Less
Submitted 16 July, 2018;
originally announced July 2018.
