-
Fast, multicolour optical sectioning over extended fields of view by combining interferometric SIM with machine learning
Authors:
Edward N. Ward,
Rebecca M. McClelland,
Jacob R. Lamb,
Roger Rubio-Sánchez,
Charles N. Christensen,
Bismoy Mazumder,
Sofia Kapsiani,
Luca Mascheroni,
Lorenzo Di Michele,
Gabriele S. Kaminski Schierle,
Clemens F. Kaminski
Abstract:
Structured illumination can reject out-of-focus signal from a sample, enabling high-speed and high-contrast imaging over large areas with widefield detection optics. Currently, this optical-sectioning technique is limited by image reconstruction artefacts and the need for sequential imaging of multiple colour channels. We combine multicolour interferometric pattern generation with machine-learning…
▽ More
Structured illumination can reject out-of-focus signal from a sample, enabling high-speed and high-contrast imaging over large areas with widefield detection optics. Currently, this optical-sectioning technique is limited by image reconstruction artefacts and the need for sequential imaging of multiple colour channels. We combine multicolour interferometric pattern generation with machine-learning processing, permitting high-contrast, real-time reconstruction of image data. The method is insensitive to background noise and unevenly phase-stepped illumination patterns. We validate the method in silico and demonstrate its application on diverse specimens, ranging from fixed and live biological cells to synthetic biosystems, imaging at up to 37 Hz across a 44 x 44 $μm^2$ field of view.
△ Less
Submitted 31 October, 2023;
originally announced November 2023.
-
Fourier-space generalized magneto-optical ellipsometry
Authors:
Miguel A. Cascales Sandoval,
A. Hierro-Rodríguez,
D. Sanz-Hernández,
L. Skoric,
C. N. Christensen,
C. Donnelly,
A. Fernández-Pacheco
Abstract:
The magneto-optical Kerr effect (MOKE) is a widely used lab-based technique for the study of thin films and nanostructures, providing magnetic characterization with good spatial and temporal resolutions. Due to the complex coupling of light with a magnetic sample, conventional MOKE magnetometers normally work by selecting a small range of incident wave-vector values, focusing the incident light be…
▽ More
The magneto-optical Kerr effect (MOKE) is a widely used lab-based technique for the study of thin films and nanostructures, providing magnetic characterization with good spatial and temporal resolutions. Due to the complex coupling of light with a magnetic sample, conventional MOKE magnetometers normally work by selecting a small range of incident wave-vector values, focusing the incident light beam to a small spot, and recording the reflected intensity at that angular range by means of photodetectors. This generally provides signals proportional to a mixture of magnetization components, requiring additional methodologies for full vectorial magnetic characterization.
Here, we computationally investigate a Fourier-space MOKE setup, where a focused beam ellipsometer using high numerical aperture optics and a camera detector is employed to simultaneously map the intensity distribution for a wide range of incident and reflected wave-vectors. We employ circularly incident polarized light and no analyzing optics, in combination with a fitting procedure of the light intensity maps to the analytical expression of the Kerr effect under linear approximation. In this way, we are able to retrieve the three unknown components of the magnetization vector as well as the material's optical and magneto-optical constants with high accuracy and short acquisition times, with the possibility of single shot measurements. Fourier MOKE is thus proposed as a powerful method to perform generalized magneto-optical ellipsometry for a wide range of magnetic materials and devices.
△ Less
Submitted 6 December, 2022;
originally announced December 2022.
-
MAI-SIM: interferometric multicolor structured illumination microscopy for everybody
Authors:
Edward N. Ward,
Lisa Hecker,
Charles N. Christensen,
Jacob R Lamb,
Meng Lu,
Luca Mascheroni,
Chyi Wei Chung,
Anna Wang,
Christopher J. Rowlands,
Gabriele S. Kaminski Schierle,
Clemens F. Kaminski
Abstract:
Sub-diffraction resolution, gentle sample illumination, and the possibility to image in multiple colors make Structured Illumination Microscopy (SIM) an imaging technique which is particularly well suited for live cell observations. Here, we present Machine learning Assisted Interferometric-SIM (MAI-SIM), an easy-to-implement method for high speed SIM imaging in multiple colors. The instrument is…
▽ More
Sub-diffraction resolution, gentle sample illumination, and the possibility to image in multiple colors make Structured Illumination Microscopy (SIM) an imaging technique which is particularly well suited for live cell observations. Here, we present Machine learning Assisted Interferometric-SIM (MAI-SIM), an easy-to-implement method for high speed SIM imaging in multiple colors. The instrument is based on an interferometer design in which illumination patterns are generated, rotated, and stepped in phase through movement of a single galvanometric mirror element. The design is robust, flexible, and the pattern generation process works for all wavelengths. We complement the unique properties of interferometric SIM with a machine learning toolbox that is simple and efficient to use and is superior to existing methods for the reconstruction of super-resolved images recorded by the instrument. The framework permits real-time SIM reconstructions to be performed in multiple colors, providing the user with instant visualization of the super-resolved images. We demonstrate the capability of MAI-SIM on live biological samples and capture super-resolution images in multiple colors simultaneously over large fields of view. Finally, we embrace a fully open design philosophy to bring the advantages of MAI-SIM to as many users as possible and provide full details on system design and software.
△ Less
Submitted 1 June, 2022;
originally announced June 2022.
-
Spatio-temporal Vision Transformer for Super-resolution Microscopy
Authors:
Charles N. Christensen,
Meng Lu,
Edward N. Ward,
Pietro Lio,
Clemens F. Kaminski
Abstract:
Structured illumination microscopy (SIM) is an optical super-resolution technique that enables live-cell imaging beyond the diffraction limit. Reconstruction of SIM data is prone to artefacts, which becomes problematic when imaging highly dynamic samples because previous methods rely on the assumption that samples are static. We propose a new transformer-based reconstruction method, VSR-SIM, that…
▽ More
Structured illumination microscopy (SIM) is an optical super-resolution technique that enables live-cell imaging beyond the diffraction limit. Reconstruction of SIM data is prone to artefacts, which becomes problematic when imaging highly dynamic samples because previous methods rely on the assumption that samples are static. We propose a new transformer-based reconstruction method, VSR-SIM, that uses shifted 3-dimensional window multi-head attention in addition to channel attention mechanism to tackle the problem of video super-resolution (VSR) in SIM. The attention mechanisms are found to capture motion in sequences without the need for common motion estimation techniques such as optical flow. We take an approach to training the network that relies solely on simulated data using videos of natural scenery with a model for SIM image formation. We demonstrate a use case enabled by VSR-SIM referred to as rolling SIM imaging, which increases temporal resolution in SIM by a factor of 9. Our method can be applied to any SIM setup enabling precise recordings of dynamic processes in biomedical research with high temporal resolution.
△ Less
Submitted 28 February, 2022;
originally announced March 2022.
-
ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images
Authors:
Charles N. Christensen,
Edward N. Ward,
Pietro Lio,
Clemens F. Kaminski
Abstract:
Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging because it allows a doubling of image resolution at speeds compatible for live-cell imaging. However, the reconstruction of SIM images is often slow and prone to artefacts. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. The model is an end…
▽ More
Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging because it allows a doubling of image resolution at speeds compatible for live-cell imaging. However, the reconstruction of SIM images is often slow and prone to artefacts. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. The model is an end-to-end deep residual neural network that is trained on a simulated data set to be free of common SIM artefacts. ML-SIM is thus robust to noise and irregularities in the illumination patterns of the raw SIM input frames. The reconstruction method is widely applicable and does not require the acquisition of experimental training data. Since the training data are generated from simulations of the SIM process on images from generic libraries the method can be efficiently adapted to specific experimental SIM implementations. The reconstruction quality enabled by our method is compared with traditional SIM reconstruction methods, and we demonstrate advantages in terms of noise, reconstruction fidelity and contrast for both simulated and experimental inputs. In addition, reconstruction of one SIM frame typically only takes ~100ms to perform on PCs with modern Nvidia graphics cards, making the technique compatible with real-time imaging. The full implementation and the trained networks are available at http://ML-SIM.com.
△ Less
Submitted 24 March, 2020;
originally announced March 2020.
-
Simple and robust speckle detection method for fire and heat detection in harsh environments
Authors:
Charles N. Christensen,
Yevgen Zainchkovskyy,
Salvador Barrera-Figueroa,
Antoni Torras-Rosell,
Giorgio Marinelli,
Kim Sommerlund-Thorsen,
Jan Kleven,
Kristian Kleven,
Erlend Voll,
Jan C. Petersen,
Mikael Lassen
Abstract:
Standard laser based fire detection systems are often based on measuring variation of optical signal amplitude. However, mechanical noise interference and loss from dust and steam can obscure the detection signal, resulting in faulty results or inability to detect a potential fire. The presented fire detection technology will allow the detection of fire in harsh and dusty areas, which are prone to…
▽ More
Standard laser based fire detection systems are often based on measuring variation of optical signal amplitude. However, mechanical noise interference and loss from dust and steam can obscure the detection signal, resulting in faulty results or inability to detect a potential fire. The presented fire detection technology will allow the detection of fire in harsh and dusty areas, which are prone to fires, where current systems show limited performance or are unable to operate. It is not the amount of light nor its wavelength that is used for detecting fire, but how the refractive index randomly fluctuates due to the heat convection from the fire. In practical terms this means that light obstruction from ambient dust particles will not be a problem as long as a small fraction of the light is detected and that fires without visible flames can still be detected. The standalone laser system consists of a Linux-based Red Pitaya system, a cheap 650 nm laser diode, and a PIN photo-detector. Laser light propagates through the monitored area and reflects off a retroreflector generating a speckle pattern. Every 3 seconds time traces and frequency noise spectra are measured and 8 descriptors are deduced to identify a potential fire. Both laboratory and factory acceptance tests have been performed with success.
△ Less
Submitted 12 August, 2019;
originally announced September 2019.
