-
Thermally Adaptive Surface Microscopy for brain functional imaging
Authors:
Hadrien L. M. Robert,
Giulia Faini,
Chang F. Liu,
Nadja Rutz,
Anis Aggoun,
Elena Putti,
Jose Garcia-Guirado,
Filippo Del Bene,
Romain Quidant,
Gilles Tessier,
Pascal Berto
Abstract:
Fluorescence microscopes can record the dynamics of living cells with high spatio-temporal resolution in a single plane. However, monitoring rapid and dim fluorescence fluctuations, e.g induced by neuronal activity in the brain, remains challenging for 3D-distributed emitters due to out-of-focus fluorescence background, a restricted photon budget, and the speed limit of conventional scanning syste…
▽ More
Fluorescence microscopes can record the dynamics of living cells with high spatio-temporal resolution in a single plane. However, monitoring rapid and dim fluorescence fluctuations, e.g induced by neuronal activity in the brain, remains challenging for 3D-distributed emitters due to out-of-focus fluorescence background, a restricted photon budget, and the speed limit of conventional scanning systems. Here, we introduce a Thermally Adaptive Surface strategy, capable of simultaneously recording, at camera framerate, the activity of 3D-distributed objects. This innovative microscope leverages on an array of thermally tuneable microlenses that offer low chromatic aberration and high transmission, and can be combined with patterned illumination to provide optical sectioning. We demonstrate its potential in vivo, by simultaneously monitoring fast fluorescent dynamics at different depths in the zebrafish larval brain, at a rate of 0.5 kHz and over a large field of view (360um x 360um).
△ Less
Submitted 9 April, 2025; v1 submitted 7 January, 2025;
originally announced January 2025.
-
PlenoptiCam v1.0: A light-field imaging framework
Authors:
Christopher Hahne,
Amar Aggoun
Abstract:
Light-field cameras play a vital role for rich 3-D information retrieval in narrow range depth sensing applications. The key obstacle in composing light-fields from exposures taken by a plenoptic camera is to computationally calibrate, align and rearrange four-dimensional image data. Several attempts have been proposed to enhance the overall image quality by tailoring pipelines dedicated to partic…
▽ More
Light-field cameras play a vital role for rich 3-D information retrieval in narrow range depth sensing applications. The key obstacle in composing light-fields from exposures taken by a plenoptic camera is to computationally calibrate, align and rearrange four-dimensional image data. Several attempts have been proposed to enhance the overall image quality by tailoring pipelines dedicated to particular plenoptic cameras and improving the consistency across viewpoints at the expense of high computational loads. The framework presented herein advances prior outcomes thanks to its novel micro image scale-space analysis for generic camera calibration independent of the lens specifications and its parallax-invariant, cost-effective viewpoint color equalization from optimal transport theory. Artifacts from the sensor and micro lens grid are compensated in an innovative way to enable superior quality in sub-aperture image extraction, computational refocusing and Scheimpflug rendering with sub-sampling capabilities. Benchmark comparisons using established image metrics suggest that our proposed pipeline outperforms state-of-the-art tool chains in the majority of cases. Results from a Wasserstein distance further show that our color transfer outdoes the existing transport methods. Our algorithms are released under an open-source license, offer cross-platform compatibility with few dependencies and different user interfaces. This makes the reproduction of results and experimentation with plenoptic camera technology convenient for peer researchers, developers, photographers, data scientists and others working in this field.
△ Less
Submitted 25 July, 2021; v1 submitted 14 October, 2020;
originally announced October 2020.
-
Real-Time Refocusing using an FPGA-based Standard Plenoptic Camera
Authors:
Christopher Hahne,
Andrew Lumsdaine,
Amar Aggoun,
Vladan Velisavljevic
Abstract:
Plenoptic cameras are receiving increasing attention in scientific and commercial applications because they capture the entire structure of light in a scene, enabling optical transforms (such as focusing) to be applied computationally after the fact, rather than once and for all at the time a picture is taken. In many settings, real-time interactive performance is also desired, which in turn requi…
▽ More
Plenoptic cameras are receiving increasing attention in scientific and commercial applications because they capture the entire structure of light in a scene, enabling optical transforms (such as focusing) to be applied computationally after the fact, rather than once and for all at the time a picture is taken. In many settings, real-time interactive performance is also desired, which in turn requires significant computational power due to the large amount of data required to represent a plenoptic image. Although GPUs have been shown to provide acceptable performance for real-time plenoptic rendering, their cost and power requirements make them prohibitive for embedded uses (such as in-camera). On the other hand, the computation to accomplish plenoptic rendering is well-structured, suggesting the use of specialized hardware. Accordingly, this paper presents an array of switch-driven Finite Impulse Response (FIR) filters, implemented with FPGA to accomplish high-throughput spatial-domain rendering. The proposed architecture provides a power-efficient rendering hardware design suitable for full-video applications as required in broadcasting or cinematography. A benchmark assessment of the proposed hardware implementation shows that real-time performance can readily be achieved, with a one order of magnitude performance improvement over a GPU implementation and three orders of magnitude performance improvement over a general-purpose CPU implementation.
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
-
Baseline and Triangulation Geometry in a Standard Plenoptic Camera
Authors:
Christopher Hahne,
Amar Aggoun,
Vladan Velisavljevic,
Susanne Fiebig,
Matthias Pesch
Abstract:
In this paper, we demonstrate light field triangulation to determine depth distances and baselines in a plenoptic camera. Advances in micro lenses and image sensors have enabled plenoptic cameras to capture a scene from different viewpoints with sufficient spatial resolution. While object distances can be inferred from disparities in a stereo viewpoint pair using triangulation, this concept remain…
▽ More
In this paper, we demonstrate light field triangulation to determine depth distances and baselines in a plenoptic camera. Advances in micro lenses and image sensors have enabled plenoptic cameras to capture a scene from different viewpoints with sufficient spatial resolution. While object distances can be inferred from disparities in a stereo viewpoint pair using triangulation, this concept remains ambiguous when applied in the case of plenoptic cameras. We present a geometrical light field model allowing the triangulation to be applied to a plenoptic camera in order to predict object distances or specify baselines as desired. It is shown that distance estimates from our novel method match those of real objects placed in front of the camera. Additional benchmark tests with an optical design software further validate the model's accuracy with deviations of less than +-0.33 % for several main lens types and focus settings. A variety of applications in the automotive and robotics field can benefit from this estimation model.
△ Less
Submitted 20 January, 2021; v1 submitted 9 October, 2020;
originally announced October 2020.
-
PlenoptiSign: an optical design tool for plenoptic imaging
Authors:
Christopher Hahne,
Amar Aggoun
Abstract:
Plenoptic imaging enables a light-field to be captured by a single monocular objective lens and an array of micro lenses attached to an image sensor. Metric distances of the light-field's depth planes remain unapparent prior to acquisition. Recent research showed that sampled depth locations rely on the parameters of the system's optical components. This paper presents PlenoptiSign, which implemen…
▽ More
Plenoptic imaging enables a light-field to be captured by a single monocular objective lens and an array of micro lenses attached to an image sensor. Metric distances of the light-field's depth planes remain unapparent prior to acquisition. Recent research showed that sampled depth locations rely on the parameters of the system's optical components. This paper presents PlenoptiSign, which implements these findings as a Python software package to help assist in an experimental or prototyping stage of a plenoptic system.
△ Less
Submitted 1 June, 2020;
originally announced June 2020.
