Showing 1–50 of 50 results for author: Tian, L

Whitened Score Diffusion: A Structured Prior for Imaging Inverse Problems

Authors: Jeffrey Alido, Tongyu Li, Yu Sun, Lei Tian

Abstract: Conventional score-based diffusion models (DMs) may struggle with anisotropic Gaussian diffusion processes due to the required inversion of covariance matrices in the denoising score matching training objective \cite{vincent_connection_2011}. We propose Whitened Score (WS) diffusion models, a novel framework based on stochastic differential equations that learns the Whitened Score function instead… ▽ More Conventional score-based diffusion models (DMs) may struggle with anisotropic Gaussian diffusion processes due to the required inversion of covariance matrices in the denoising score matching training objective \cite{vincent_connection_2011}. We propose Whitened Score (WS) diffusion models, a novel framework based on stochastic differential equations that learns the Whitened Score function instead of the standard score. This approach circumvents covariance inversion, extending score-based DMs by enabling stable training of DMs on arbitrary Gaussian forward noising processes. WS DMs establish equivalence with flow matching for arbitrary Gaussian noise, allow for tailored spectral inductive biases, and provide strong Bayesian priors for imaging inverse problems with structured noise. We experiment with a variety of computational imaging tasks using the CIFAR and CelebA ($64\times64$) datasets and demonstrate that WS diffusion priors trained on anisotropic Gaussian noising processes consistently outperform conventional diffusion priors based on isotropic Gaussian noise. Our code is open-sourced at \href{https://github.com/jeffreyalido/wsdiffusion}{\texttt{github.com/jeffreyalido/wsdiffusion}}. △ Less

Submitted 20 May, 2025; v1 submitted 15 May, 2025; originally announced May 2025.

Empirical Study on Near-Field and Spatial Non-Stationarity Modeling for THz XL-MIMO Channel in Indoor Scenario

Authors: Huixin Xu, Jianhua Zhang, Pan Tang, Hongbo Xing, Chong Han, Lei Tian, Qixing Wang, Guangyi Liu

Abstract: Terahertz (THz) extremely large-scale MIMO (XL-MIMO) is considered a key enabling technology for 6G and beyond due to its advantages such as wide bandwidth and high beam gain. As the frequency and array size increase, users are more likely to fall within the near-field (NF) region, where the far-field plane-wave assumption no longer holds. This also introduces spatial non-stationarity (SnS), as di… ▽ More Terahertz (THz) extremely large-scale MIMO (XL-MIMO) is considered a key enabling technology for 6G and beyond due to its advantages such as wide bandwidth and high beam gain. As the frequency and array size increase, users are more likely to fall within the near-field (NF) region, where the far-field plane-wave assumption no longer holds. This also introduces spatial non-stationarity (SnS), as different antenna elements observe distinct multipath characteristics. Therefore, this paper proposes a THz XL-MIMO channel model that accounts for both NF propagation and SnS, validated using channel measurement data. In this work, we first conduct THz XL-MIMO channel measurements at 100 GHz and 132 GHz using 301- and 531-element ULAs in indoor environments, revealing pronounced NF effects characterized by nonlinear inter-element phase variations, as well as element-dependent delay and angle shifts. Moreover, the SnS phenomenon is observed, arising not only from blockage but also from inconsistent reflection or scattering. Based on these observations, a hybrid NF channel modeling approach combining the scatterer-excited point-source model and the specular reflection model is proposed to capture nonlinear phase variation. For SnS modeling, amplitude attenuation factors (AAFs) are introduced to characterize the continuous variation of path power across the array. By analyzing the statistical distribution and spatial autocorrelation properties of AAFs, a statistical rank-matching-based method is proposed for their generation. Finally, the model is validated using measured data. Evaluation across metrics such as entropy capacity, condition number, spatial correlation, channel gain, Rician K-factor, and RMS delay spread confirms that the proposed model closely aligns with measurements and effectively characterizes the essential features of THz XL-MIMO channels. △ Less

Submitted 14 May, 2025; originally announced May 2025.

A Unified Deterministic Channel Model for Multi-Type RIS with Reflective, Transmissive, and Polarization Operations

Authors: Yuxiang Zhang, Jianhua Zhang, Zhengfu Zhou, Huiwen Gong, Hongbo Xing, Zhiqiang Yuan, Lei Tian, Li Yu, Guangyi Liu, Tao Jiang

Abstract: Reconfigurable Intelligent Surface (RIS) technologies have been considered as a promising enabler for 6G, enabling advantageous control of electromagnetic (EM) propagation. RIS can be categorized into multiple types based on their reflective/transmissive modes and polarization control capabilities, all of which are expected to be widely deployed in practical environments. A reliable RIS channel mo… ▽ More Reconfigurable Intelligent Surface (RIS) technologies have been considered as a promising enabler for 6G, enabling advantageous control of electromagnetic (EM) propagation. RIS can be categorized into multiple types based on their reflective/transmissive modes and polarization control capabilities, all of which are expected to be widely deployed in practical environments. A reliable RIS channel model is essential for the design and development of RIS communication systems. While deterministic modeling approaches such as ray-tracing (RT) offer significant benefits, a unified model that accommodates all RIS types is still lacking. This paper addresses this gap by developing a high-precision deterministic channel model based on RT, supporting multiple RIS types: reflective, transmissive, hybrid, and three polarization operation modes. To achieve this, a unified EM response model for the aforementioned RIS types is developed. The reflection and transmission coefficients of RIS elements are derived using a tensor-based equivalent impedance approach, followed by calculating the scattered fields of the RIS to establish an EM response model. The performance of different RIS types is compared through simulations in typical scenarios. During this process, passive and lossless constraints on the reflection and transmission coefficients are incorporated to ensure fairness in the performance evaluation. Simulation results validate the framework's accuracy in characterizing the RIS channel, and specific cases tailored for dual-polarization independent control and polarization rotating RISs are highlighted as insights for their future deployment. This work can be helpful for the evaluation and optimization of RIS-enabled wireless communication systems. △ Less

Submitted 11 May, 2025; originally announced May 2025.

Comments: Submitted to IEEE Transactions on Vehicular Technology

High-Resolution Multipath Angle Estimation Based on Power-Angle-Delay Profile for Directional Scanning Sounding

Authors: Huixin Xu, Jianhua Zhang, Pan Tang, Hongbo Xing, Lei Tian, Qixing Wang

Abstract: Directional scanning sounding (DSS) has become widely adopted for high-frequency channel measurements because it effectively compensates for severe path loss. However, the resolution of existing multipath component (MPC) angle estimation methods is constrained by the DSS angle sampling interval. Therefore, this communication proposes a high-resolution MPC angle estimation method based on power-ang… ▽ More Directional scanning sounding (DSS) has become widely adopted for high-frequency channel measurements because it effectively compensates for severe path loss. However, the resolution of existing multipath component (MPC) angle estimation methods is constrained by the DSS angle sampling interval. Therefore, this communication proposes a high-resolution MPC angle estimation method based on power-angle-delay profile (PADP) for DSS. By exploiting the mapping relationship between the power difference of adjacent angles in the PADP and MPC offset angle, the resolution of MPC angle estimation is refined, significantly enhancing the accuracy of MPC angle and amplitude estimation without increasing measurement complexity. Numerical simulation results demonstrate that the proposed method reduces the mean squared estimation errors of angle and amplitude by one order of magnitude compared to traditional omnidirectional synthesis methods. Furthermore, the estimation errors approach the Cramér-Rao Lower Bounds (CRLBs) derived for wideband DSS, thereby validating its superior performance in MPC angle and amplitude estimation. Finally, experiments conducted in an indoor scenario at 37.5 GHz validate the excellent performance of the proposed method in practical applications. △ Less

Submitted 17 April, 2025; originally announced April 2025.

Research and Experimental Validation for 3GPP ISAC Channel Modeling Standardization

Authors: Yuxiang Zhang, Jianhua Zhang, Jiwei Zhang, Yuanpeng Pei, Yameng Liu, Lei Tian, Tao Jiang, Guangyi Liu

Abstract: Integrated Sensing and Communication (ISAC) is considered a key technology in 6G networks. An accurate sensing channel model is crucial for the design and sensing performance evaluation of ISAC systems. The widely used Geometry-Based Stochastic Model (GBSM), typically applied in standardized channel modeling, mainly focuses on the statistical fading characteristics of the channel. However, it fail… ▽ More Integrated Sensing and Communication (ISAC) is considered a key technology in 6G networks. An accurate sensing channel model is crucial for the design and sensing performance evaluation of ISAC systems. The widely used Geometry-Based Stochastic Model (GBSM), typically applied in standardized channel modeling, mainly focuses on the statistical fading characteristics of the channel. However, it fails to capture the characteristics of targets in ISAC systems, such as their positions and velocities, as well as the impact of the targets on the background. To address this issue, this paper proposes an extended GBSM (E-GBSM) sensing channel model that incorporates newly discovered channel characteristics into a unified modeling framework. In this framework, the sensing channel is divided into target and background channels. For the target channel, the model introduces a concatenated modeling approach, while for the background channel, a parameter called the power control factor is introduced to assess impact of the target on the background channel, making the modeling framework applicable to both mono-static and bi-static sensing modes. To validate the proposed model's effectiveness, measurements of target and background channels are conducted in both indoor and outdoor scenarios, covering various sensing targets such as metal plates, reconfigurable intelligent surfaces, human bodies, UAVs, and vehicles. The experimental results provide important theoretical support and empirical data for the standardization of ISAC channel modeling. △ Less

Submitted 13 April, 2025; originally announced April 2025.

Comments: 12 pages, 10 figures

A Survey of New Mid-Band/FR3 for 6G: Channel Measurement, Characterization and Modeling in Outdoor Environment

Authors: Haiyang Miao, Jianhua Zhang, Pan Tang, Jie Meng, Qi Zhen, Ximan Liu, Enrui Liu, Peijie Liu, Lei Tian, Guangyi Liu

Abstract: The new mid-band (6-24 GHz) has attracted significant attention from both academia and industry, which is the spectrum with continuous bandwidth that combines the coverage benefits of low frequency with the capacity advantages of high frequency. Since outdoor environments represent the primary application scenario for mobile communications, this paper presents the first comprehensive review and su… ▽ More The new mid-band (6-24 GHz) has attracted significant attention from both academia and industry, which is the spectrum with continuous bandwidth that combines the coverage benefits of low frequency with the capacity advantages of high frequency. Since outdoor environments represent the primary application scenario for mobile communications, this paper presents the first comprehensive review and summary of multi-scenario and multi-frequency channel characteristics based on extensive outdoor new mid-band channel measurement data, including UMa, UMi, and O2I. Specifically, a survey of the progress of the channel characteristics is presented, such as path loss, delay spread, angular spread, channel sparsity, capacity and near-field spatial non-stationary characteristics. Then, considering that satellite communication will be an important component of future communication systems, we examine the impact of clutter loss in air-ground communications. Our analysis of the frequency dependence of mid-band clutter loss suggests that its impact is not significant. Additionally, given that penetration loss is frequency-dependent, we summarize its variation within the FR3 band. Based on experimental results, comparisons with the standard model reveal that while the 3GPP TR 38.901 model remains a useful reference for penetration loss in wood and glass, it shows significant deviations for concrete and glass, indicating the need for further refinement. In summary, the findings of this survey provide both empirical data and theoretical support for the deployment of mid-band in future communication systems, as well as guidance for optimizing mid-band base station deployment in the outdoor environment. This survey offers the reference for improving standard models and advancing channel modeling. △ Less

Submitted 9 April, 2025; originally announced April 2025.

A Novel Environment Object Modeling Method for Vehicular ISAC Scenarios

Authors: Hanyuan Jiang, Yuxiang Zhang, Yameng Liu, Jianhua Zhang, Lei Tian, Tao Jiang

Abstract: Integrated Sensing and Communication (ISAC), as a fundamental technology of 6G, empowers Vehicle-to-Everything (V2X) systems with enhanced sensing capabilities. One of its promising applications is the reliance on constructed maps for vehicle positioning. Traditional positioning methods primarily rely on Line-of-Sight (LOS), but in urban vehicular scenarios, obstructions often result in predominan… ▽ More Integrated Sensing and Communication (ISAC), as a fundamental technology of 6G, empowers Vehicle-to-Everything (V2X) systems with enhanced sensing capabilities. One of its promising applications is the reliance on constructed maps for vehicle positioning. Traditional positioning methods primarily rely on Line-of-Sight (LOS), but in urban vehicular scenarios, obstructions often result in predominantly Non-Line-of-Sight (NLOS) conditions. Existing research indicates that NLOS paths, characterized by one-bounce reflection on building walls with determined delay and angle, can support sensing and positioning. However, experimental validation remains insufficient. To address this gap, channel measurements are conducted in an urban street to explore the existence of strong reflected paths in the presence of a vehicle target. The results show significant power contribution from NLOS paths, with large Environmental Objects (EOs) playing a key role in shaping NLOS propagation. Then, a novel model for EO reflection is proposed to extend the Geometry-Based Stochastic Model (GBSM) for ISAC channel standardization. Simulation results validate the model's ability to capture EO's power and position characteristics, showing that higher EO-reflected power and closer distance to Rx reduce Delay Spread (DS), which is more favorable for positioning. This model provides theoretical guidance and empirical support for ISAC positioning algorithms and system design in vehicular scenarios. △ Less

Submitted 15 March, 2025; originally announced March 2025.

Comments: 6 pages, 8 figures

Reference-Free 3D Reconstruction of Brain Dissection Photographs with Machine Learning

Authors: Lin Tian, Sean I. Young, Jonathan Williams Ramirez, Dina Zemlyanker, Lucas Jacob Deden Binder, Rogeny Herisse, Theresa R. Connors, Derek H. Oakley, Bradley T. Hyman, Oula Puonti, Matthew S. Rosen, Juan Eugenio Iglesias

Abstract: Correlation of neuropathology with MRI has the potential to transfer microscopic signatures of pathology to invivo scans. Recently, a classical registration method has been proposed, to build these correlations from 3D reconstructed stacks of dissection photographs, which are routinely taken at brain banks. These photographs bypass the need for exvivo MRI, which is not widely accessible. However,… ▽ More Correlation of neuropathology with MRI has the potential to transfer microscopic signatures of pathology to invivo scans. Recently, a classical registration method has been proposed, to build these correlations from 3D reconstructed stacks of dissection photographs, which are routinely taken at brain banks. These photographs bypass the need for exvivo MRI, which is not widely accessible. However, this method requires a full stack of brain slabs and a reference mask (e.g., acquired with a surface scanner), which severely limits the applicability of the technique. Here we propose RefFree, a dissection photograph reconstruction method without external reference. RefFree is a learning approach that estimates the 3D coordinates in the atlas space for every pixel in every photograph; simple least-squares fitting can then be used to compute the 3D reconstruction. As a by-product, RefFree also produces an atlas-based segmentation of the reconstructed stack. RefFree is trained on synthetic photographs generated from digitally sliced 3D MRI data, with randomized appearance for enhanced generalization ability. Experiments on simulated and real data show that RefFree achieves performance comparable to the baseline method without an explicit reference while also enabling reconstruction of partial stacks. Our code is available at https://github.com/lintian-a/reffree. △ Less

Submitted 12 March, 2025; originally announced March 2025.

Cascaded channel modeling and experimental validation for RIS assisted communication system

Authors: Jiwei Zhang, Yuxiang Zhang, Tao Jiang, Huiwen Gong, Hongbo Xing, Lei Tian

Abstract: Reconfigurable Intelligent Surface (RIS) is considered as a promising technology for 6G due to its ability to actively modify the electromagnetic propagation environment. Accurate channel modeling is essential for the design and evaluation of RIS assisted communication systems. Most current research models the RIS channel as a cascade of Tx-RIS and RIS-Rx sub-channels. However, most validation eff… ▽ More Reconfigurable Intelligent Surface (RIS) is considered as a promising technology for 6G due to its ability to actively modify the electromagnetic propagation environment. Accurate channel modeling is essential for the design and evaluation of RIS assisted communication systems. Most current research models the RIS channel as a cascade of Tx-RIS and RIS-Rx sub-channels. However, most validation efforts regarding this assumption focus on large-scale path loss. To further explore this, in this paper, we derive and extend a convolution expression of RIS cascaded channel model based on the previously proposed Geometry-based Stochastic Model (GBSM)-based RIS cascaded channels. This model follows the 3GPP standard framework and leverages parameters such as angles, delays, and path powers defined in the GBSM model to more accurately reflect the smallscale characteristics of RIS multipath cascades. To verify the accuracy of this model, we conduct measurements of the TxRIS-Rx channel, Tx-RIS, and RIS-Rx sub-channels in a factory environment at 6.9 GHz, using the measured data to demonstrate the models validity and applicability in real-world scenarios. Validation with measured data shows that the proposed model accurately describes the characteristics of the RIS cascaded channel in terms of delay, angle, and power in complex multipath environments, providing important references for the design and deployment of RIS systems. △ Less

Submitted 10 December, 2024; originally announced December 2024.

BUPTCMCC-6G-CMG+: A GBSM-Based ISAC Standard Channel Model Generator

Authors: Changsheng Zhao, Jianhua Zhang, Yuxiang Zhang, Lei Tian, Heng Wang, Hanyuan Jiang, Yameng Liu, Wenjun Chen, Tao Jiang, Guangyi Liu

Abstract: Integrated sensing and communication (ISAC) has been recognized as the key technology in the vision of the sixth generation (6G) era. With the emergence of new concepts in mobile communications, the channel model is the prerequisite for system design and performance evaluation. Currently, 3GPP Release 19 is advancing the standardization of ISAC channel models. Nevertheless, a unified modeling fram… ▽ More Integrated sensing and communication (ISAC) has been recognized as the key technology in the vision of the sixth generation (6G) era. With the emergence of new concepts in mobile communications, the channel model is the prerequisite for system design and performance evaluation. Currently, 3GPP Release 19 is advancing the standardization of ISAC channel models. Nevertheless, a unified modeling framework has yet to be established. This paper provides a simulation diagram of ISAC channel modeling extended based on the Geometry-Based Stochastic Model (GBSM), compatible with existing 5G channel models and the latest progress in the 3rd Generation Partnership Project (3GPP) standardization. We first introduce the progress of the ISAC channel model standardization in general. Then, a concatenated channel modeling approach is presented considering the team's standardization proposals, which is implemented on the BUPTCMCC-6G-CMG+ channel model generator. We validated the model in cumulative probability density function (CDF) in statistical extension of angle and delay, and radar cross section (RCS). Simulation results show that the proposed model can realistically characterize the feature of channel concatenation and RCS within the ISAC channel. △ Less

Submitted 18 April, 2025; v1 submitted 22 September, 2024; originally announced September 2024.

Comments: 15 pages,7 fiures,5 tables

Journal ref: Sci China Inf Sci, 2025, 68(5): 150304

Self-Supervised Elimination of Non-Independent Noise in Hyperspectral Imaging

Authors: Guangrui Ding, Chang Liu, Jiaze Yin, Xinyan Teng, Yuying Tan, Hongjian He, Haonan Lin, Lei Tian, Ji-Xin Cheng

Abstract: Hyperspectral imaging has been widely used for spectral and spatial identification of target molecules, yet often contaminated by sophisticated noise. Current denoising methods generally rely on independent and identically distributed noise statistics, showing corrupted performance for non-independent noise removal. Here, we demonstrate Self-supervised PErmutation Noise2noise Denoising (SPEND), a… ▽ More Hyperspectral imaging has been widely used for spectral and spatial identification of target molecules, yet often contaminated by sophisticated noise. Current denoising methods generally rely on independent and identically distributed noise statistics, showing corrupted performance for non-independent noise removal. Here, we demonstrate Self-supervised PErmutation Noise2noise Denoising (SPEND), a deep learning denoising architecture tailor-made for removing non-independent noise from a single hyperspectral image stack. We utilize hyperspectral stimulated Raman scattering and mid-infrared photothermal microscopy as the testbeds, where the noise is spatially correlated and spectrally varied. Based on single hyperspectral images, SPEND permutates odd and even spectral frames to generate two stacks with identical noise properties, and uses the pairs for efficient self-supervised noise-to-noise training. SPEND achieved an 8-fold signal-to-noise improvement without having access to the ground truth data. SPEND enabled accurate mapping of low concentration biomolecules in both fingerprint and silent regions, demonstrating its robustness in sophisticated cellular environments. △ Less

Submitted 15 September, 2024; originally announced September 2024.

multiGradICON: A Foundation Model for Multimodal Medical Image Registration

Authors: Basar Demir, Lin Tian, Thomas Hastings Greer, Roland Kwitt, Francois-Xavier Vialard, Raul San Jose Estepar, Sylvain Bouix, Richard Jarrett Rushmore, Ebrahim Ebrahim, Marc Niethammer

Abstract: Modern medical image registration approaches predict deformations using deep networks. These approaches achieve state-of-the-art (SOTA) registration accuracy and are generally fast. However, deep learning (DL) approaches are, in contrast to conventional non-deep-learning-based approaches, anatomy-specific. Recently, a universal deep registration approach, uniGradICON, has been proposed. However, u… ▽ More Modern medical image registration approaches predict deformations using deep networks. These approaches achieve state-of-the-art (SOTA) registration accuracy and are generally fast. However, deep learning (DL) approaches are, in contrast to conventional non-deep-learning-based approaches, anatomy-specific. Recently, a universal deep registration approach, uniGradICON, has been proposed. However, uniGradICON focuses on monomodal image registration. In this work, we therefore develop multiGradICON as a first step towards universal *multimodal* medical image registration. Specifically, we show that 1) we can train a DL registration model that is suitable for monomodal *and* multimodal registration; 2) loss function randomization can increase multimodal registration accuracy; and 3) training a model with multimodal data helps multimodal generalization. Our code and the multiGradICON model are available at https://github.com/uncbiag/uniGradICON. △ Less

Submitted 7 February, 2025; v1 submitted 31 July, 2024; originally announced August 2024.

Analysis of Near-Field Effects, Spatial Non-Stationary Characteristics Based on 11-15 GHz Channel Measurement in Indoor Scenario

Authors: Haiyang Miao, Pan Tang, Weirang Zuo, Qi Wei, Lei Tian, Jianhua Zhang

Abstract: In the sixth-generation (6G), with the further expansion of array element number and frequency bands, the wireless communications are expected to operate in the near-field region. The near-field radio communications (NFRC) will become crucial in 6G communication systems. The new mid-band (6-24 GHz) is the 6G potential candidate spectrum. In this paper, we will investigate the channel measurements… ▽ More In the sixth-generation (6G), with the further expansion of array element number and frequency bands, the wireless communications are expected to operate in the near-field region. The near-field radio communications (NFRC) will become crucial in 6G communication systems. The new mid-band (6-24 GHz) is the 6G potential candidate spectrum. In this paper, we will investigate the channel measurements and characteristics for the emerging NFRC. First, the near-field spherical-wave signal model is derived in detail, and the stationary interval (SI) division method is discussed based on the channel statistical properties. Then, the influence of line-of-sight (LOS) and obstructed-LOS (OLOS) environments on the near-field effects and spatial non-stationary (SnS) characteristic are explored based on the near-field channel measurements at 11-15 GHz band. We hope that this work will give some reference to the NFRC research. △ Less

Submitted 19 April, 2024; originally announced May 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2404.17270

Empirical Studies of Propagation Characteristics and Modeling Based on XL-MIMO Channel Measurement: From Far-Field to Near-Field

Authors: Haiyang Miao, Jianhua Zhang, Pan Tang, Lei Tian, Weirang Zuo, Qi Wei, Guangyi Liu

Abstract: In the sixth-generation (6G), the extremely large-scale multiple-input-multiple-output (XL-MIMO) is considered a promising enabling technology. With the further expansion of array element number and frequency bands, near-field effects will be more likely to occur in 6G communication systems. The near-field radio communications (NFRC) will become crucial in 6G communication systems. It is known tha… ▽ More In the sixth-generation (6G), the extremely large-scale multiple-input-multiple-output (XL-MIMO) is considered a promising enabling technology. With the further expansion of array element number and frequency bands, near-field effects will be more likely to occur in 6G communication systems. The near-field radio communications (NFRC) will become crucial in 6G communication systems. It is known that the channel research is very important for the development and performance evaluation of the communication systems. In this paper, we will systematically investigate the channel measurements and modeling for the emerging NFRC. First, the principle design of massive MIMO channel measurement platform are solved. Second, an indoor XL-MIMO channel measurement campaign with 1600 array elements is conducted, and the channel characteristics are extracted and validated in the near-field region. Then, the outdoor XL-MIMO channel measurement campaign with 320 array elements is conducted, and the channel characteristics are extracted and modeled from near-field to far-field (NF-FF) region. The spatial non-stationary characteristics of angular spread at the transmitting end are more important in modeling. We hope that this work will give some reference to the near-field and far-field research for 6G. △ Less

Submitted 26 April, 2024; originally announced April 2024.

The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report

Authors: Bin Ren, Yawei Li, Nancy Mehta, Radu Timofte, Hongyuan Yu, Cheng Wan, Yuxin Hong, Bingnan Han, Zhuoyuan Wu, Yajun Zou, Yuqing Liu, Jizhe Li, Keji He, Chao Fan, Heng Zhang, Xiaolin Zhang, Xuanwu Yin, Kunlong Zuo, Bohao Liao, Peizhe Xia, Long Peng, Zhibo Du, Xin Di, Wangkai Li, Yang Wang , et al. (109 additional authors not shown)

Abstract: This paper provides a comprehensive review of the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions and their outcomes. The task of this challenge is to super-resolve an input image with a magnification factor of x4 based on pairs of low and corresponding high-resolution images. The primary objective is to develop networks that optimize various aspects such… ▽ More This paper provides a comprehensive review of the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions and their outcomes. The task of this challenge is to super-resolve an input image with a magnification factor of x4 based on pairs of low and corresponding high-resolution images. The primary objective is to develop networks that optimize various aspects such as runtime, parameters, and FLOPs, while still maintaining a peak signal-to-noise ratio (PSNR) of approximately 26.90 dB on the DIV2K_LSDIR_valid dataset and 26.99 dB on the DIV2K_LSDIR_test dataset. In addition, this challenge has 4 tracks including the main track (overall performance), sub-track 1 (runtime), sub-track 2 (FLOPs), and sub-track 3 (parameters). In the main track, all three metrics (ie runtime, FLOPs, and parameter count) were considered. The ranking of the main track is calculated based on a weighted sum-up of the scores of all other sub-tracks. In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking. In sub-track 2, the number of FLOPs was considered. The score calculated based on the corresponding FLOPs was used to determine the ranking. In sub-track 3, the number of parameters was considered. The score calculated based on the corresponding parameters was used to determine the ranking. RLFN is set as the baseline for efficiency measurement. The challenge had 262 registered participants, and 34 teams made valid submissions. They gauge the state-of-the-art in efficient single-image super-resolution. To facilitate the reproducibility of the challenge and enable other researchers to build upon these findings, the code and the pre-trained model of validated solutions are made publicly available at https://github.com/Amazingren/NTIRE2024_ESR/. △ Less

Submitted 25 June, 2024; v1 submitted 16 April, 2024; originally announced April 2024.

Comments: The report paper of NTIRE2024 Efficient Super-resolution, accepted by CVPRW2024

Wide-Field, High-Resolution Reconstruction in Computational Multi-Aperture Miniscope Using a Fourier Neural Network

Authors: Qianwan Yang, Ruipeng Guo, Guorong Hu, Yujia Xue, Yunzhe Li, Lei Tian

Abstract: Traditional fluorescence microscopy is constrained by inherent trade-offs among resolution, field-of-view, and system complexity. To navigate these challenges, we introduce a simple and low-cost computational multi-aperture miniature microscope, utilizing a microlens array for single-shot wide-field, high-resolution imaging. Addressing the challenges posed by extensive view multiplexing and non-lo… ▽ More Traditional fluorescence microscopy is constrained by inherent trade-offs among resolution, field-of-view, and system complexity. To navigate these challenges, we introduce a simple and low-cost computational multi-aperture miniature microscope, utilizing a microlens array for single-shot wide-field, high-resolution imaging. Addressing the challenges posed by extensive view multiplexing and non-local, shift-variant aberrations in this device, we present SV-FourierNet, a novel multi-channel Fourier neural network. SV-FourierNet facilitates high-resolution image reconstruction across the entire imaging field through its learned global receptive field. We establish a close relationship between the physical spatially-varying point-spread functions and the network's learned effective receptive field. This ensures that SV-FourierNet has effectively encapsulated the spatially-varying aberrations in our system, and learned a physically meaningful function for image reconstruction. Training of SV-FourierNet is conducted entirely on a physics-based simulator. We showcase wide-field, high-resolution video reconstructions on colonies of freely moving C. elegans and imaging of a mouse brain section. Our computational multi-aperture miniature microscope, augmented with SV-FourierNet, represents a major advancement in computational microscopy and may find broad applications in biomedical research and other fields requiring compact microscopy solutions. △ Less

Submitted 30 May, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

A Robust Deep Learning Method with Uncertainty Estimation for the Pathological Classification of Renal Cell Carcinoma based on CT Images

Authors: Ni Yao, Hang Hu, Kaicong Chen, Chen Zhao, Yuan Guo, Boya Li, Jiaofen Nan, Yanting Li, Chuang Han, Fubao Zhu, Weihua Zhou, Li Tian

Abstract: Objectives To develop and validate a deep learning-based diagnostic model incorporating uncertainty estimation so as to facilitate radiologists in the preoperative differentiation of the pathological subtypes of renal cell carcinoma (RCC) based on CT images. Methods Data from 668 consecutive patients, pathologically proven RCC, were retrospectively collected from Center 1. By using five-fold cross… ▽ More Objectives To develop and validate a deep learning-based diagnostic model incorporating uncertainty estimation so as to facilitate radiologists in the preoperative differentiation of the pathological subtypes of renal cell carcinoma (RCC) based on CT images. Methods Data from 668 consecutive patients, pathologically proven RCC, were retrospectively collected from Center 1. By using five-fold cross-validation, a deep learning model incorporating uncertainty estimation was developed to classify RCC subtypes into clear cell RCC (ccRCC), papillary RCC (pRCC), and chromophobe RCC (chRCC). An external validation set of 78 patients from Center 2 further evaluated the model's performance. Results In the five-fold cross-validation, the model's area under the receiver operating characteristic curve (AUC) for the classification of ccRCC, pRCC, and chRCC was 0.868 (95% CI: 0.826-0.923), 0.846 (95% CI: 0.812-0.886), and 0.839 (95% CI: 0.802-0.88), respectively. In the external validation set, the AUCs were 0.856 (95% CI: 0.838-0.882), 0.787 (95% CI: 0.757-0.818), and 0.793 (95% CI: 0.758-0.831) for ccRCC, pRCC, and chRCC, respectively. Conclusions The developed deep learning model demonstrated robust performance in predicting the pathological subtypes of RCC, while the incorporated uncertainty emphasized the importance of understanding model confidence, which is crucial for assisting clinical decision-making for patients with renal tumors. Clinical relevance statement Our deep learning approach, integrated with uncertainty estimation, offers clinicians a dual advantage: accurate RCC subtype predictions complemented by diagnostic confidence references, promoting informed decision-making for patients with RCC. △ Less

Submitted 12 November, 2023; v1 submitted 1 November, 2023; originally announced November 2023.

Comments: 16 pages, 6 figures

EventLFM: Event Camera integrated Fourier Light Field Microscopy for Ultrafast 3D imaging

Authors: Ruipeng Guo, Qianwan Yang, Andrew S. Chang, Guorong Hu, Joseph Greene, Christopher V. Gabel, Sixian You, Lei Tian

Abstract: Ultrafast 3D imaging is indispensable for visualizing complex and dynamic biological processes. Conventional scanning-based techniques necessitate an inherent trade-off between acquisition speed and space-bandwidth product (SBP). Emerging single-shot 3D wide-field techniques offer a promising alternative but are bottlenecked by the synchronous readout constraints of conventional CMOS systems, thus… ▽ More Ultrafast 3D imaging is indispensable for visualizing complex and dynamic biological processes. Conventional scanning-based techniques necessitate an inherent trade-off between acquisition speed and space-bandwidth product (SBP). Emerging single-shot 3D wide-field techniques offer a promising alternative but are bottlenecked by the synchronous readout constraints of conventional CMOS systems, thus restricting data throughput to maintain high SBP at limited frame rates. To address this, we introduce EventLFM, a straightforward and cost-effective system that overcomes these challenges by integrating an event camera with Fourier light field microscopy (LFM), a state-of-the-art single-shot 3D wide-field imaging technique. The event camera operates on a novel asynchronous readout architecture, thereby bypassing the frame rate limitations inherent to conventional CMOS systems. We further develop a simple and robust event-driven LFM reconstruction algorithm that can reliably reconstruct 3D dynamics from the unique spatiotemporal measurements captured by EventLFM. Experimental results demonstrate that EventLFM can robustly reconstruct fast-moving and rapidly blinking 3D fluorescent samples at kHz frame rates. Furthermore, we highlight EventLFM's capability for imaging of blinking neuronal signals in scattering mouse brain tissues and 3D tracking of GFP-labeled neurons in freely moving C. elegans. We believe that the combined ultrafast speed and large 3D SBP offered by EventLFM may open up new possibilities across many biomedical applications. △ Less

Submitted 3 April, 2024; v1 submitted 1 October, 2023; originally announced October 2023.

Local Conditional Neural Fields for Versatile and Generalizable Large-Scale Reconstructions in Computational Imaging

Authors: Hao Wang, Jiabei Zhu, Yunzhe Li, QianWan Yang, Lei Tian

Abstract: Deep learning has transformed computational imaging, but traditional pixel-based representations limit their ability to capture continuous, multiscale details of objects. Here we introduce a novel Local Conditional Neural Fields (LCNF) framework, leveraging a continuous implicit neural representation to address this limitation. LCNF enables flexible object representation and facilitates the recons… ▽ More Deep learning has transformed computational imaging, but traditional pixel-based representations limit their ability to capture continuous, multiscale details of objects. Here we introduce a novel Local Conditional Neural Fields (LCNF) framework, leveraging a continuous implicit neural representation to address this limitation. LCNF enables flexible object representation and facilitates the reconstruction of multiscale information. We demonstrate the capabilities of LCNF in solving the highly ill-posed inverse problem in Fourier ptychographic microscopy (FPM) with multiplexed measurements, achieving robust, scalable, and generalizable large-scale phase retrieval. Unlike traditional neural fields frameworks, LCNF incorporates a local conditional representation that promotes model generalization, learning multiscale information, and efficient processing of large-scale imaging data. By combining an encoder and a decoder conditioned on a learned latent vector, LCNF achieves versatile continuous-domain super-resolution image reconstruction. We demonstrate accurate reconstruction of wide field-of-view, high-resolution phase images using only a few multiplexed measurements. LCNF robustly captures the continuous object priors and eliminates various phase artifacts, even when it is trained on imperfect datasets. The framework exhibits strong generalization, reconstructing diverse objects even with limited training data. Furthermore, LCNF can be trained on a physics simulator using natural images and successfully applied to experimental measurements on biological samples. Our results highlight the potential of LCNF for solving large-scale inverse problems in computational imaging, with broad applicability in various deep-learning-based techniques. △ Less

Submitted 22 July, 2023; v1 submitted 12 July, 2023; originally announced July 2023.

Channel Measurement, Modeling, and Simulation for 6G: A Survey and Tutorial

Authors: Jianhua Zhang, Jiaxin Lin, Pan Tang, Yuxiang Zhang, Huixin Xu, Tianyang Gao, Haiyang Miao, Zeyong Chai, Zhengfu Zhou, Yi Li, Huiwen Gong, Yameng Liu, Zhiqiang Yuan, Lei Tian, Shaoshi Yang, Liang Xia, Guangyi Liu, Ping Zhang

Abstract: The sixth generation (6G) mobile communications have attracted substantial attention in the global research community of information and communication technologies (ICT). 6G systems are expected to support not only extended 5G usage scenarios, but also new usage scenarios, such as integrated sensing and communication (ISAC), integrated artificial intelligence (AI) and communication, and communicat… ▽ More The sixth generation (6G) mobile communications have attracted substantial attention in the global research community of information and communication technologies (ICT). 6G systems are expected to support not only extended 5G usage scenarios, but also new usage scenarios, such as integrated sensing and communication (ISAC), integrated artificial intelligence (AI) and communication, and communication and ubiquitous connectivity. To realize this goal, channel characteristics must be comprehensively studied and properly exploited, so as to promote the design, standardization, and optimization of 6G systems. In this paper, we first summarize the requirements and challenges in 6G channel research. Our focus is on channels for five promising technologies enabling 6G, including terahertz (THz), extreme MIMO (E-MIMO), ISAC, reconfigurable intelligent surface (RIS), and space-air-ground integrated network (SAGIN). Then, a survey of the progress of the 6G channel research regarding the above five promising technologies is presented in terms of the latest measurement campaigns, new characteristics, modeling methods, and research prospects. Moreover, a tutorial on the 6G channel simulations is presented. We introduce the BUPTCMCCCMG-IMT2030, a 6G link-level channel simulator, developed based on the ITU/3GPP 3D geometry-based stochastic model (GBSM) methodology. The simulator supports the channel simulation of the aforementioned 6G potential technologies. To facilitate the use of the simulator, the tutorial encompasses the design framework, user guidelines, and application examples. This paper offers in-depth, hands-on insights into the best practices of channel measurements, modeling, and simulations for the evaluation of 6G technologies, the development of 6G standards, and the implementation and optimization of 6G systems. △ Less

Submitted 10 March, 2025; v1 submitted 26 May, 2023; originally announced May 2023.

Comments: 41 pages,52 figures

3GPP-Like GBSM THz Channel Characterization, Modeling, and Simulation Based on Experimental Observations

Authors: Zhaowei Chang, Jianhua Zhang, Pan Tang, Lei Tian, Hao Jiang, Ximan Liu, and Guangyi Liu

Abstract: Terahertz (THz) communication is envisioned as one of the possible technologies for the sixth-generation (6G) communication system due to its rich spectrum. To evaluate the performance of THz communication, it is essential to propose THz channel models within the common framework of the geometry-based stochastic model (GBSM) in the 3rd Generation Partnership Project (3GPP). This paper focuses on T… ▽ More Terahertz (THz) communication is envisioned as one of the possible technologies for the sixth-generation (6G) communication system due to its rich spectrum. To evaluate the performance of THz communication, it is essential to propose THz channel models within the common framework of the geometry-based stochastic model (GBSM) in the 3rd Generation Partnership Project (3GPP). This paper focuses on THz channel modeling and simulation by a 3GPP-like GBSM, based on channel measurements. We first present channel measurements at 100 GHz in an indoor office scenario and 132 GHz in an urban microcellular scenario. Subsequently, channel characteristics such as path loss, delay spread, angle spread, K-factor, cluster characteristic, cross-correlations, and correlation distances are obtained and analyzed based on channel measurement. Additionally, the channel characteristics are modeled by the statistical distribution of 3GPP channel models, which can be used to reconstruct the channel impulse response (CIR). Furthermore, these obtained distributions are studied referring to the default models in the 3GPP, revealing the channel sparsity in the THz channel. For instance, in the case of line-of-sight links in the indoor office, the mean of the measured cluster number is 4 while the default value is 15. Finally, we propose the THz channel model and its simulation framework to reconstruct CIRs based on the obtained models, which aim at characterizing the sparser THz channels. The obvious channel sparsity is characterized in both scenarios, as the Gini factors obtained by the proposed model only have the maximum deviation of 0.04 for those of the measurement. Overall, these findings are helpful in understanding and modeling the THz channel, facilitating the application of THz communication techniques for 6G. △ Less

Submitted 26 July, 2024; v1 submitted 24 May, 2023; originally announced May 2023.

Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network

Authors: Jeffrey Alido, Joseph Greene, Yujia Xue, Guorong Hu, Yunzhe Li, Mitchell Gilmore, Kevin J. Monk, Brett T. DiBenedictis, Ian G. Davison, Lei Tian

Abstract: Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacer… ▽ More Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacerbates the condition of the inverse problem. Here, we develop a scattering simulator that models low-contrast target signals buried in heterogeneous strong background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). We apply this network to our previously developed Computational Miniature Mesoscope and demonstrate the robustness of our deep learning algorithm on scattering phantoms with different scattering conditions. The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model's generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking. △ Less

Submitted 8 December, 2023; v1 submitted 22 March, 2023; originally announced March 2023.

Roadmap on Deep Learning for Microscopy

Authors: Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C. D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu , et al. (50 additional authors not shown)

Abstract: Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the… ▽ More Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences. △ Less

Submitted 7 March, 2023; originally announced March 2023.

I Know Your Feelings Before You Do: Predicting Future Affective Reactions in Human-Computer Dialogue

Authors: Yuanchao Li, Koji Inoue, Leimin Tian, Changzeng Fu, Carlos Ishi, Hiroshi Ishiguro, Tatsuya Kawahara, Catherine Lai

Abstract: Current Spoken Dialogue Systems (SDSs) often serve as passive listeners that respond only after receiving user speech. To achieve human-like dialogue, we propose a novel future prediction architecture that allows an SDS to anticipate future affective reactions based on its current behaviors before the user speaks. In this work, we investigate two scenarios: speech and laughter. In speech, we propo… ▽ More Current Spoken Dialogue Systems (SDSs) often serve as passive listeners that respond only after receiving user speech. To achieve human-like dialogue, we propose a novel future prediction architecture that allows an SDS to anticipate future affective reactions based on its current behaviors before the user speaks. In this work, we investigate two scenarios: speech and laughter. In speech, we propose to predict the user's future emotion based on its temporal relationship with the system's current emotion and its causal relationship with the system's current Dialogue Act (DA). In laughter, we propose to predict the occurrence and type of the user's laughter using the system's laughter behaviors in the current turn. Preliminary analysis of human-robot dialogue demonstrated synchronicity in the emotions and laughter displayed by the human and robot, as well as DA-emotion causality in their dialogue. This verifies that our architecture can contribute to the development of an anticipatory SDS. △ Less

Submitted 17 December, 2024; v1 submitted 28 February, 2023; originally announced March 2023.

Comments: Accepted to CHI2023 Late-Breaking Work

Channel Sparsity Variation and Model-Based Analysis on 6, 26, and 132 GHz Measurements

Authors: Ximan Liu, Jianhua Zhang, Pan Tang, Lei Tian, Harsh Tataria, Shu Sun, Mansoor Shafi

Abstract: In this paper, the level of sparsity is examined at 6, 26, and 132 GHz carrier frequencies by conducting channel measurements in an indoor office environment. By using the Gini index (value between 0 and 1) as a metric for characterizing sparsity, we show that increasing carrier frequency leads to increased levels of sparsity. The measured channel impulse responses are used to derive a Third-Gener… ▽ More In this paper, the level of sparsity is examined at 6, 26, and 132 GHz carrier frequencies by conducting channel measurements in an indoor office environment. By using the Gini index (value between 0 and 1) as a metric for characterizing sparsity, we show that increasing carrier frequency leads to increased levels of sparsity. The measured channel impulse responses are used to derive a Third-Generation Partnership Project (3GPP)-style propagation model, used to calculate the Gini index for the comparison of the channel sparsity between the measurement and simulation based on the 3GPP model. Our results show that the mean value of the Gini index in measurement is over twice the value in simulation, implying that the 3GPP channel model does not capture the effects of sparsity in the delay domain as frequency increases. In addition, a new intra-cluster power allocation model based on measurements is proposed to characterize the effects of sparsity in the delay domain of the 3GPP channel model. The accuracy of the proposed model is analyzed using theoretical derivations and simulations. Using the derived intra-cluster power allocation model, the mean value of the Gini index is 0.97, while the spread of variability is restricted to 0.01, demonstrating that the proposed model is suitable for 3GPP-type channels. To our best knowledge, this paper is the first to perform measurements and analysis at three different frequencies for the evaluation of channel sparsity in the same environment. △ Less

Submitted 17 February, 2023; originally announced February 2023.

Multimodal Brain Disease Classification with Functional Interaction Learning from Single fMRI Volume

Authors: Wei Dai, Ziyao Zhang, Lixia Tian, Shengyuan Yu, Shuhui Wang, Zhao Dong, Hairong Zheng

Abstract: In neuroimaging analysis, fMRI can well assess the function changes for brain diseases with no obvious structural lesions. To date, most deep-learning-based fMRI studies have employed functional connectivity (FC) as the basic feature for disease classification. However, FC is calculated on time series of predefined regions of interest and neglects detailed information contained in each voxel. Anot… ▽ More In neuroimaging analysis, fMRI can well assess the function changes for brain diseases with no obvious structural lesions. To date, most deep-learning-based fMRI studies have employed functional connectivity (FC) as the basic feature for disease classification. However, FC is calculated on time series of predefined regions of interest and neglects detailed information contained in each voxel. Another drawback of using FC is the limited sample size for the training of deep models. The low representation ability of FC leads to poor performance in clinical practice, especially when dealing with multimodal medical data involving multiple types of visual signals and textual records for brain diseases. To overcome this bottleneck problem in the fMRI feature modality, we propose BrainFormer, an end-to-end functional interaction learning method for brain disease classification with single fMRI volume. Unlike traditional deep learning methods that construct convolution and transformers on FC, BrainFormer learns the functional interaction from fMRI signals, by modeling the local cues within each voxel with 3D convolutions and capturing the global correlations among distant regions with specially designed global attention mechanisms from shallow layers to deep layers. Meanwhile, BrainFormer can deal with multimodal medical data including fMRI volume, structural MRI, FC features and phenotypic data to achieve more comprehensive brain disease diagnosis. We evaluate BrainFormer on five independent multi-site datasets on autism, Alzheimer's disease, depression, attention deficit hyperactivity disorder and headache disorders. The results demonstrate its effectiveness and generalizability for multiple brain diseases diagnosis with multimodal features. BrainFormer may promote precision of neuroimaging-based diagnosis in clinical practice and motivate future studies on fMRI analysis. △ Less

Submitted 1 March, 2023; v1 submitted 5 August, 2022; originally announced August 2022.

High-fidelity intensity diffraction tomography with a non-paraxial multiple-scattering model

Authors: Jiabei Zhu, Hao Wang, Lei Tian

Abstract: We propose a novel intensity diffraction tomography (IDT) reconstruction algorithm based on the split-step non-paraxial (SSNP) model for recovering the 3D refractive index (RI) distribution of multiple-scattering biological samples. High-quality IDT reconstruction requires high-angle illumination to encode both low- and high- spatial frequency information of the 3D biological sample. We show that… ▽ More We propose a novel intensity diffraction tomography (IDT) reconstruction algorithm based on the split-step non-paraxial (SSNP) model for recovering the 3D refractive index (RI) distribution of multiple-scattering biological samples. High-quality IDT reconstruction requires high-angle illumination to encode both low- and high- spatial frequency information of the 3D biological sample. We show that our SSNP model can more accurately compute multiple scattering from high-angle illumination compared to paraxial approximation-based multiple-scattering models. We apply this SSNP model to both sequential and multiplexed IDT techniques. We develop a unified reconstruction algorithm for both IDT modalities that is highly computationally efficient and is implemented by a modular automatic differentiation framework. We demonstrate the capability of our reconstruction algorithm on both weakly scattering buccal epithelial cells and strongly scattering live $\textit{C. elegans}$ worms and live $\textit{C. elegans}$ embryos. △ Less

Submitted 9 August, 2022; v1 submitted 13 July, 2022; originally announced July 2022.

Journal ref: Opt. Express 30, 32808-32821 (2022)

Frequency-Angle Two-Dimensional Reflection Coefficient Modeling Based on Terahertz Channel Measurement

Authors: Zhaowei Chang, Jianhua Zhang, Pan Tang, Lei Tian, Li Yu, Guangyi Liu, Liang Xia

Abstract: Terahertz (THz) channel propagation characteristics are vital for the design, evaluation, and optimization for THz communication systems. Moreover, reflection plays a significant role in channel propagation. In this letter, the reflection coefficient of the THz channel is researched based on extensive measurement campaigns. Firstly, we set up the THz channel sounder from 220 to 320 GHz with the in… ▽ More Terahertz (THz) channel propagation characteristics are vital for the design, evaluation, and optimization for THz communication systems. Moreover, reflection plays a significant role in channel propagation. In this letter, the reflection coefficient of the THz channel is researched based on extensive measurement campaigns. Firstly, we set up the THz channel sounder from 220 to 320 GHz with the incident angle ranging from 10° to 80°. Based on the measured propagation loss, the reflection coefficients of five building materials, i.e., glass, tile, aluminium alloy, board, and plasterboard, are calculated separately for frequencies and incident angles. It is found that the lack of THz relative parameters leads to the Fresnel model of non-metallic materials can not fit the measured data well. Thus, we propose a frequency-angle two-dimensional reflection coefficient model by modifying the Fresnel model with the Lorenz and Drude model. The proposed model characterizes the frequency and incident angle for reflection coefficients and shows low root-mean-square error with the measured data. Generally, these results are useful for modeling THz channels. △ Less

Submitted 10 July, 2022; originally announced July 2022.

Real-time Dual-channel 2 * 2 MIMO Fiber-THz-Fiber Seamless Integration System at 385 GHz and 435 GHz

Authors: Jiao Zhang, Min Zhu, Bingchang Hua, Mingzheng Lei, Yuancheng Cai, Liang Tian, Yucong Zou, Like Ma, Yongming Huang, Jianjun Yu, Xiaohu You

Abstract: We demonstrate the first practical real-time dual-channel fiber-THz-fiber 2 * 2 MIMO seamless integration system with a record net data rate of 2 * 103.125 Gb/s at 385 GHz and 435 GHz over two spans of 20 km SSMF and 3 m wireless link. We demonstrate the first practical real-time dual-channel fiber-THz-fiber 2 * 2 MIMO seamless integration system with a record net data rate of 2 * 103.125 Gb/s at 385 GHz and 435 GHz over two spans of 20 km SSMF and 3 m wireless link. △ Less

Submitted 24 June, 2022; originally announced June 2022.

Comments: This paper has been accepted by ECOC 2022

$\texttt{GradICON}$: Approximate Diffeomorphisms via Gradient Inverse Consistency

Authors: Lin Tian, Hastings Greer, François-Xavier Vialard, Roland Kwitt, Raúl San José Estépar, Richard Jarrett Rushmore, Nikolaos Makris, Sylvain Bouix, Marc Niethammer

Abstract: We present an approach to learning regular spatial transformations between image pairs in the context of medical image registration. Contrary to optimization-based registration techniques and many modern learning-based methods, we do not directly penalize transformation irregularities but instead promote transformation regularity via an inverse consistency penalty. We use a neural network to predi… ▽ More We present an approach to learning regular spatial transformations between image pairs in the context of medical image registration. Contrary to optimization-based registration techniques and many modern learning-based methods, we do not directly penalize transformation irregularities but instead promote transformation regularity via an inverse consistency penalty. We use a neural network to predict a map between a source and a target image as well as the map when swapping the source and target images. Different from existing approaches, we compose these two resulting maps and regularize deviations of the $\bf{Jacobian}$ of this composition from the identity matrix. This regularizer -- $\texttt{GradICON}$ -- results in much better convergence when training registration models compared to promoting inverse consistency of the composition of maps directly while retaining the desirable implicit regularization effects of the latter. We achieve state-of-the-art registration performance on a variety of real-world medical image datasets using a single set of hyperparameters and a single non-dataset-specific training protocol. △ Less

Submitted 9 October, 2023; v1 submitted 13 June, 2022; originally announced June 2022.

Comments: 29 pages, 16 figures, CVPR 2023

NTIRE 2022 Challenge on Efficient Super-Resolution: Methods and Results

Authors: Yawei Li, Kai Zhang, Radu Timofte, Luc Van Gool, Fangyuan Kong, Mingxi Li, Songwei Liu, Zongcai Du, Ding Liu, Chenhui Zhou, Jingyi Chen, Qingrui Han, Zheyuan Li, Yingqi Liu, Xiangyu Chen, Haoming Cai, Yu Qiao, Chao Dong, Long Sun, Jinshan Pan, Yi Zhu, Zhikai Zong, Xiaoxiao Liu, Zheng Hui, Tao Yang , et al. (86 additional authors not shown)

Abstract: This paper reviews the NTIRE 2022 challenge on efficient single image super-resolution with focus on the proposed solutions and results. The task of the challenge was to super-resolve an input image with a magnification factor of $\times$4 based on pairs of low and corresponding high resolution images. The aim was to design a network for single image super-resolution that achieved improvement of e… ▽ More This paper reviews the NTIRE 2022 challenge on efficient single image super-resolution with focus on the proposed solutions and results. The task of the challenge was to super-resolve an input image with a magnification factor of $\times$4 based on pairs of low and corresponding high resolution images. The aim was to design a network for single image super-resolution that achieved improvement of efficiency measured according to several metrics including runtime, parameters, FLOPs, activations, and memory consumption while at least maintaining the PSNR of 29.00dB on DIV2K validation set. IMDN is set as the baseline for efficiency measurement. The challenge had 3 tracks including the main track (runtime), sub-track one (model complexity), and sub-track two (overall performance). In the main track, the practical runtime performance of the submissions was evaluated. The rank of the teams were determined directly by the absolute value of the average runtime on the validation set and test set. In sub-track one, the number of parameters and FLOPs were considered. And the individual rankings of the two metrics were summed up to determine a final ranking in this track. In sub-track two, all of the five metrics mentioned in the description of the challenge including runtime, parameter count, FLOPs, activations, and memory consumption were considered. Similar to sub-track one, the rankings of five metrics were summed up to determine a final ranking. The challenge had 303 registered participants, and 43 teams made valid submissions. They gauge the state-of-the-art in efficient single image super-resolution. △ Less

Submitted 11 May, 2022; originally announced May 2022.

Comments: Validation code of the baseline model is available at https://github.com/ofsoundof/IMDN. Validation of all submitted models is available at https://github.com/ofsoundof/NTIRE2022_ESR

Deep-learning-augmented Computational Miniature Mesoscope

Authors: Yujia Xue, Qianwan Yang, Guorong Hu, Kehan Guo, Lei Tian

Abstract: Fluorescence microscopy is essential to study biological structures and dynamics. However, existing systems suffer from a tradeoff between field-of-view (FOV), resolution, and complexity, and thus cannot fulfill the emerging need of miniaturized platforms providing micron-scale resolution across centimeter-scale FOVs. To overcome this challenge, we developed Computational Miniature Mesoscope (CM… ▽ More Fluorescence microscopy is essential to study biological structures and dynamics. However, existing systems suffer from a tradeoff between field-of-view (FOV), resolution, and complexity, and thus cannot fulfill the emerging need of miniaturized platforms providing micron-scale resolution across centimeter-scale FOVs. To overcome this challenge, we developed Computational Miniature Mesoscope (CM$^2$) that exploits a computational imaging strategy to enable single-shot 3D high-resolution imaging across a wide FOV in a miniaturized platform. Here, we present CM$^2$ V2 that significantly advances both the hardware and computation. We complement the 3$\times$3 microlens array with a new hybrid emission filter that improves the imaging contrast by 5$\times$, and design a 3D-printed freeform collimator for the LED illuminator that improves the excitation efficiency by 3$\times$. To enable high-resolution reconstruction across the large imaging volume, we develop an accurate and efficient 3D linear shift-variant (LSV) model that characterizes the spatially varying aberrations. We then train a multi-module deep learning model, CM$^2$Net, using only the 3D-LSV simulator. We show that CM$^2$Net generalizes well to experiments and achieves accurate 3D reconstruction across a $\sim$7-mm FOV and 800-$μ$m depth, and provides $\sim$6-$μ$m lateral and $\sim$25-$μ$m axial resolution. This provides $\sim$8$\times$ better axial localization and $\sim$1400$\times$ faster speed as compared to the previous model-based algorithm. We anticipate this simple and low-cost computational miniature imaging system will be impactful to many large-scale 3D fluorescence imaging applications. △ Less

Submitted 7 September, 2022; v1 submitted 29 April, 2022; originally announced May 2022.

Journal ref: Optica 9, 1009-1021 (2022)

LiftReg: Limited Angle 2D/3D Deformable Registration

Authors: Lin Tian, Yueh Z. Lee, Raúl San José Estépar, Marc Niethammer

Abstract: We propose LiftReg, a 2D/3D deformable registration approach. LiftReg is a deep registration framework which is trained using sets of digitally reconstructed radiographs (DRR) and computed tomography (CT) image pairs. By using simulated training data, LiftReg can use a high-quality CT-CT image similarity measure, which helps the network to learn a high-quality deformation space. To further improve… ▽ More We propose LiftReg, a 2D/3D deformable registration approach. LiftReg is a deep registration framework which is trained using sets of digitally reconstructed radiographs (DRR) and computed tomography (CT) image pairs. By using simulated training data, LiftReg can use a high-quality CT-CT image similarity measure, which helps the network to learn a high-quality deformation space. To further improve registration quality and to address the inherent depth ambiguities of very limited angle acquisitions, we propose to use features extracted from the backprojected 2D images and a statistical deformation model. We test our approach on the DirLab lung registration dataset and show that it outperforms an existing learning-based pairwise registration approach. △ Less

Submitted 4 April, 2023; v1 submitted 9 March, 2022; originally announced March 2022.

Comments: MICCAI 2022

Fluid registration between lung CT and stationary chest tomosynthesis images

Authors: Lin Tian, Connor Puett, Peirong Liu, Zhengyang Shen, Stephen R. Aylward, Yueh Z. Lee, Marc Niethammer

Abstract: Registration is widely used in image-guided therapy and image-guided surgery to estimate spatial correspondences between organs of interest between planning and treatment images. However, while high-quality computed tomography (CT) images are often available at planning time, limited angle acquisitions are frequently used during treatment because of radiation concerns or imaging time constraints.… ▽ More Registration is widely used in image-guided therapy and image-guided surgery to estimate spatial correspondences between organs of interest between planning and treatment images. However, while high-quality computed tomography (CT) images are often available at planning time, limited angle acquisitions are frequently used during treatment because of radiation concerns or imaging time constraints. This requires algorithms to register CT images based on limited angle acquisitions. We, therefore, formulate a 3D/2D registration approach which infers a 3D deformation based on measured projections and digitally reconstructed radiographs of the CT. Most 3D/2D registration approaches use simple transformation models or require complex mathematical derivations to formulate the underlying optimization problem. Instead, our approach entirely relies on differentiable operations which can be combined with modern computational toolboxes supporting automatic differentiation. This then allows for rapid prototyping, integration with deep neural networks, and to support a variety of transformation models including fluid flow models. We demonstrate our approach for the registration between CT and stationary chest tomosynthesis (sDCT) images and show how it naturally leads to an iterative image reconstruction approach. △ Less

Submitted 6 March, 2022; originally announced March 2022.

The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients

Authors: Bhakti Baheti, Satrajit Chakrabarty, Hamed Akbari, Michel Bilello, Benedikt Wiestler, Julian Schwarting, Evan Calabrese, Jeffrey Rudie, Syed Abidi, Mina Mousa, Javier Villanueva-Meyer, Brandon K. K. Fields, Florian Kofler, Russell Takeshi Shinohara, Juan Eugenio Iglesias, Tony C. W. Mok, Albert C. S. Chung, Marek Wodzinski, Artur Jurgas, Niccolo Marini, Manfredo Atzori, Henning Muller, Christoph Grobroehmer, Hanna Siebert, Lasse Hansen , et al. (48 additional authors not shown)

Abstract: Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registr… ▽ More Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registration (BraTS-Reg) challenge, as the first public benchmark environment for deformable registration algorithms focusing on estimating correspondences between pre-operative and follow-up scans of the same patient diagnosed with a diffuse brain glioma. The BraTS-Reg data comprise de-identified multi-institutional multi-parametric MRI (mpMRI) scans, curated for size and resolution according to a canonical anatomical template, and divided into training, validation, and testing sets. Clinical experts annotated ground truth (GT) landmark points of anatomical locations distinct across the temporal domain. Quantitative evaluation and ranking were based on the Median Euclidean Error (MEE), Robustness, and the determinant of the Jacobian of the displacement field. The top-ranked methodologies yielded similar performance across all evaluation metrics and shared several methodological commonalities, including pre-alignment, deep neural networks, inverse consistency analysis, and test-time instance optimization per-case basis as a post-processing step. The top-ranked method attained the MEE at or below that of the inter-rater variability for approximately 60% of the evaluated landmarks, underscoring the scope for further accuracy and robustness improvements, especially relative to human experts. The aim of BraTS-Reg is to continue to serve as an active resource for research, with the data and online evaluation tools accessible at https://bratsreg.github.io/. △ Less

Submitted 17 April, 2024; v1 submitted 13 December, 2021; originally announced December 2021.

Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fields

Authors: Renhao Liu, Yu Sun, Jiabei Zhu, Lei Tian, Ulugbek Kamilov

Abstract: Intensity diffraction tomography (IDT) refers to a class of optical microscopy techniques for imaging the 3D refractive index (RI) distribution of a sample from a set of 2D intensity-only measurements. The reconstruction of artifact-free RI maps is a fundamental challenge in IDT due to the loss of phase information and the missing cone problem. Neural fields (NF) has recently emerged as a new deep… ▽ More Intensity diffraction tomography (IDT) refers to a class of optical microscopy techniques for imaging the 3D refractive index (RI) distribution of a sample from a set of 2D intensity-only measurements. The reconstruction of artifact-free RI maps is a fundamental challenge in IDT due to the loss of phase information and the missing cone problem. Neural fields (NF) has recently emerged as a new deep learning (DL) approach for learning continuous representations of physical fields. NF uses a coordinate-based neural network to represent the field by mapping the spatial coordinates to the corresponding physical quantities, in our case the complex-valued refractive index values. We present DeCAF as the first NF-based IDT method that can learn a high-quality continuous representation of a RI volume from its intensity-only and limited-angle measurements. The representation in DeCAF is learned directly from the measurements of the test sample by using the IDT forward model, without any ground-truth RI maps. We qualitatively and quantitatively evaluate DeCAF on the simulated and experimental biological samples. Our results show that DeCAF can generate high-contrast and artifact-free RI maps and lead to up to 2.1 times reduction in MSE over existing methods. △ Less

Submitted 14 August, 2022; v1 submitted 27 November, 2021; originally announced December 2021.

A New Journey from SDRTV to HDRTV

Authors: Xiangyu Chen, Zhengwen Zhang, Jimmy S. Ren, Lynhoo Tian, Yu Qiao, Chao Dong

Abstract: Nowadays modern displays are capable to render video content with high dynamic range (HDR) and wide color gamut (WCG). However, most available resources are still in standard dynamic range (SDR). Therefore, there is an urgent demand to transform existing SDR-TV contents into their HDR-TV versions. In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV… ▽ More Nowadays modern displays are capable to render video content with high dynamic range (HDR) and wide color gamut (WCG). However, most available resources are still in standard dynamic range (SDR). Therefore, there is an urgent demand to transform existing SDR-TV contents into their HDR-TV versions. In this paper, we conduct an analysis of SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Base on the analysis, we propose a three-step solution pipeline including adaptive global color mapping, local enhancement and highlight generation. Moreover, the above analysis inspires us to present a lightweight network that utilizes global statistics as guidance to conduct image-adaptive color mapping. In addition, we construct a dataset using HDR videos in HDR10 standard, named HDRTV1K, and select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Furthermore, our final results achieve state-of-the-art performance in quantitative comparisons and visual quality. The code and dataset are available at https://github.com/chxy95/HDRTVNet. △ Less

Submitted 25 September, 2021; v1 submitted 18 August, 2021; originally announced August 2021.

Comments: Accepted to ICCV

Adaptive 3D descattering with a dynamic synthesis network

Authors: Waleed Tahir, Hao Wang, Lei Tian

Abstract: Deep learning has been broadly applied to imaging in scattering applications. A common framework is to train a descattering network for image recovery by removing scattering artifacts. To achieve the best results on a broad spectrum of scattering conditions, individual "expert" networks need to be trained for each condition. However, the expert's performance sharply degrades when the testing condi… ▽ More Deep learning has been broadly applied to imaging in scattering applications. A common framework is to train a descattering network for image recovery by removing scattering artifacts. To achieve the best results on a broad spectrum of scattering conditions, individual "expert" networks need to be trained for each condition. However, the expert's performance sharply degrades when the testing condition differs from the training. An alternative brute-force approach is to train a "generalist" network using data from diverse scattering conditions. It generally requires a larger network to encapsulate the diversity in the data and a sufficiently large training set to avoid overfitting. Here, we propose an adaptive learning framework, termed dynamic synthesis network (DSN), which dynamically adjusts the model weights and adapts to different scattering conditions. The adaptability is achieved by a novel "mixture of experts" architecture that enables dynamically synthesizing a network by blending multiple experts using a gating network. We demonstrate the DSN in holographic 3D particle imaging for a variety of scattering conditions. We show in simulation that our DSN provides generalization across a continuum of scattering conditions. In addition, we show that by training the DSN entirely on simulated data, the network can generalize to experiments and achieve robust 3D descattering. We expect the same concept can find many other applications, such as denoising and imaging in scattering media. Broadly, our dynamic synthesis framework opens up a new paradigm for designing highly adaptive deep learning and computational imaging techniques. △ Less

Submitted 2 February, 2022; v1 submitted 1 July, 2021; originally announced July 2021.

Journal ref: Light Sci Appl 11, 42 (2022)

Physical model simulator-trained neural network for computational 3D phase imaging of multiple-scattering samples

Authors: Alex Matlock, Lei Tian

Abstract: Recovering 3D phase features of complex, multiple-scattering biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy and reconstruction quality. This trade-off hinders the rapid analysis of living, dynamic biological samples that are often of greatest interest to biological research. Here, we overcome this bottleneck by combining annular… ▽ More Recovering 3D phase features of complex, multiple-scattering biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy and reconstruction quality. This trade-off hinders the rapid analysis of living, dynamic biological samples that are often of greatest interest to biological research. Here, we overcome this bottleneck by combining annular intensity diffraction tomography (aIDT) with an approximant-guided deep learning framework. Using a novel physics model simulator-based learning strategy trained entirely on natural image datasets, we show our network can robustly reconstruct complex 3D biological samples of arbitrary size and structure. This approach highlights that large-scale multiple-scattering models can be leveraged in place of acquiring experimental datasets for achieving highly generalizable deep learning models. We devise a new model-based data normalization pre-processing procedure for homogenizing the sample contrast and achieving uniform prediction quality regardless of scattering strength. To achieve highly efficient training and prediction, we implement a lightweight 2D network structure that utilizes a multi-channel input for encoding the axial information. We demonstrate this framework's capabilities on experimental measurements of epithelial buccal cells and Caenorhabditis elegans worms. We highlight the robustness of this approach by evaluating dynamic samples on a living worm video, and we emphasize our approach's generalizability by recovering algae samples evaluated with different experimental setups. To assess the prediction quality, we develop a novel quantitative evaluation metric and show that our predictions are consistent with our experimental measurements and multiple-scattering physics. △ Less

Submitted 29 March, 2021; originally announced March 2021.

Comments: 16 Pages, 6 Figures

Discovering Hidden Physics Behind Transport Dynamics

Authors: Peirong Liu, Lin Tian, Yubo Zhang, Stephen R. Aylward, Yueh Z. Lee, Marc Niethammer

Abstract: Transport processes are ubiquitous. They are, for example, at the heart of optical flow approaches; or of perfusion imaging, where blood transport is assessed, most commonly by injecting a tracer. An advection-diffusion equation is widely used to describe these transport phenomena. Our goal is estimating the underlying physics of advection-diffusion equations, expressed as velocity and diffusion t… ▽ More Transport processes are ubiquitous. They are, for example, at the heart of optical flow approaches; or of perfusion imaging, where blood transport is assessed, most commonly by injecting a tracer. An advection-diffusion equation is widely used to describe these transport phenomena. Our goal is estimating the underlying physics of advection-diffusion equations, expressed as velocity and diffusion tensor fields. We propose a learning framework (YETI) building on an auto-encoder structure between 2D and 3D image time-series, which incorporates the advection-diffusion model. To help with identifiability, we develop an advection-diffusion simulator which allows pre-training of our model by supervised learning using the velocity and diffusion tensor fields. Instead of directly learning these velocity and diffusion tensor fields, we introduce representations that assure incompressible flow and symmetric positive semi-definite diffusion fields and demonstrate the additional benefits of these representations on improving estimation accuracy. We further use transfer learning to apply YETI on a public brain magnetic resonance (MR) perfusion dataset of stroke patients and show its ability to successfully distinguish stroke lesions from normal brain regions via the estimated velocity and diffusion tensor fields. △ Less

Submitted 29 March, 2021; v1 submitted 24 November, 2020; originally announced November 2020.

Comments: Accepted as ORAL at CVPR 2021 (20 pages, 13 figures)

Displacement-agnostic coherent imaging through scatter with an interpretable deep neural network

Authors: Yuzhe Li, Shiyi Cheng, Yujia Xue, Lei Tian

Abstract: Coherent imaging through scatter is a challenging task in computational imaging. Both model-based and data-driven approaches have been explored to solve the inverse scattering problem. In our previous work, we have shown that a deep learning approach can make high-quality and highly generalizable predictions through unseen diffusers. Here, we propose a new deep neural network (DNN) model that is a… ▽ More Coherent imaging through scatter is a challenging task in computational imaging. Both model-based and data-driven approaches have been explored to solve the inverse scattering problem. In our previous work, we have shown that a deep learning approach can make high-quality and highly generalizable predictions through unseen diffusers. Here, we propose a new deep neural network (DNN) model that is agnostic to a broader class of perturbations including scatterer change, displacements, and system defocus up to 10X depth of field. In addition, we develop a new analysis framework for interpreting the mechanism of our DNN model and visualizing its generalizability based on an unsupervised dimension reduction technique. We show that our DNN can unmix the scattering-specific information and extract the object-specific information so as to achieve generalization under different scattering conditions. Our work paves the way to a highly robust and interpretable deep learning approach to imaging through scattering media. △ Less

Submitted 1 September, 2020; v1 submitted 14 May, 2020; originally announced May 2020.

Single-Shot 3D Widefield Fluorescence Imaging with a Computational Miniature Mesoscope

Authors: Yujia Xue, Ian G. Davison, David A. Boas, Lei Tian

Abstract: Fluorescence imaging is indispensable to biology and neuroscience. The need for large-scale imaging in freely behaving animals has further driven the development in miniaturized microscopes (miniscopes). However, conventional microscopes / miniscopes are inherently constrained by their limited space-bandwidth-product, shallow depth-of-field, and the inability to resolve 3D distributed emitters. He… ▽ More Fluorescence imaging is indispensable to biology and neuroscience. The need for large-scale imaging in freely behaving animals has further driven the development in miniaturized microscopes (miniscopes). However, conventional microscopes / miniscopes are inherently constrained by their limited space-bandwidth-product, shallow depth-of-field, and the inability to resolve 3D distributed emitters. Here, we present a Computational Miniature Mesoscope (CM$^2$) that overcomes these bottlenecks and enables single-shot 3D imaging across an 8 $\times$ 7-mm$^2$ field-of-view and 2.5-mm depth-of-field, achieving 7-$μ$m lateral resolution and better than 200-$μ$m axial resolution. Notably, the CM$^2$ has a compact lightweight design that integrates a microlens array for imaging and an LED array for excitation in a single platform. Its expanded imaging capability is enabled by computational imaging that augments the optics by algorithms. We experimentally validate the mesoscopic 3D imaging capability on volumetrically distributed fluorescent beads and fibers. We further quantify the effects of bulk scattering and background fluorescence on phantom experiments. △ Less

Submitted 31 August, 2020; v1 submitted 26 March, 2020; originally announced March 2020.

Journal ref: Science Advances 21 Oct 2020: Vol. 6, no. 43, eabb7508

Inverse scattering for reflection intensity phase microscopy

Authors: Alex Matlock, Anne Sentenac, Patrick C. Chaumet, Ji Yi, Lei Tian

Abstract: Reflection phase imaging provides label-free, high-resolution characterization of biological samples, typically using interferometric-based techniques. Here, we investigate reflection phase microscopy from intensity-only measurements under diverse illumination. We evaluate the forward and inverse scattering model based on the first Born approximation for imaging scattering objects above a glass sl… ▽ More Reflection phase imaging provides label-free, high-resolution characterization of biological samples, typically using interferometric-based techniques. Here, we investigate reflection phase microscopy from intensity-only measurements under diverse illumination. We evaluate the forward and inverse scattering model based on the first Born approximation for imaging scattering objects above a glass slide. Under this design, the measured field combines linear forward-scattering and height-dependent nonlinear back-scattering from the object that complicates object phase recovery. Using only the forward-scattering, we derive a linear inverse scattering model and evaluate this model's validity range in simulation and experiment using a standard reflection microscope modified with a programmable light source. Our method provides enhanced contrast of thin, weakly scattering samples that complement transmission techniques. This model provides a promising development for creating simplified intensity-based reflection quantitative phase imaging systems easily adoptable for biological research. △ Less

Submitted 16 December, 2019; originally announced December 2019.

Comments: 25 pages, 5 figures

On a universal solution to the transport-of-intensity equation

Authors: Jialin Zhang, Qian Chen, Jiasong Sun, Long Tian, Chao Zuo

Abstract: Transport-of-intensity equation (TIE) is one of the most well-known approaches for phase retrieval and quantitative phase imaging. It directly recovers the quantitative phase distribution of an optical field by through-focus intensity measurements in a noninterferometic, deterministic manner. Nevertheless, the accuracy and validity of state-of-the-art TIE solvers depend on restrictive preknowledge… ▽ More Transport-of-intensity equation (TIE) is one of the most well-known approaches for phase retrieval and quantitative phase imaging. It directly recovers the quantitative phase distribution of an optical field by through-focus intensity measurements in a noninterferometic, deterministic manner. Nevertheless, the accuracy and validity of state-of-the-art TIE solvers depend on restrictive preknowledge or assumptions, including appropriate boundary conditions, a well-defined closed region, and quasi-uniform in-focus intensity distribution, which, however, cannot be strictly satisfied simultaneously under practical experimental conditions. In this Letter, we propose a universal solution to TIE with the advantages of high accuracy, convergence guarantee, applicability to arbitrarily-shaped regions, and simplified implementation and computation. With the "maximum intensity assumption", we firstly simplified TIE as a standard Possion equation to get an initial guess of the solution. Then the initial solution is further refined iteratively by solving the same Possion equation, and thus, the instability associated with the division by zero/small intensity values and large intensity variations can be effectively bypassed. Simulations and experiments with arbitrary phase, arbitrary aperture shapes, and nonuniform intensity distributions verify the effectiveness and universality of the proposed method. △ Less

Submitted 2 March, 2020; v1 submitted 12 December, 2019; originally announced December 2019.

SIMBA: Scalable Inversion in Optical Tomography using Deep Denoising Priors

Authors: Zihui Wu, Yu Sun, Alex Matlock, Jiaming Liu, Lei Tian, Ulugbek S. Kamilov

Abstract: Two features desired in a three-dimensional (3D) optical tomographic image reconstruction algorithm are the ability to reduce imaging artifacts and to do fast processing of large data volumes. Traditional iterative inversion algorithms are impractical in this context due to their heavy computational and memory requirements. We propose and experimentally validate a novel scalable iterative mini-bat… ▽ More Two features desired in a three-dimensional (3D) optical tomographic image reconstruction algorithm are the ability to reduce imaging artifacts and to do fast processing of large data volumes. Traditional iterative inversion algorithms are impractical in this context due to their heavy computational and memory requirements. We propose and experimentally validate a novel scalable iterative mini-batch algorithm (SIMBA) for fast and high-quality optical tomographic imaging. SIMBA enables high-quality imaging by combining two complementary information sources: the physics of the imaging system characterized by its forward model and the imaging prior characterized by a denoising deep neural net. SIMBA easily scales to very large 3D tomographic datasets by processing only a small subset of measurements at each iteration. We establish the theoretical fixed-point convergence of SIMBA under nonexpansive denoisers for convex data-fidelity terms. We validate SIMBA on both simulated and experimentally collected intensity diffraction tomography (IDT) datasets. Our results show that SIMBA can significantly reduce the computational burden of 3D image formation without sacrificing the imaging quality. △ Less

Submitted 11 June, 2020; v1 submitted 29 November, 2019; originally announced November 2019.

Comments: In press for IEEE Journal Selected Topics in Signal Processing

Reliable deep-learning-based phase imaging with uncertainty quantification

Authors: Yujia Xue, Shiyi Cheng, Yunzhe Li, Lei Tian

Abstract: Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedical imaging. However, one outstanding challenge is the lack of reliability assessment in the DL predictions, whose errors are commonly revealed only in hindsight. Here, we propose a new Bayesian convolutional neural network (BNN)-based framework that overcomes this issue by quantifying the uncertainty o… ▽ More Emerging deep-learning (DL)-based techniques have significant potential to revolutionize biomedical imaging. However, one outstanding challenge is the lack of reliability assessment in the DL predictions, whose errors are commonly revealed only in hindsight. Here, we propose a new Bayesian convolutional neural network (BNN)-based framework that overcomes this issue by quantifying the uncertainty of DL predictions. Foremost, we show that BNN-predicted uncertainty maps provide surrogate estimates of the true error from the network model and measurement itself. The uncertainty maps characterize imperfections often unknown in real-world applications, such as noise, model error, incomplete training data, and out-of-distribution testing data. Quantifying this uncertainty provides a per-pixel estimate of the confidence level of the DL prediction as well as the quality of the model and dataset. We demonstrate this framework in the application of large space-bandwidth product phase imaging using a physics-guided coded illumination scheme. From only five multiplexed illumination measurements, our BNN predicts gigapixel phase images in both static and dynamic biological samples with quantitative credibility assessment. Furthermore, we show that low-certainty regions can identify spatially and temporally rare biological phenomena. We believe our uncertainty learning framework is widely applicable to many DL-based biomedical imaging techniques for assessing the reliability of DL predictions. △ Less

Submitted 4 May, 2019; v1 submitted 7 January, 2019; originally announced January 2019.

Journal ref: Optica 6, 618-629 (2019)

Energy Efficiency Optimization of Generalized Spatial Modulation with Sub-Connected Hybrid Precoding

Authors: Kai Chen, Jing Yang, Xiaohu Ge, Yonghui Li, Lin Tian, Jinglin Shi

Abstract: Energy efficiency (EE) optimization of millimeter wave (mm-Wave) massive multiple-input multiple-output (MIMO) systems is emerging as an important challenge for the fifth generation (5G) mobile communication systems. However, the power of radio frequency (RF) chains increases sharply due to the high carrier frequency in mm-Wave massive MIMO systems. To overcome this issue, a new energy efficiency… ▽ More Energy efficiency (EE) optimization of millimeter wave (mm-Wave) massive multiple-input multiple-output (MIMO) systems is emerging as an important challenge for the fifth generation (5G) mobile communication systems. However, the power of radio frequency (RF) chains increases sharply due to the high carrier frequency in mm-Wave massive MIMO systems. To overcome this issue, a new energy efficiency optimization solution is proposed based on the structure of the generalized spatial modulation (GSM) and sub-connected hybrid precoding (HP). Moreover, the computation power of mm-Wave massive MIMO systems is considered for optimizing the EE. Simulation results indicate that the EE of the GSM-HP scheme outperforms the full digital precoding (FDP) scheme in the mm-Wave massive MIMO scene, and 88\% computation power can be saved by the proposed GSM-HP scheme. △ Less

Submitted 1 January, 2019; originally announced January 2019.

Holographic particle localization under multiple scattering

Authors: Waleed Tahir, Ulugbek S. Kamilov, Lei Tian

Abstract: We introduce a novel framework that incorporates multiple scattering for large-scale 3D particle-localization using single-shot in-line holography. Traditional holographic techniques rely on single-scattering models which become inaccurate under high particle-density. We demonstrate that by exploiting multiple-scattering, localization is significantly improved. Both forward and back-scattering are… ▽ More We introduce a novel framework that incorporates multiple scattering for large-scale 3D particle-localization using single-shot in-line holography. Traditional holographic techniques rely on single-scattering models which become inaccurate under high particle-density. We demonstrate that by exploiting multiple-scattering, localization is significantly improved. Both forward and back-scattering are computed by our method under a tractable recursive framework, in which each recursion estimates the next higher-order field within the volume. The inverse scattering is presented as a nonlinear optimization that promotes sparsity, and can be implemented efficiently. We experimentally reconstruct 100 million object voxels from a single 1-megapixel hologram. Our work promises utilization of multiple scattering for versatile large-scale applications. △ Less

Submitted 1 June, 2019; v1 submitted 31 July, 2018; originally announced July 2018.

Journal ref: Advanced Photonics, 1(3), 036003 (2019)

Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media

Authors: Yunzhe Li, Yujia Xue, Lei Tian

Abstract: Imaging through scattering is an important, yet challenging problem. Tremendous progress has been made by exploiting the deterministic input-output "transmission matrix" for a fixed medium. However, this "one-to-one" mapping is highly susceptible to speckle decorrelations - small perturbations to the scattering medium lead to model errors and severe degradation of the imaging performance. Our goal… ▽ More Imaging through scattering is an important, yet challenging problem. Tremendous progress has been made by exploiting the deterministic input-output "transmission matrix" for a fixed medium. However, this "one-to-one" mapping is highly susceptible to speckle decorrelations - small perturbations to the scattering medium lead to model errors and severe degradation of the imaging performance. Our goal here is to develop a new framework that is highly scalable to both medium perturbations and measurement requirement. To do so, we propose a statistical "one-to-all" deep learning technique that encapsulates a wide range of statistical variations for the model to be resilient to speckle decorrelations. Specifically, we develop a convolutional neural network (CNN) that is able to learn the statistical information contained in the speckle intensity patterns captured on a set of diffusers having the same macroscopic parameter. We then show for the first time, to the best of our knowledge, that the trained CNN is able to generalize and make high-quality object predictions through an entirely different set of diffusers of the same class. Our work paves the way to a highly scalable deep learning approach for imaging through scattering media. △ Less

Submitted 26 September, 2018; v1 submitted 11 June, 2018; originally announced June 2018.

Journal ref: Yunzhe Li, Yujia Xue, and Lei Tian, "Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media," Optica 5, 1181-1190 (2018)

High-throughput intensity diffraction tomography with a computational microscope

Authors: Ruilong Ling, Waleed Tahir, Hsing-Ying Lin, Hakho Lee, Lei Tian

Abstract: We demonstrate a motion-free intensity diffraction tomography technique that enables direct inversion of 3D phase and absorption from intensity-only measurements for weakly scattering samples. We derive a novel linear forward model, featuring slice-wise phase and absorption transfer functions using angled illumination. This new framework facilitates flexible and efficient data acquisition, enablin… ▽ More We demonstrate a motion-free intensity diffraction tomography technique that enables direct inversion of 3D phase and absorption from intensity-only measurements for weakly scattering samples. We derive a novel linear forward model, featuring slice-wise phase and absorption transfer functions using angled illumination. This new framework facilitates flexible and efficient data acquisition, enabling arbitrary sampling of the illumination angles. The reconstruction algorithm performs 3D synthetic aperture using a robust, computation and memory efficient slice-wise deconvolution to achieve resolution up to the incoherent limit. We demonstrate our technique with thick biological samples having both sparse 3D structures and dense cell clusters. We further investigate the limitation of our technique when imaging strongly scattering samples. Imaging performance and the influence of multiple scattering is evaluated using a 3D sample consisting of stacked phase and absorption resolution targets. This computational microscopy system is directly built on a standard commercial microscope with a simple LED array source add-on, and promises broad applications by leveraging the ubiquitous microscopy platforms with minimal hardware modifications. △ Less

Submitted 8 April, 2018; v1 submitted 29 January, 2018; originally announced January 2018.