Showing 1–45 of 45 results for author: Shi, K

Negative Imaginary Neural ODEs: Learning to Control Mechanical Systems with Stability Guarantees

Authors: Kanghong Shi, Ruigang Wang, Ian R. Manchester

Abstract: We propose a neural control method to provide guaranteed stabilization for mechanical systems using a novel negative imaginary neural ordinary differential equation (NINODE) controller. Specifically, we employ neural networks with desired properties as state-space function matrices within a Hamiltonian framework to ensure the system possesses the NI property. This NINODE system can serve as a cont… ▽ More We propose a neural control method to provide guaranteed stabilization for mechanical systems using a novel negative imaginary neural ordinary differential equation (NINODE) controller. Specifically, we employ neural networks with desired properties as state-space function matrices within a Hamiltonian framework to ensure the system possesses the NI property. This NINODE system can serve as a controller that asymptotically stabilizes an NI plant under certain conditions. For mechanical plants with colocated force actuators and position sensors, we demonstrate that all the conditions required for stability can be translated into regularity constraints on the neural networks used in the controller. We illustrate the utility, effectiveness, and stability guarantees of the NINODE controller through an example involving a nonlinear mass-spring system. △ Less

Submitted 28 April, 2025; originally announced April 2025.

Uncertainty-guided Perturbation for Image Super-Resolution Diffusion Model

Authors: Leheng Zhang, Weiyi You, Kexuan Shi, Shuhang Gu

Abstract: Diffusion-based image super-resolution methods have demonstrated significant advantages over GAN-based approaches, particularly in terms of perceptual quality. Building upon a lengthy Markov chain, diffusion-based methods possess remarkable modeling capacity, enabling them to achieve outstanding performance in real-world scenarios. Unlike previous methods that focus on modifying the noise schedule… ▽ More Diffusion-based image super-resolution methods have demonstrated significant advantages over GAN-based approaches, particularly in terms of perceptual quality. Building upon a lengthy Markov chain, diffusion-based methods possess remarkable modeling capacity, enabling them to achieve outstanding performance in real-world scenarios. Unlike previous methods that focus on modifying the noise schedule or sampling process to enhance performance, our approach emphasizes the improved utilization of LR information. We find that different regions of the LR image can be viewed as corresponding to different timesteps in a diffusion process, where flat areas are closer to the target HR distribution but edge and texture regions are farther away. In these flat areas, applying a slight noise is more advantageous for the reconstruction. We associate this characteristic with uncertainty and propose to apply uncertainty estimate to guide region-specific noise level control, a technique we refer to as Uncertainty-guided Noise Weighting. Pixels with lower uncertainty (i.e., flat regions) receive reduced noise to preserve more LR information, therefore improving performance. Furthermore, we modify the network architecture of previous methods to develop our Uncertainty-guided Perturbation Super-Resolution (UPSR) model. Extensive experimental results demonstrate that, despite reduced model size and training overhead, the proposed UWSR method outperforms current state-of-the-art methods across various datasets, both quantitatively and qualitatively. △ Less

Submitted 24 March, 2025; originally announced March 2025.

Comments: Accepted to CVPR 2025

Fed-NDIF: A Noise-Embedded Federated Diffusion Model For Low-Count Whole-Body PET Denoising

Authors: Yinchi Zhou, Huidong Xie, Menghua Xia, Qiong Liu, Bo Zhou, Tianqi Chen, Jun Hou, Liang Guo, Xinyuan Zheng, Hanzhong Wang, Biao Li, Axel Rominger, Kuangyu Shi, Nicha C. Dvorneka, Chi Liu

Abstract: Low-count positron emission tomography (LCPET) imaging can reduce patients' exposure to radiation but often suffers from increased image noise and reduced lesion detectability, necessitating effective denoising techniques. Diffusion models have shown promise in LCPET denoising for recovering degraded image quality. However, training such models requires large and diverse datasets, which are challe… ▽ More Low-count positron emission tomography (LCPET) imaging can reduce patients' exposure to radiation but often suffers from increased image noise and reduced lesion detectability, necessitating effective denoising techniques. Diffusion models have shown promise in LCPET denoising for recovering degraded image quality. However, training such models requires large and diverse datasets, which are challenging to obtain in the medical domain. To address data scarcity and privacy concerns, we combine diffusion models with federated learning -- a decentralized training approach where models are trained individually at different sites, and their parameters are aggregated on a central server over multiple iterations. The variation in scanner types and image noise levels within and across institutions poses additional challenges for federated learning in LCPET denoising. In this study, we propose a novel noise-embedded federated learning diffusion model (Fed-NDIF) to address these challenges, leveraging a multicenter dataset and varying count levels. Our approach incorporates liver normalized standard deviation (NSTD) noise embedding into a 2.5D diffusion model and utilizes the Federated Averaging (FedAvg) algorithm to aggregate locally trained models into a global model, which is subsequently fine-tuned on local datasets to optimize performance and obtain personalized models. Extensive validation on datasets from the University of Bern, Ruijin Hospital in Shanghai, and Yale-New Haven Hospital demonstrates the superior performance of our method in enhancing image quality and improving lesion quantification. The Fed-NDIF model shows significant improvements in PSNR, SSIM, and NMSE of the entire 3D volume, as well as enhanced lesion detectability and quantification, compared to local diffusion models and federated UNet-based models. △ Less

Submitted 20 March, 2025; originally announced March 2025.

PET Image Denoising via Text-Guided Diffusion: Integrating Anatomical Priors through Text Prompts

Authors: Boxiao Yu, Savas Ozdemir, Jiong Wu, Yizhou Chen, Ruogu Fang, Kuangyu Shi, Kuang Gong

Abstract: Low-dose Positron Emission Tomography (PET) imaging presents a significant challenge due to increased noise and reduced image quality, which can compromise its diagnostic accuracy and clinical utility. Denoising diffusion probabilistic models (DDPMs) have demonstrated promising performance for PET image denoising. However, existing DDPM-based methods typically overlook valuable metadata such as pa… ▽ More Low-dose Positron Emission Tomography (PET) imaging presents a significant challenge due to increased noise and reduced image quality, which can compromise its diagnostic accuracy and clinical utility. Denoising diffusion probabilistic models (DDPMs) have demonstrated promising performance for PET image denoising. However, existing DDPM-based methods typically overlook valuable metadata such as patient demographics, anatomical information, and scanning parameters, which should further enhance the denoising performance if considered. Recent advances in vision-language models (VLMs), particularly the pre-trained Contrastive Language-Image Pre-training (CLIP) model, have highlighted the potential of incorporating text-based information into visual tasks to improve downstream performance. In this preliminary study, we proposed a novel text-guided DDPM for PET image denoising that integrated anatomical priors through text prompts. Anatomical text descriptions were encoded using a pre-trained CLIP text encoder to extract semantic guidance, which was then incorporated into the diffusion process via the cross-attention mechanism. Evaluations based on paired 1/20 low-dose and normal-dose 18F-FDG PET datasets demonstrated that the proposed method achieved better quantitative performance than conventional UNet and standard DDPM methods at both the whole-body and organ levels. These results underscored the potential of leveraging VLMs to integrate rich metadata into the diffusion framework to enhance the image quality of low-dose PET scans. △ Less

Submitted 28 February, 2025; originally announced February 2025.

Joint System Latency and Data Freshness Optimization for Cache-enabled Mobile Crowdsensing Networks

Authors: Kexin Shi, Yaru Fu, Yongna Guo, Fu Lee Wang, Yan Zhang

Abstract: Mobile crowdsensing (MCS) networks enable large-scale data collection by leveraging the ubiquity of mobile devices. However, frequent sensing and data transmission can lead to significant resource consumption. To mitigate this issue, edge caching has been proposed as a solution for storing recently collected data. Nonetheless, this approach may compromise data freshness. In this paper, we investig… ▽ More Mobile crowdsensing (MCS) networks enable large-scale data collection by leveraging the ubiquity of mobile devices. However, frequent sensing and data transmission can lead to significant resource consumption. To mitigate this issue, edge caching has been proposed as a solution for storing recently collected data. Nonetheless, this approach may compromise data freshness. In this paper, we investigate the trade-off between re-using cached task results and re-sensing tasks in cache-enabled MCS networks, aiming to minimize system latency while maintaining information freshness. To this end, we formulate a weighted delay and age of information (AoI) minimization problem, jointly optimizing sensing decisions, user selection, channel selection, task allocation, and caching strategies. The problem is a mixed-integer non-convex programming problem which is intractable. Therefore, we decompose the long-term problem into sequential one-shot sub-problems and design a framework that optimizes system latency, task sensing decision, and caching strategy subproblems. When one task is re-sensing, the one-shot problem simplifies to the system latency minimization problem, which can be solved optimally. The task sensing decision is then made by comparing the system latency and AoI. Additionally, a Bayesian update strategy is developed to manage the cached task results. Building upon this framework, we propose a lightweight and time-efficient algorithm that makes real-time decisions for the long-term optimization problem. Extensive simulation results validate the effectiveness of our approach. △ Less

Submitted 24 January, 2025; originally announced January 2025.

AADNet: Exploring EEG Spatiotemporal Information for Fast and Accurate Orientation and Timbre Detection of Auditory Attention Based on A Cue-Masked Paradigm

Authors: Keren Shi, Xu Liu, Xue Yuan, Haijie Shang, Ruiting Dai, Hanbin Wang, Yunfa Fu, Ning Jiang, Jiayuan He

Abstract: Auditory attention decoding from electroencephalogram (EEG) could infer to which source the user is attending in noisy environments. Decoding algorithms and experimental paradigm designs are crucial for the development of technology in practical applications. To simulate real-world scenarios, this study proposed a cue-masked auditory attention paradigm to avoid information leakage before the exper… ▽ More Auditory attention decoding from electroencephalogram (EEG) could infer to which source the user is attending in noisy environments. Decoding algorithms and experimental paradigm designs are crucial for the development of technology in practical applications. To simulate real-world scenarios, this study proposed a cue-masked auditory attention paradigm to avoid information leakage before the experiment. To obtain high decoding accuracy with low latency, an end-to-end deep learning model, AADNet, was proposed to exploit the spatiotemporal information from the short time window of EEG signals. The results showed that with a 0.5-second EEG window, AADNet achieved an average accuracy of 93.46% and 91.09% in decoding auditory orientation attention (OA) and timbre attention (TA), respectively. It significantly outperformed five previous methods and did not need the knowledge of the original audio source. This work demonstrated that it was possible to detect the orientation and timbre of auditory attention from EEG signals fast and accurately. The results are promising for the real-time multi-property auditory attention decoding, facilitating the application of the neuro-steered hearing aids and other assistive listening devices. △ Less

Submitted 7 January, 2025; originally announced January 2025.

When Mamba Meets xLSTM: An Efficient and Precise Method with the xLSTM-VMUNet Model for Skin lesion Segmentation

Authors: Zhuoyi Fang, Jiajia Liu, Kexuan Shi, Qiang Han

Abstract: Automatic melanoma segmentation is essential for early skin cancer detection, yet challenges arise from the heterogeneity of melanoma, as well as interfering factors like blurred boundaries, low contrast, and imaging artifacts. While numerous algorithms have been developed to address these issues, previous approaches have often overlooked the need to jointly capture spatial and sequential features… ▽ More Automatic melanoma segmentation is essential for early skin cancer detection, yet challenges arise from the heterogeneity of melanoma, as well as interfering factors like blurred boundaries, low contrast, and imaging artifacts. While numerous algorithms have been developed to address these issues, previous approaches have often overlooked the need to jointly capture spatial and sequential features within dermatological images. This limitation hampers segmentation accuracy, especially in cases with indistinct borders or structurally similar lesions. Additionally, previous models lacked both a global receptive field and high computational efficiency. In this work, we present the xLSTM-VMUNet Model, which jointly capture spatial and sequential features within dermatological images successfully. xLSTM-VMUNet can not only specialize in extracting spatial features from images, focusing on the structural characteristics of skin lesions, but also enhance contextual understanding, allowing more effective handling of complex medical image structures. Experiment results on the ISIC2018 dataset demonstrate that xLSTM-VMUNet outperforms VMUNet by 4.85% on DSC and 6.41% on IoU on the ISIC2017 dataset, by 1.25% on DSC and 2.07% on IoU on the ISIC2018 dataset, with faster convergence and consistently high segmentation performance. Our code is available at https://github.com/FangZhuoyi/XLSTM-VMUNet. △ Less

Submitted 12 March, 2025; v1 submitted 14 November, 2024; originally announced November 2024.

GPT-4o System Card

Authors: OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis , et al. (395 additional authors not shown)

Abstract: GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 mil… ▽ More GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities. △ Less

Submitted 25 October, 2024; originally announced October 2024.

Discrete-time Integral Resonant Control of Negative Imaginary Systems: Application to a High-speed Nanopositioner

Authors: Kanghong Shi, Erfan Khodabakhshi, Prosanto Biswas, Ian R. Petersen, S. O. Reza Moheimani

Abstract: We propose a discrete-time integral resonant control (IRC) approach for negative imaginary (NI) systems, which overcomes several limitations of continuous-time IRC. We show that a discrete-time IRC has a step-advanced negative imaginary property. A zero-order hold-sampled NI system can be asymptotically stabilized using a discrete-time IRC with suitable parameters. A hardware experiment is conduct… ▽ More We propose a discrete-time integral resonant control (IRC) approach for negative imaginary (NI) systems, which overcomes several limitations of continuous-time IRC. We show that a discrete-time IRC has a step-advanced negative imaginary property. A zero-order hold-sampled NI system can be asymptotically stabilized using a discrete-time IRC with suitable parameters. A hardware experiment is conducted where a high-speed flexure-guided nanopositioner is efficiently damped using the proposed discrete-time IRC with the discrete-time controller being implemented in FPGA hardware at the sampling rate of 1.25 MHz. △ Less

Submitted 23 June, 2024; originally announced June 2024.

Comments: 10 pages, 10 figures

Global-Local Convolution with Spiking Neural Networks for Energy-efficient Keyword Spotting

Authors: Shuai Wang, Dehao Zhang, Kexin Shi, Yuchen Wang, Wenjie Wei, Jibin Wu, Malu Zhang

Abstract: Thanks to Deep Neural Networks (DNNs), the accuracy of Keyword Spotting (KWS) has made substantial progress. However, as KWS systems are usually implemented on edge devices, energy efficiency becomes a critical requirement besides performance. Here, we take advantage of spiking neural networks' energy efficiency and propose an end-to-end lightweight KWS model. The model consists of two innovative… ▽ More Thanks to Deep Neural Networks (DNNs), the accuracy of Keyword Spotting (KWS) has made substantial progress. However, as KWS systems are usually implemented on edge devices, energy efficiency becomes a critical requirement besides performance. Here, we take advantage of spiking neural networks' energy efficiency and propose an end-to-end lightweight KWS model. The model consists of two innovative modules: 1) Global-Local Spiking Convolution (GLSC) module and 2) Bottleneck-PLIF module. Compared to the hand-crafted feature extraction methods, the GLSC module achieves speech feature extraction that is sparser, more energy-efficient, and yields better performance. The Bottleneck-PLIF module further processes the signals from GLSC with the aim to achieve higher accuracy with fewer parameters. Extensive experiments are conducted on the Google Speech Commands Dataset (V1 and V2). The results show our method achieves competitive performance among SNN-based KWS models with fewer parameters. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Unified Control of Voltage, Frequency and Angle in Electrical Power Systems: A Passivity and Negative-Imaginary based Approach

Authors: Yijun Chen, Kanghong Shi, Ian R. Petersen, Elizabeth L. Ratnam

Abstract: This paper proposes a unified methodology for voltage regulation, frequency synchronization, and rotor angle control in power transmission systems considering a one-axis generator model with time-varying voltages. First, we formulate an output consensus problem with a passivity and negative-imaginary (NI) based control framework. We establish output consensus results for both networked passive sys… ▽ More This paper proposes a unified methodology for voltage regulation, frequency synchronization, and rotor angle control in power transmission systems considering a one-axis generator model with time-varying voltages. First, we formulate an output consensus problem with a passivity and negative-imaginary (NI) based control framework. We establish output consensus results for both networked passive systems and networked NI systems. Next, we apply the output consensus problem by controlling large-scale batteries co-located with synchronous generators -- using real-time voltage phasor measurements. By controlling the battery storage systems so as to dispatch real and reactive power, we enable simultaneous control of voltage, frequency, and power angle differences across a transmission network. Validation through numerical simulations on a four-area transmission network confirms the robustness of our unified control framework. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: 8 pages, 7 figures, the 63rd IEEE Conference on Decision and Control. arXiv admin note: text overlap with arXiv:2406.01206

On the Stability of Networked Nonlinear Negative Imaginary Systems with Applications to Electrical Power Systems

Authors: Yijun Chen, Kanghong Shi, Ian R. Petersen, Elizabeth L. Ratnam

Abstract: In the transition to achieving net zero emissions, it has been suggested that a substantial expansion of electric power grids will be necessary to support emerging renewable energy zones. In this paper, we propose employing battery-based feedback control and nonlinear negative imaginary (NI) systems theory to reduce the need for such expansion. By formulating a novel Luré-Postnikov-like Lyapunov f… ▽ More In the transition to achieving net zero emissions, it has been suggested that a substantial expansion of electric power grids will be necessary to support emerging renewable energy zones. In this paper, we propose employing battery-based feedback control and nonlinear negative imaginary (NI) systems theory to reduce the need for such expansion. By formulating a novel Luré-Postnikov-like Lyapunov function, stability results are presented for the feedback interconnection of two single nonlinear NI systems, while output feedback consensus results are established for the feedback interconnection of two networked nonlinear NI systems based on a network topology. This theoretical framework underpins our design of battery-based control in power transmission systems. We demonstrate that the power grid can be gradually transitioned into the proposed NI systems, one transmission line at a time. △ Less

Submitted 11 July, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Comments: 8 pages, 2 figures, 26th International Symposium on Mathematical Theory of Networks and Systems

Enhancing Global Sensitivity and Uncertainty Quantification in Medical Image Reconstruction with Monte Carlo Arbitrary-Masked Mamba

Authors: Jiahao Huang, Liutao Yang, Fanwen Wang, Yang Nan, Weiwen Wu, Chengyan Wang, Kuangyu Shi, Angelica I. Aviles-Rivero, Carola-Bibiane Schönlieb, Daoqiang Zhang, Guang Yang

Abstract: Deep learning has been extensively applied in medical image reconstruction, where Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) represent the predominant paradigms, each possessing distinct advantages and inherent limitations: CNNs exhibit linear complexity with local sensitivity, whereas ViTs demonstrate quadratic complexity with global sensitivity. The emerging Mamba has sh… ▽ More Deep learning has been extensively applied in medical image reconstruction, where Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) represent the predominant paradigms, each possessing distinct advantages and inherent limitations: CNNs exhibit linear complexity with local sensitivity, whereas ViTs demonstrate quadratic complexity with global sensitivity. The emerging Mamba has shown superiority in learning visual representation, which combines the advantages of linear scalability and global sensitivity. In this study, we introduce MambaMIR, an Arbitrary-Masked Mamba-based model with wavelet decomposition for joint medical image reconstruction and uncertainty estimation. A novel Arbitrary Scan Masking (ASM) mechanism "masks out" redundant information to introduce randomness for further uncertainty estimation. Compared to the commonly used Monte Carlo (MC) dropout, our proposed MC-ASM provides an uncertainty map without the need for hyperparameter tuning and mitigates the performance drop typically observed when applying dropout to low-level tasks. For further texture preservation and better perceptual quality, we employ the wavelet transformation into MambaMIR and explore its variant based on the Generative Adversarial Network, namely MambaMIR-GAN. Comprehensive experiments have been conducted for multiple representative medical image reconstruction tasks, demonstrating that the proposed MambaMIR and MambaMIR-GAN outperform other baseline and state-of-the-art methods in different reconstruction tasks, where MambaMIR achieves the best reconstruction fidelity and MambaMIR-GAN has the best perceptual quality. In addition, our MC-ASM provides uncertainty maps as an additional tool for clinicians, while mitigating the typical performance drop caused by the commonly used dropout. △ Less

Submitted 25 June, 2024; v1 submitted 27 May, 2024; originally announced May 2024.

Dose-aware Diffusion Model for 3D PET Image Denoising: Multi-institutional Validation with Reader Study and Real Low-dose Data

Authors: Huidong Xie, Weijie Gan, Reimund Bayerlein, Bo Zhou, Ming-Kai Chen, Michal Kulon, Annemarie Boustani, Kuan-Yin Ko, Der-Shiun Wang, Benjamin A. Spencer, Wei Ji, Xiongchao Chen, Qiong Liu, Xueqi Guo, Menghua Xia, Yinchi Zhou, Hui Liu, Liang Guo, Hongyu An, Ulugbek S. Kamilov, Hanzhong Wang, Biao Li, Axel Rominger, Kuangyu Shi, Ge Wang , et al. (2 additional authors not shown)

Abstract: Reducing scan times, radiation dose, and enhancing image quality for lower-performance scanners, are critical in low-dose PET imaging. Deep learning techniques have been investigated for PET image denoising. However, existing models have often resulted in compromised image quality when achieving low-count/low-dose PET and have limited generalizability to different image noise-levels, acquisition p… ▽ More Reducing scan times, radiation dose, and enhancing image quality for lower-performance scanners, are critical in low-dose PET imaging. Deep learning techniques have been investigated for PET image denoising. However, existing models have often resulted in compromised image quality when achieving low-count/low-dose PET and have limited generalizability to different image noise-levels, acquisition protocols, and patient populations. Recently, diffusion models have emerged as the new state-of-the-art generative model to generate high-quality samples and have demonstrated strong potential for medical imaging tasks. However, for low-dose PET imaging, existing diffusion models failed to generate consistent 3D reconstructions, unable to generalize across varying noise-levels, often produced visually-appealing but distorted image details, and produced images with biased tracer uptake. Here, we develop DDPET-3D, a dose-aware diffusion model for 3D low-dose PET imaging to address these challenges. Collected from 4 medical centers globally with different scanners and clinical protocols, we evaluated the proposed model using a total of 9,783 18F-FDG studies with low-dose levels ranging from 1% to 50%. With a cross-center, cross-scanner validation, the proposed DDPET-3D demonstrated its potential to generalize to different low-dose levels, different scanners, and different clinical protocols. As confirmed with reader studies performed by board-certified nuclear medicine physicians, experienced readers judged the images to be similar or superior to the full-dose images and previous DL baselines based on qualitative visual impression. Lesion-level quantitative accuracy was evaluated using a Monte Carlo simulation study and a lesion segmentation network. The presented results show the potential to achieve low-dose PET while maintaining image quality. Real low-dose scans was also included for evaluation. △ Less

Submitted 16 June, 2025; v1 submitted 2 May, 2024; originally announced May 2024.

Comments: 18 Pages, 16 Figures, 5 Tables. Paper under review. First-place Freek J. Beekman Young Investigator Award at SNMMI 2024. Code available after paper publication. arXiv admin note: substantial text overlap with arXiv:2311.04248

LeqMod: Adaptable Lesion-Quantification-Consistent Modulation for Deep Learning Low-Count PET Image Denoising

Authors: Menghua Xia, Huidong Xie, Qiong Liu, Bo Zhou, Hanzhong Wang, Biao Li, Axel Rominger, Quanzheng Li, Ramsey D. Badawi, Kuangyu Shi, Georges El Fakhri, Chi Liu

Abstract: Deep learning-based positron emission tomography (PET) image denoising offers the potential to reduce radiation exposure and scanning time by transforming low-count images into high-count equivalents. However, existing methods typically blur crucial details, leading to inaccurate lesion quantification. This paper proposes a lesion-perceived and quantification-consistent modulation (LeqMod) strateg… ▽ More Deep learning-based positron emission tomography (PET) image denoising offers the potential to reduce radiation exposure and scanning time by transforming low-count images into high-count equivalents. However, existing methods typically blur crucial details, leading to inaccurate lesion quantification. This paper proposes a lesion-perceived and quantification-consistent modulation (LeqMod) strategy for enhanced PET image denoising, via employing downstream lesion quantification analysis as auxiliary tools. The LeqMod is a plug-and-play design adaptable to a wide range of model architectures, modulating the sampling and optimization procedures of model training without adding any computational burden to the inference phase. Specifically, the LeqMod consists of two components, the lesion-perceived modulation (LeMod) and the multiscale quantification-consistent modulation (QuMod). The LeMod enhances lesion contrast and visibility by allocating higher sampling weights and stricter loss criteria to lesion-present samples determined by an auxiliary segmentation network than lesion-absent ones. The QuMod further emphasizes quantification accuracy for both the mean and maximum standardized uptake value (SUVmean and SUVmax) across multiscale sub-regions throughout the entire image, thereby reducing biases of denoised results relative to high-count references. Experiments conducted on large PET datasets from multiple centers and vendors, and varying noise levels demonstrated the LeqMod efficacy across various denoising frameworks. Compared to frameworks without LeqMod, the integration of LeqMod reduces the lesion SUVmax bias by 5.92% on average and increases the peak signal-to-noise ratio (PSNR) by 0.36 on average, when denoising images across participating sites. △ Less

Submitted 4 March, 2025; v1 submitted 27 April, 2024; originally announced April 2024.

Digital control of negative imaginary systems: a discrete-time hybrid integrator-gain system approach

Authors: Kanghong Shi, Ian R. Petersen

Abstract: A hybrid integrator-gain system (HIGS) is a control element that switches between an integrator and a gain, which overcomes some inherent limitations of linear controllers. In this paper, we consider using discrete-time HIGS controllers for the digital control of negative imaginary (NI) systems. We show that the discrete-time HIGS themselves are step-advanced negative imaginary systems. For a mini… ▽ More A hybrid integrator-gain system (HIGS) is a control element that switches between an integrator and a gain, which overcomes some inherent limitations of linear controllers. In this paper, we consider using discrete-time HIGS controllers for the digital control of negative imaginary (NI) systems. We show that the discrete-time HIGS themselves are step-advanced negative imaginary systems. For a minimal linear NI system, there always exists a HIGS controller that can asymptotically stablize it. An illustrative example is provided, where we use the proposed HIGS control method to stabilize a discrete-time mass-spring system. △ Less

Submitted 24 March, 2024; originally announced March 2024.

Comments: To appear in the 2024 European Control Conference. 7 pages, 3 figures

Hybrid integrator-gain system based integral resonant controllers for negative imaginary systems

Authors: Kanghong Shi, Ian R. Petersen

Abstract: We introduce a hybrid control system called a hybrid integrator-gain system (HIGS) based integral resonant controller (IRC) to stabilize negative imaginary (NI) systems. A HIGS-based IRC has a similar structure to an IRC, with the integrator replaced by a HIGS. We show that a HIGS-based IRC is an NI system. Also, for a SISO NI system with a minimal realization, we show there exists a HIGS-based IR… ▽ More We introduce a hybrid control system called a hybrid integrator-gain system (HIGS) based integral resonant controller (IRC) to stabilize negative imaginary (NI) systems. A HIGS-based IRC has a similar structure to an IRC, with the integrator replaced by a HIGS. We show that a HIGS-based IRC is an NI system. Also, for a SISO NI system with a minimal realization, we show there exists a HIGS-based IRC such that their closed-loop interconnection is asymptotically stable. Also, we propose a proportional-integral-double-integral resonant controller and a HIGS-based proportional-integral-double-integral resonant controller, and we show that both of them can be applied to asymptotically stabilize an NI system. An example is provided to illustrate the proposed results. △ Less

Submitted 9 September, 2024; v1 submitted 22 March, 2024; originally announced March 2024.

Comments: 9 pages, 9 figures. The 63rd IEEE Conference on Decision and Control (CDC 2024)

Discrete-time Negative Imaginary Systems from ZOH Sampling

Authors: Kanghong Shi, Ian R. Petersen, Igor G. Vladimirov

Abstract: A new definition of discrete-time negative imaginary (NI) systems is provided. This definition characterizes the dissipative property of a zero-order hold sampled continuous-time NI system. Under some assumptions, asymptotic stability can be guaranteed for the closed-loop interconnection of an NI system and an output strictly negative imaginary system, with one of them having a one step advance. I… ▽ More A new definition of discrete-time negative imaginary (NI) systems is provided. This definition characterizes the dissipative property of a zero-order hold sampled continuous-time NI system. Under some assumptions, asymptotic stability can be guaranteed for the closed-loop interconnection of an NI system and an output strictly negative imaginary system, with one of them having a one step advance. In the case of linear systems, we also provide necessary and sufficient frequency-domain and LMI conditions under which the definition is satisfied. Also provided is a simple DC gain condition for the stability results in the linear case. △ Less

Submitted 11 June, 2024; v1 submitted 8 December, 2023; originally announced December 2023.

Comments: 8 pages, 4 figures. 26th International Symposium on Mathematical Theory of Networks and Systems

A Nonlinear Negative Imaginary Systems Framework with Actuator Saturation for Control of Electrical Power Systems

Authors: Yijun Chen, Kanghong Shi, Ian R. Petersen, Elizabeth L. Ratnam

Abstract: In the transition to net zero, it has been suggested that a massive expansion of the electric power grid will be required to support emerging renewable energy zones. In this paper, we propose the use of battery-based feedback control and nonlinear negative imaginary systems theory to reduce the need for such an expansion by enabling the more complete utilization of existing grid infrastructure. By… ▽ More In the transition to net zero, it has been suggested that a massive expansion of the electric power grid will be required to support emerging renewable energy zones. In this paper, we propose the use of battery-based feedback control and nonlinear negative imaginary systems theory to reduce the need for such an expansion by enabling the more complete utilization of existing grid infrastructure. By constructing a novel Lur'e-Postnikov-like Lyapunov function, a stability result is developed for the feedback interconnection of a nonlinear negative imaginary system and a nonlinear negative imaginary controller. Additionally, a new class of nonlinear negative imaginary controllers is proposed to deal with actuator saturation. We show that in this control framework, the controller eventually leaves the saturation boundary, and the feedback system is locally stable in the sense of Lyapunov. This provides theoretical support for the application of battery-based control in electrical power systems. Validation through simulation results for single-machine-infinite-bus power systems supports our results. Our approach has the potential to enable a transmission line to operate at its maximum power capacity, as stability robustness is ensured by the use of a feedback controller. △ Less

Submitted 12 November, 2023; originally announced November 2023.

Comments: 8 pages, 5 figures, European Control Conference

Negative Imaginary Control Using Hybrid Integrator-Gain Systems: Application to MEMS Nanopositioner

Authors: Kanghong Shi, Nastaran Nikooienejad, Ian R. Petersen, S. O. Reza Moheimani

Abstract: In this paper, we propose a new approach to address the control problem for negative imaginary (NI) systems by using hybrid integrator-gain systems (HIGS). We investigate the single HIGS of its original form and its two variations, including a multi-HIGS and the serial cascade of two HIGS. A single HIGS is shown to be a nonlinear negative imaginary system, and so is the multi-HIGS and the cascade… ▽ More In this paper, we propose a new approach to address the control problem for negative imaginary (NI) systems by using hybrid integrator-gain systems (HIGS). We investigate the single HIGS of its original form and its two variations, including a multi-HIGS and the serial cascade of two HIGS. A single HIGS is shown to be a nonlinear negative imaginary system, and so is the multi-HIGS and the cascade of two HIGS. We show that these three types of HIGS can be used as controllers to asymptotically stabilize linear NI systems. The results of this paper are then illustrated in a real-world experiment where a 2-DOF microelectromechanical system nanopositioner is stabilized by a multi-HIGS. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Comments: 13 pages, 9 figures. Accepted for publication as a Full Paper in the IEEE Transactions on Control Systems Technology (TCST)

Extreme Parkour with Legged Robots

Authors: Xuxin Cheng, Kexin Shi, Ananye Agarwal, Deepak Pathak

Abstract: Humans can perform parkour by traversing obstacles in a highly dynamic fashion requiring precise eye-muscle coordination and movement. Getting robots to do the same task requires overcoming similar challenges. Classically, this is done by independently engineering perception, actuation, and control systems to very low tolerances. This restricts them to tightly controlled settings such as a predete… ▽ More Humans can perform parkour by traversing obstacles in a highly dynamic fashion requiring precise eye-muscle coordination and movement. Getting robots to do the same task requires overcoming similar challenges. Classically, this is done by independently engineering perception, actuation, and control systems to very low tolerances. This restricts them to tightly controlled settings such as a predetermined obstacle course in labs. In contrast, humans are able to learn parkour through practice without significantly changing their underlying biology. In this paper, we take a similar approach to developing robot parkour on a small low-cost robot with imprecise actuation and a single front-facing depth camera for perception which is low-frequency, jittery, and prone to artifacts. We show how a single neural net policy operating directly from a camera image, trained in simulation with large-scale RL, can overcome imprecise sensing and actuation to output highly precise control behavior end-to-end. We show our robot can perform a high jump on obstacles 2x its height, long jump across gaps 2x its length, do a handstand and run across tilted ramps, and generalize to novel obstacle courses with different physical properties. Parkour videos at https://extreme-parkour.github.io/ △ Less

Submitted 25 September, 2023; originally announced September 2023.

Comments: Website and videos at https://extreme-parkour.github.io/

Self-Supervised Learning for Physiologically-Based Pharmacokinetic Modeling in Dynamic PET

Authors: Francesca De Benetti, Walter Simson, Magdalini Paschali, Hasan Sari, Axel Romiger, Kuangyu Shi, Nassir Navab, Thomas Wendler

Abstract: Dynamic positron emission tomography imaging (dPET) provides temporally resolved images of a tracer enabling a quantitative measure of physiological processes. Voxel-wise physiologically-based pharmacokinetic (PBPK) modeling of the time activity curves (TAC) can provide relevant diagnostic information for clinical workflow. Conventional fitting strategies for TACs are slow and ignore the spatial r… ▽ More Dynamic positron emission tomography imaging (dPET) provides temporally resolved images of a tracer enabling a quantitative measure of physiological processes. Voxel-wise physiologically-based pharmacokinetic (PBPK) modeling of the time activity curves (TAC) can provide relevant diagnostic information for clinical workflow. Conventional fitting strategies for TACs are slow and ignore the spatial relation between neighboring voxels. We train a spatio-temporal UNet to estimate the kinetic parameters given TAC from F-18-fluorodeoxyglucose (FDG) dPET. This work introduces a self-supervised loss formulation to enforce the similarity between the measured TAC and those generated with the learned kinetic parameters. Our method provides quantitatively comparable results at organ-level to the significantly slower conventional approaches, while generating pixel-wise parametric images which are consistent with expected physiology. To the best of our knowledge, this is the first self-supervised network that allows voxel-wise computation of kinetic parameters consistent with a non-linear kinetic model. The code will become publicly available upon acceptance. △ Less

Submitted 17 May, 2023; originally announced May 2023.

Nonlinear Negative Imaginary Systems with Switching

Authors: Kanghong Shi, Ian R. Petersen, Igor G. Vladimirov

Abstract: In this paper, we extend nonlinear negative imaginary (NI) systems theory to switched systems. Switched nonlinear NI systems and switched nonlinear output strictly negative imaginary (OSNI) systems are defined. We show that the interconnection of two switched nonlinear NI systems is still switched nonlinear NI. The interconnection of a switched nonlinear NI system and a switched nonlinear OSNI sys… ▽ More In this paper, we extend nonlinear negative imaginary (NI) systems theory to switched systems. Switched nonlinear NI systems and switched nonlinear output strictly negative imaginary (OSNI) systems are defined. We show that the interconnection of two switched nonlinear NI systems is still switched nonlinear NI. The interconnection of a switched nonlinear NI system and a switched nonlinear OSNI system is asymptotically stable under some assumptions. This stability result is then illustrated using a numerical example. △ Less

Submitted 2 April, 2023; originally announced April 2023.

Comments: 7 pages, 4 figures. Full archive version for the paper of the same title to appear in the proceedings of IFAC World Congress 2023

FedFTN: Personalized Federated Learning with Deep Feature Transformation Network for Multi-institutional Low-count PET Denoising

Authors: Bo Zhou, Huidong Xie, Qiong Liu, Xiongchao Chen, Xueqi Guo, Zhicheng Feng, Jun Hou, S. Kevin Zhou, Biao Li, Axel Rominger, Kuangyu Shi, James S. Duncan, Chi Liu

Abstract: Low-count PET is an efficient way to reduce radiation exposure and acquisition time, but the reconstructed images often suffer from low signal-to-noise ratio (SNR), thus affecting diagnosis and other downstream tasks. Recent advances in deep learning have shown great potential in improving low-count PET image quality, but acquiring a large, centralized, and diverse dataset from multiple institutio… ▽ More Low-count PET is an efficient way to reduce radiation exposure and acquisition time, but the reconstructed images often suffer from low signal-to-noise ratio (SNR), thus affecting diagnosis and other downstream tasks. Recent advances in deep learning have shown great potential in improving low-count PET image quality, but acquiring a large, centralized, and diverse dataset from multiple institutions for training a robust model is difficult due to privacy and security concerns of patient data. Moreover, low-count PET data at different institutions may have different data distribution, thus requiring personalized models. While previous federated learning (FL) algorithms enable multi-institution collaborative training without the need of aggregating local data, addressing the large domain shift in the application of multi-institutional low-count PET denoising remains a challenge and is still highly under-explored. In this work, we propose FedFTN, a personalized federated learning strategy that addresses these challenges. FedFTN uses a local deep feature transformation network (FTN) to modulate the feature outputs of a globally shared denoising network, enabling personalized low-count PET denoising for each institution. During the federated learning process, only the denoising network's weights are communicated and aggregated, while the FTN remains at the local institutions for feature transformation. We evaluated our method using a large-scale dataset of multi-institutional low-count PET imaging data from three medical centers located across three continents, and showed that FedFTN provides high-quality low-count PET images, outperforming previous baseline FL reconstruction methods across all low-count levels at all three institutions. △ Less

Submitted 6 October, 2023; v1 submitted 2 April, 2023; originally announced April 2023.

Comments: 13 pages, 6 figures, Accepted at Medical Image Analysis Journal (MedIA)

Learned Smartphone ISP on Mobile GPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report

Authors: Andrey Ignatov, Radu Timofte, Shuai Liu, Chaoyu Feng, Furui Bai, Xiaotao Wang, Lei Lei, Ziyao Yi, Yan Xiang, Zibin Liu, Shaoqing Li, Keming Shi, Dehui Kong, Ke Xu, Minsu Kwon, Yaqi Wu, Jiesi Zheng, Zhihao Fan, Xun Wu, Feng Zhang, Albert No, Minhyeok Cho, Zewen Chen, Xiaze Zhang, Ran Li , et al. (13 additional authors not shown)

Abstract: The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. Th… ▽ More The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper. △ Less

Submitted 7 November, 2022; originally announced November 2022.

Attention Augmented ConvNeXt UNet For Rectal Tumour Segmentation

Authors: Hongwei Wu, Junlin Wang, Xin Wang, Hui Nan, Yaxin Wang, Haonan Jing, Kaixuan Shi

Abstract: It is a challenge to segment the location and size of rectal cancer tumours through deep learning. In this paper, in order to improve the ability of extracting suffi-cient feature information in rectal tumour segmentation, attention enlarged ConvNeXt UNet (AACN-UNet), is proposed. The network mainly includes two improvements: 1) the encoder stage of UNet is changed to ConvNeXt structure for encodi… ▽ More It is a challenge to segment the location and size of rectal cancer tumours through deep learning. In this paper, in order to improve the ability of extracting suffi-cient feature information in rectal tumour segmentation, attention enlarged ConvNeXt UNet (AACN-UNet), is proposed. The network mainly includes two improvements: 1) the encoder stage of UNet is changed to ConvNeXt structure for encoding operation, which can not only integrate multi-scale semantic information on a large scale, but al-so reduce information loss and extract more feature information from CT images; 2) CBAM attention mechanism is added to improve the connection of each feature in channel and space, which is conducive to extracting the effective feature of the target and improving the segmentation accuracy.The experiment with UNet and its variant network shows that AACN-UNet is 0.9% ,1.1% and 1.4% higher than the current best results in P, F1 and Miou.Compared with the training time, the number of parameters in UNet network is less. This shows that our proposed AACN-UNet has achieved ex-cellent results in CT image segmentation of rectal cancer. △ Less

Submitted 26 October, 2022; v1 submitted 1 October, 2022; originally announced October 2022.

Comments: I plan to replace this article, and supplement and confirm the structure and experimental content of this article

Application of the nnU-Net for automatic segmentation of lung lesion on CT images, and implication on radiomic models

Authors: Matteo Ferrante, Lisa Rinaldi, Francesca Botta, Xiaobin Hu, Andreas Dolp, Marta Minotti, Francesca De Piano, Gianluigi Funicelli, Stefania Volpe, Federica Bellerba, Paolo De Marco, Sara Raimondi, Stefania Rizzo, Kuangyu Shi, Marta Cremonesi, Barbara A. Jereczek-Fossa, Lorenzo Spaggiari, Filippo De Marinis, Roberto Orecchia, Daniela Origgi

Abstract: Lesion segmentation is a crucial step of the radiomic workflow. Manual segmentation requires long execution time and is prone to variability, impairing the realisation of radiomic studies and their robustness. In this study, a deep-learning automatic segmentation method was applied on computed tomography images of non-small-cell lung cancer patients. The use of manual vs automatic segmentation in… ▽ More Lesion segmentation is a crucial step of the radiomic workflow. Manual segmentation requires long execution time and is prone to variability, impairing the realisation of radiomic studies and their robustness. In this study, a deep-learning automatic segmentation method was applied on computed tomography images of non-small-cell lung cancer patients. The use of manual vs automatic segmentation in the performance of survival radiomic models was assessed, as well. METHODS A total of 899 NSCLC patients were included (2 proprietary: A and B, 1 public datasets: C). Automatic segmentation of lung lesions was performed by training a previously developed architecture, the nnU-Net, including 2D, 3D and cascade approaches. The quality of automatic segmentation was evaluated with DICE coefficient, considering manual contours as reference. The impact of automatic segmentation on the performance of a radiomic model for patient survival was explored by extracting radiomic hand-crafted and deep-learning features from manual and automatic contours of dataset A, and feeding different machine learning algorithms to classify survival above/below median. Models' accuracies were assessed and compared. RESULTS The best agreement between automatic and manual contours with DICE=0.78 +(0.12) was achieved by averaging predictions from 2D and 3D models, and applying a post-processing technique to extract the maximum connected component. No statistical differences were observed in the performances of survival models when using manual or automatic contours, hand-crafted, or deep features. The best classifier showed an accuracy between 0.65 and 0.78. CONCLUSION The promising role of nnU-Net for automatic segmentation of lung lesions was confirmed, dramatically reducing the time-consuming physicians' workload without impairing the accuracy of survival predictive models based on radiomics. △ Less

Submitted 24 September, 2022; originally announced September 2022.

A negative imaginary approach to hybrid integrator-gain system control

Authors: Kanghong Shi, Nastaran Nikooienejad, Ian R. Petersen, S. O. Reza Moheimani

Abstract: In this paper, we show that a hybrid integrator-gain system (HIGS) is a nonlinear negative imaginary (NNI) system. We prove that the positive feedback interconnection of a linear negative imaginary (NI) system and a HIGS is asymptotically stable. We apply the HIGS to a MEMS nanopositioner, as an example of a linear NI system, in a single-input single-output framework. We analyze the stability and… ▽ More In this paper, we show that a hybrid integrator-gain system (HIGS) is a nonlinear negative imaginary (NNI) system. We prove that the positive feedback interconnection of a linear negative imaginary (NI) system and a HIGS is asymptotically stable. We apply the HIGS to a MEMS nanopositioner, as an example of a linear NI system, in a single-input single-output framework. We analyze the stability and the performance of the closed-loop interconnection in both time and frequency domains through simulations and demonstrate the applicability of HIGS as an NNI controller to a linear NI system. △ Less

Submitted 23 March, 2023; v1 submitted 5 September, 2022; originally announced September 2022.

Comments: This paper was presented at the 61st IEEE Conference on Decision and Control (CDC), 2022. A short version was published in the proceedings of the conference

Negative Imaginary State Feedback Equivalence for a Class of Nonlinear Systems

Authors: Kanghong Shi, Ian R. Petersen, Igor G. Vladimirov

Abstract: In this paper, we investigate the necessary and sufficient conditions under which a class of nonlinear systems are state feedback equivalent to nonlinear negative imaginary (NI) systems with positive definite storage functions. The nonlinear systems of interest have a normal form of relative degree less than or equal to two. The nonlinearity of the system is restricted with respect to a subset of… ▽ More In this paper, we investigate the necessary and sufficient conditions under which a class of nonlinear systems are state feedback equivalent to nonlinear negative imaginary (NI) systems with positive definite storage functions. The nonlinear systems of interest have a normal form of relative degree less than or equal to two. The nonlinearity of the system is restricted with respect to a subset of the state variables, which are the state variables that have external dynamics. Under mild assumptions, such systems are state feedback equivalent to nonlinear NI systems and nonlinear output strictly negative imaginary (OSNI) systems if and only if they are weakly minimum phase. Such a state feedback control approach can also asymptotically stabilize the systems in question against nonlinear OSNI system uncertainties. A numerical example is provided to show the process of the state feedback equivalence control and stabilization of uncertain systems. △ Less

Submitted 7 June, 2022; originally announced June 2022.

Comments: 8 pages, 2 figures

Suboptimal Safety-Critical Control for Continuous Systems Using Prediction-Correction Online Optimization

Authors: Shengbo Wang, Shiping Wen, Yin Yang, Yuting Cao, Kaibo Shi, Tingwen Huang

Abstract: This paper investigates the control barrier function (CBF) based safety-critical control for continuous nonlinear control affine systems using the more efficient online algorithms through time-varying optimization. The idea lies in that when quadratic programming (QP) or other convex optimization algorithms needed in the CBF-based method is not computation affordable, the alternative suboptimal fe… ▽ More This paper investigates the control barrier function (CBF) based safety-critical control for continuous nonlinear control affine systems using the more efficient online algorithms through time-varying optimization. The idea lies in that when quadratic programming (QP) or other convex optimization algorithms needed in the CBF-based method is not computation affordable, the alternative suboptimal feasible solutions can be obtained more economically. By using the barrier-based interior point method, the constrained CBF-QP problems are converted into the unconstrained ones with suboptimal solutions tracked by two continuous descent-based algorithms. Considering the lag effect of tracking and exploiting the system information, the prediction method is added to the algorithms which thereby achieves a exponential convergence rate to the time-varying suboptimal solutions. The convergence and robustness of the designed methods as well as the safety criteria of the algorithms are analyzed theoretically. In the end, the effectiveness is illustrated by simulations on the anti-swing and obstacle avoidance tasks. △ Less

Submitted 20 March, 2023; v1 submitted 29 March, 2022; originally announced March 2022.

Making Nonlinear Systems Negative Imaginary via State Feedback

Authors: Kanghong Shi, Ian R. Petersen, Igor G. Vladimirov

Abstract: This paper provides a state feedback stabilization approach for nonlinear systems of relative degree less than or equal to two by rendering them nonlinear negative imaginary (NI) systems. Conditions are provided under which a nonlinear system can be made a nonlinear NI system or a nonlinear output strictly negative imaginary (OSNI) system. Roughly speaking, an affine nonlinear system that has a no… ▽ More This paper provides a state feedback stabilization approach for nonlinear systems of relative degree less than or equal to two by rendering them nonlinear negative imaginary (NI) systems. Conditions are provided under which a nonlinear system can be made a nonlinear NI system or a nonlinear output strictly negative imaginary (OSNI) system. Roughly speaking, an affine nonlinear system that has a normal form with relative degree less than or equal to two, after possible output transformation, can be rendered nonlinear NI and nonlinear OSNI. In addition, if the internal dynamics of the normal form are input-to-state stable, then there exists a state feedback input that stabilizes the system. This stabilization result is then extended to achieve stability for systems with a nonlinear NI uncertainty. △ Less

Submitted 5 September, 2022; v1 submitted 25 March, 2022; originally announced March 2022.

Comments: 10 pages, 2 figures

Robust Adaptive Safety-Critical Control for Unknown Systems with Finite-Time Element-Wise Parameter Estimation

Authors: Shengbo Wang, Bo Lyu, Shiping Wen, Kaibo Shi, Song Zhu, Tingwen Huang

Abstract: Safety is always one of the most critical principles for a system to be controlled. This paper investigates a safety-critical control scheme for unknown structured systems by using the control barrier function (CBF) method. Benefited from the dynamic regressor extension and mixing (DREM), an extended element-wise parameter identification law is utilized to dismiss the uncertainty. On the one hand,… ▽ More Safety is always one of the most critical principles for a system to be controlled. This paper investigates a safety-critical control scheme for unknown structured systems by using the control barrier function (CBF) method. Benefited from the dynamic regressor extension and mixing (DREM), an extended element-wise parameter identification law is utilized to dismiss the uncertainty. On the one hand, it is shown that the proposed control scheme can always guarantee the safety in the identification process with noised signal injection excitation, which was not considered in the previous study. On the other hand, the element-wise estimation process in DREM can minimize conservatism of the safe adaptive process compared to other existing adaptive CBF algorithms. The stability as well as the forward invariance of the presented safe control-estimation scheme is proved. Furthermore, the robustness of the scheme under bounded disturbances is analyzed, where a robust CBF with modest conditions is used to ensure safety. The framework is illustrated by simulations on adaptive cruise control, where the slope resistance of the following vehicle is robustly estimated in finite time against small disturbances and the potential crash risk is avoided by the proposed safe control scheme. △ Less

Submitted 14 January, 2022; v1 submitted 27 November, 2021; originally announced November 2021.

Optimal Tracking Control for Unknown Linear Systems with Finite-Time Parameter Estimation

Authors: Shengbo Wang, Shiping Wen, Kaibo Shi, Song Zhu, Tingwen Huang

Abstract: The optimal control input for linear systems can be solved from algebraic Riccati equation (ARE), from which it remains questionable to get the form of the exact solution. In engineering, the acceptable numerical solutions of ARE can be found by iteration or optimization. Recently, the gradient descent based numerical solutions has been proven effective to approximate the optimal ones. This paper… ▽ More The optimal control input for linear systems can be solved from algebraic Riccati equation (ARE), from which it remains questionable to get the form of the exact solution. In engineering, the acceptable numerical solutions of ARE can be found by iteration or optimization. Recently, the gradient descent based numerical solutions has been proven effective to approximate the optimal ones. This paper introduces this method to tracking problem for heterogeneous linear systems. Differently, the parameters in the dynamics of the linear systems are all assumed to be unknown, which is intractable since the gradient as well as the allowable initialization needs the prior knowledge of system dynamics. To solve this problem, the method named dynamic regressor extension and mix (DREM) is improved to estimate the parameter matrices in finite time. Besides, a discounted factor is introduced to ensure the existence of optimal solutions for heterogeneous systems. Two simulation experiments are given to illustrate the effectiveness. △ Less

Submitted 6 January, 2022; v1 submitted 27 November, 2021; originally announced November 2021.

Necessary and Sufficient Conditions for State Feedback Equivalence to Negative Imaginary Systems

Authors: Kanghong Shi, Ian R. Petersen, Igor G. Vladimirov

Abstract: In this paper, we present necessary and sufficient conditions under which a linear time-invariant (LTI) system is state feedback equivalent to a negative imaginary (NI) system. More precisely, we show that a minimal LTI strictly proper system can be rendered NI using full state feedback if and only if it can be output transformed into a system, which has relative degree less than or equal to two a… ▽ More In this paper, we present necessary and sufficient conditions under which a linear time-invariant (LTI) system is state feedback equivalent to a negative imaginary (NI) system. More precisely, we show that a minimal LTI strictly proper system can be rendered NI using full state feedback if and only if it can be output transformed into a system, which has relative degree less than or equal to two and is weakly minimum phase. We also considered the problems of state feedback equivalence to output strictly negative imaginary systems and strongly strict negative imaginary systems. Then we apply the NI state feedback equivalence result to robustly stabilize an uncertain system with strictly negative imaginary uncertainty. An example is provided to illustrate the proposed results, for the purpose of stabilizing an uncertain system. △ Less

Submitted 23 September, 2021; originally announced September 2021.

Comments: 14 pages, 1 figure. arXiv admin note: text overlap with arXiv:2103.05249

Multilingual Speech Evaluation: Case Studies on English, Malay and Tamil

Authors: Huayun Zhang, Ke Shi, Nancy F. Chen

Abstract: Speech evaluation is an essential component in computer-assisted language learning (CALL). While speech evaluation on English has been popular, automatic speech scoring on low resource languages remains challenging. Work in this area has focused on monolingual specific designs and handcrafted features stemming from resource-rich languages like English. Such approaches are often difficult to genera… ▽ More Speech evaluation is an essential component in computer-assisted language learning (CALL). While speech evaluation on English has been popular, automatic speech scoring on low resource languages remains challenging. Work in this area has focused on monolingual specific designs and handcrafted features stemming from resource-rich languages like English. Such approaches are often difficult to generalize to other languages, especially if we also want to consider suprasegmental qualities such as rhythm. In this work, we examine three different languages that possess distinct rhythm patterns: English (stress-timed), Malay (syllable-timed), and Tamil (mora-timed). We exploit robust feature representations inspired by music processing and vector representation learning. Empirical validations show consistent gains for all three languages when predicting pronunciation, rhythm and intonation performance. △ Less

Submitted 8 July, 2021; originally announced July 2021.

Comments: Accepted at INTERSPEECH 2021

NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results

Authors: Ren Yang, Radu Timofte, Jing Liu, Yi Xu, Xinjian Zhang, Minyi Zhao, Shuigeng Zhou, Kelvin C. K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy, Xin Li, Fanglong Liu, He Zheng, Lielin Jiang, Qi Zhang, Dongliang He, Fu Li, Qingqing Dang, Yibin Huang, Matteo Maggioni, Zhongqian Fu, Shuai Xiao, Cheng li, Thomas Tanay , et al. (47 additional authors not shown)

Abstract: This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results. In this challenge, the new Large-scale Diverse Video (LDV) dataset is employed. The challenge has three tracks. Tracks 1 and 2 aim at enhancing the videos compressed by HEVC at a fixed QP, while Track 3 is designed for enhancing the videos compressed by x265 at… ▽ More This paper reviews the first NTIRE challenge on quality enhancement of compressed video, with a focus on the proposed methods and results. In this challenge, the new Large-scale Diverse Video (LDV) dataset is employed. The challenge has three tracks. Tracks 1 and 2 aim at enhancing the videos compressed by HEVC at a fixed QP, while Track 3 is designed for enhancing the videos compressed by x265 at a fixed bit-rate. Besides, the quality enhancement of Tracks 1 and 3 targets at improving the fidelity (PSNR), and Track 2 targets at enhancing the perceptual quality. The three tracks totally attract 482 registrations. In the test phase, 12 teams, 8 teams and 11 teams submitted the final results of Tracks 1, 2 and 3, respectively. The proposed methods and solutions gauge the state-of-the-art of video quality enhancement. The homepage of the challenge: https://github.com/RenYang-home/NTIRE21_VEnh △ Less

Submitted 31 August, 2022; v1 submitted 21 April, 2021; originally announced April 2021.

Comments: Corrected the MOS values in Table 2, and corrected some minor typos

Negative Imaginary State Feedback Equivalence for Systems of Relative Degree One and Relative Degree Two

Authors: Kanghong Shi, Ian R. Petersen, Igor G. Vladimirov

Abstract: This paper presents necessary and sufficient conditions under which a linear system of relative degree either one or two is state feedback equivalent to a negative imaginary (NI) system. More precisely, we show for a class of linear time-invariant strictly proper systems, that such a system can be rendered minimal and NI using full state feedback if and only if it is controllable and weakly minimu… ▽ More This paper presents necessary and sufficient conditions under which a linear system of relative degree either one or two is state feedback equivalent to a negative imaginary (NI) system. More precisely, we show for a class of linear time-invariant strictly proper systems, that such a system can be rendered minimal and NI using full state feedback if and only if it is controllable and weakly minimum phase. A strongly strict negative imaginary state feedback equivalence result is also provided. The NI state feedback equivalence result is then applied in a robust stabilization problem for an uncertain system with a strictly negative imaginary uncertainty. △ Less

Submitted 4 November, 2021; v1 submitted 9 March, 2021; originally announced March 2021.

Comments: 12 pages, 2 figures

Output Feedback Consensus for Networked Heterogeneous Nonlinear Negative-Imaginary Systems with Free Body Motion

Authors: Kanghong Shi, Ian R. Petersen, Igor G. Vladimirov

Abstract: This paper provides a protocol to address the robust output feedback consensus problem for networked heterogeneous nonlinear negative-imaginary (NI) systems with free body dynamics. We extend the definition of nonlinear NI systems to allow for systems with free body motion. A new stability result is developed for the interconnection of a nonlinear NI system and a nonlinear output strictly negative… ▽ More This paper provides a protocol to address the robust output feedback consensus problem for networked heterogeneous nonlinear negative-imaginary (NI) systems with free body dynamics. We extend the definition of nonlinear NI systems to allow for systems with free body motion. A new stability result is developed for the interconnection of a nonlinear NI system and a nonlinear output strictly negative-imaginary (OSNI) system. Also, a class of networked nonlinear OSNI controllers is proposed to achieve output feedback consensus for heterogeneous networked nonlinear NI systems. We show that in this control framework, the system outputs converge to the same limit trajectory. This consensus protocol is illustrated by a numerical example. △ Less

Submitted 1 July, 2021; v1 submitted 30 November, 2020; originally announced November 2020.

Comments: 8 pages, 7 figures. arXiv admin note: text overlap with arXiv:2006.13505

Exploit Camera Raw Data for Video Super-Resolution via Hidden Markov Model Inference

Authors: Xiaohong Liu, Kangdi Shi, Zhe Wang, Jun Chen

Abstract: To the best of our knowledge, the existing deep-learning-based Video Super-Resolution (VSR) methods exclusively make use of videos produced by the Image Signal Processor (ISP) of the camera system as inputs. Such methods are 1) inherently suboptimal due to information loss incurred by non-invertible operations in ISP, and 2) inconsistent with the real imaging pipeline where VSR in fact serves as a… ▽ More To the best of our knowledge, the existing deep-learning-based Video Super-Resolution (VSR) methods exclusively make use of videos produced by the Image Signal Processor (ISP) of the camera system as inputs. Such methods are 1) inherently suboptimal due to information loss incurred by non-invertible operations in ISP, and 2) inconsistent with the real imaging pipeline where VSR in fact serves as a pre-processing unit of ISP. To address this issue, we propose a new VSR method that can directly exploit camera sensor data, accompanied by a carefully built Raw Video Dataset (RawVD) for training, validation, and testing. This method consists of a Successive Deep Inference (SDI) module and a reconstruction module, among others. The SDI module is designed according to the architectural principle suggested by a canonical decomposition result for Hidden Markov Model (HMM) inference; it estimates the target high-resolution frame by repeatedly performing pairwise feature fusion using deformable convolutions. The reconstruction module, built with elaborately designed Attention-based Residual Dense Blocks (ARDBs), serves the purpose of 1) refining the fused feature and 2) learning the color information needed to generate a spatial-specific transformation for accurate color correction. Extensive experiments demonstrate that owing to the informativeness of the camera raw data, the effectiveness of the network architecture, and the separation of super-resolution and color correction processes, the proposed method achieves superior VSR results compared to the state-of-the-art and can be adapted to any specific camera-ISP. Code and dataset are available at https://github.com/proteus1991/RawVSR. △ Less

Submitted 4 January, 2021; v1 submitted 24 August, 2020; originally announced August 2020.

Comments: 13 pages, 14 figures, accepted in IEEE Transactions on Image Processing

V2I Connectivity-Based Dynamic Queue-Jump Lane for Emergency Vehicles: A Deep Reinforcement Learning Approach

Authors: Haoran Su, Kejian Shi, Li Jin, Joseph Y. J. Chow

Abstract: Emergency vehicle (EMV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. A main reason behind EMV service delay is the lack of communication and cooperation between vehicles blocking EMVs. In this paper, we study the improvement of EMV service under V2I connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs) based on real-t… ▽ More Emergency vehicle (EMV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. A main reason behind EMV service delay is the lack of communication and cooperation between vehicles blocking EMVs. In this paper, we study the improvement of EMV service under V2I connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs) based on real-time coordination of connected vehicles. We develop a novel Markov decision process formulation for the DQJL problem, which explicitly accounts for the uncertainty of drivers' reaction to approaching EMVs. We propose a deep neural network-based reinforcement learning algorithm that efficiently computes the optimal coordination instructions. We also validate our approach on a micro-simulation testbed using Simulation of Urban Mobility (SUMO). Validation results show that with our proposed methodology, the centralized control system saves approximately 15\% EMV passing time than the benchmark system. △ Less

Submitted 29 May, 2021; v1 submitted 1 August, 2020; originally announced August 2020.

Comments: 20 pages, 6 figures

Robust Output Feedback Consensus for Networked Heterogeneous Nonlinear Negative-Imaginary Systems

Authors: Kanghong Shi, Igor G. Vladimirov, Ian R. Petersen

Abstract: This paper provides a control protocol for the robust output feedback consensus of networked heterogeneous nonlinear negative-imaginary (NI) systems. Heterogeneous nonlinear output strictly negative-imaginary (OSNI) controllers are applied in positive feedback according to the network topology to achieve output feedback consensus. The main contribution of this paper is extending the previous studi… ▽ More This paper provides a control protocol for the robust output feedback consensus of networked heterogeneous nonlinear negative-imaginary (NI) systems. Heterogeneous nonlinear output strictly negative-imaginary (OSNI) controllers are applied in positive feedback according to the network topology to achieve output feedback consensus. The main contribution of this paper is extending the previous studies of the robust output feedback consensus problem for networked heterogeneous linear NI systems to nonlinear NI systems. Output feedback consensus is proved by investigating the internal stability of the closed-loop interconnection of the network of heterogeneous nonlinear NI plants and the network of heterogeneous nonlinear OSNI controllers according to the network topology. The network of heterogeneous nonlinear NI systems is proved to be also a nonlinear NI system, and the network of heterogeneous nonlinear OSNI systems is proved to be also a nonlinear OSNI system. Under suitable conditions, the nonlinear OSNI controllers lead to the convergence of the outputs of all nonlinear NI plants to a common limit trajectory, regardless of the system model of each plant. Hence, the protocol is robust with respect to parameter perturbation in the system models of the heterogeneous nonlinear NI plants in the network. △ Less

Submitted 30 November, 2020; v1 submitted 24 June, 2020; originally announced June 2020.

Comments: 6 pages, 9 figures

Robust Output Feedback Consensus for Networked Identical Nonlinear Negative-Imaginary Systems

Authors: Kanghong Shi, Igor G. Vladimirov, Ian R. Petersen

Abstract: A robust output feedback consensus problem for networked identical nonlinear negative-imaginary (NI) systems is investigated in this paper. Output consensus is achieved by applying identical linear output strictly negative-imaginary (OSNI) controllers to all the nonlinear NI plants in positive feedback through the network topology. First, we extend the definition of nonlinear NI systems from singl… ▽ More A robust output feedback consensus problem for networked identical nonlinear negative-imaginary (NI) systems is investigated in this paper. Output consensus is achieved by applying identical linear output strictly negative-imaginary (OSNI) controllers to all the nonlinear NI plants in positive feedback through the network topology. First, we extend the definition of nonlinear NI systems from single-input single-output (SISO) systems to multiple-input multiple-output (MIMO) systems and also extend the definition of OSNI systems to nonlinear scenarios. Asymptotic stability is proved for the closed-loop interconnection of a nonlinear NI system and a nonlinear OSNI system under reasonable assumptions. Then, an NI property and an OSNI-like property are proved for networked identical nonlinear NI systems and networked identical linear OSNI systems, respectively. Output feedback consensus is proved for a network of identical nonlinear NI plants by investigating the stability of its closed-loop interconnection with a network of linear OSNI controllers. This closed-loop interconnection is proposed as a protocol to deal with the output consensus problem for networked identical nonlinear NI systems and is robust against uncertainty in the individual system's model. △ Less

Submitted 8 April, 2021; v1 submitted 23 May, 2020; originally announced May 2020.

Comments: 8 pages, 8 figures

Dynamic Queue-Jump Lane for Emergency Vehicles under Partially Connected Settings: A Multi-Agent Deep Reinforcement Learning Approach

Authors: Haoran Su, Kejian Shi, Joseph. Y. J. Chow, Li Jin

Abstract: Emergency vehicle (EMV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. The main reason behind EMV service delay is the lack of communication and cooperation between vehicles blocking EMVs. In this paper, we study the improvement of EMV service under V2X connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs) based on real… ▽ More Emergency vehicle (EMV) service is a key function of cities and is exceedingly challenging due to urban traffic congestion. The main reason behind EMV service delay is the lack of communication and cooperation between vehicles blocking EMVs. In this paper, we study the improvement of EMV service under V2X connectivity. We consider the establishment of dynamic queue jump lanes (DQJLs) based on real-time coordination of connected vehicles in the presence of non-connected human-driven vehicles. We develop a novel Markov decision process formulation for the DQJL coordination strategies, which explicitly accounts for the uncertainty of drivers' yielding pattern to approaching EMVs. Based on pairs of neural networks representing actors and critics for agent vehicles, we develop a multi-agent actor-critic deep reinforcement learning algorithm that handles a varying number of vehicles and a random proportion of connected vehicles in the traffic. Approaching the optimal coordination strategies via indirect and direct reinforcement learning, we present two schemata to address multi-agent reinforcement learning on this connected vehicle application. Both approaches are validated, on a micro-simulation testbed SUMO, to establish a DQJL fast and safely. Validation results reveal that, with DQJL coordination strategies, it saves up to 30% time for EMVs to pass a link-level intelligent urban roadway than the baseline scenario. △ Less

Submitted 15 January, 2021; v1 submitted 2 March, 2020; originally announced March 2020.

Comments: 42 pages, 13 figures, 7 tables

Learning Instance Occlusion for Panoptic Segmentation

Authors: Justin Lazarow, Kwonjoon Lee, Kunyu Shi, Zhuowen Tu

Abstract: Panoptic segmentation requires segments of both "things" (countable object instances) and "stuff" (uncountable and amorphous regions) within a single output. A common approach involves the fusion of instance segmentation (for "things") and semantic segmentation (for "stuff") into a non-overlapping placement of segments, and resolves overlaps. However, instance ordering with detection confidence do… ▽ More Panoptic segmentation requires segments of both "things" (countable object instances) and "stuff" (uncountable and amorphous regions) within a single output. A common approach involves the fusion of instance segmentation (for "things") and semantic segmentation (for "stuff") into a non-overlapping placement of segments, and resolves overlaps. However, instance ordering with detection confidence do not correlate well with natural occlusion relationship. To resolve this issue, we propose a branch that is tasked with modeling how two instance masks should overlap one another as a binary relation. Our method, named OCFusion, is lightweight but particularly effective in the instance fusion process. OCFusion is trained with the ground truth relation derived automatically from the existing dataset annotations. We obtain state-of-the-art results on COCO and show competitive results on the Cityscapes panoptic segmentation benchmark. △ Less

Submitted 8 April, 2020; v1 submitted 13 June, 2019; originally announced June 2019.

Comments: Accepted to CVPR 2020

Comparison of Low Complexity Coherent Receivers for UDWDM-PONs ($λ$-to-the-user)

Authors: M. Sezer Erkılınç, Domaniç Lavery, Kai Shi, Benn C. Thomsen, Robert I. Killey, Seb J. Savory, Polina Bayvel

Abstract: It is predicted that demand in optical access networks will reach multi-Gb/s per user. However, the limited performance of the direct detection receiver technology currently used in the optical network units at the customers' premises restricts data rates/user. Therefore, the concept of coherent-enabled access networks has attracted attention in recent years, as this technology offers high receive… ▽ More It is predicted that demand in optical access networks will reach multi-Gb/s per user. However, the limited performance of the direct detection receiver technology currently used in the optical network units at the customers' premises restricts data rates/user. Therefore, the concept of coherent-enabled access networks has attracted attention in recent years, as this technology offers high receiver sensitivity, inherent frequency selectivity, and linear field detection enabling the full compensation of linear channel impairments. However, the complexity of conventional (dual-polarisation digital) coherent receivers has so far prevented their introduction into access networks. Thus, to exploit the benefits of coherent technology in the ONUs, low complexity coherent receivers, suitable for implementation in ONUs, are needed. In this paper, the recently proposed low complexity coherent (i.e., polarisation-independent Alamouti-coding heterodyne) receiver is, for the first time, compared in terms of its minimum receiver sensitivity with five previously reported receiver designs, including a detailed discussion on their advantages and limitations. It is shown that the Alamouti-coding based receiver approach allows the lowest number of photons per bit (PPB) transmitted (with a lower bound of 15.5 PPB in an ideal system simulations) whilst requiring the lowest optical receiver hardware complexity. It also exhibits comparable complexity to the currently deployed direct-detection receivers, which typically require >1000 PPB. Finally, a comparison of experimentally achieved receiver sensitivities and transmission distances using these receivers is presented. The highest spectral efficiency and longest transmission distance at the highest bit rate reported using the Alamouti-coding receiver, which is also the only one, to date, to have been demonstrated in a full system bidirectional transmission. △ Less

Submitted 28 February, 2018; v1 submitted 2 November, 2017; originally announced November 2017.

Comments: 12 pages, 10 figures, 2 tables and 46 referecens