Showing 1–50 of 69 results for author: Qi, J

Multimodal Fusion with Semi-Supervised Learning Minimizes Annotation Quantity for Modeling Videoconference Conversation Experience

Authors: Andrew Chang, Chenkai Hu, Ji Qi, Zhuojian Wei, Kexin Zhang, Viswadruth Akkaraju, David Poeppel, Dustin Freeman

Abstract: Group conversations over videoconferencing are a complex social behavior. However, the subjective moments of negative experience, where the conversation loses fluidity or enjoyment remain understudied. These moments are infrequent in naturalistic data, and thus training a supervised learning (SL) model requires costly manual data annotation. We applied semi-supervised learning (SSL) to leverage ta… ▽ More Group conversations over videoconferencing are a complex social behavior. However, the subjective moments of negative experience, where the conversation loses fluidity or enjoyment remain understudied. These moments are infrequent in naturalistic data, and thus training a supervised learning (SL) model requires costly manual data annotation. We applied semi-supervised learning (SSL) to leverage targeted labeled and unlabeled clips for training multimodal (audio, facial, text) deep features to predict non-fluid or unenjoyable moments in holdout videoconference sessions. The modality-fused co-training SSL achieved an ROC-AUC of 0.9 and an F1 score of 0.6, outperforming SL models by up to 4% with the same amount of labeled data. Remarkably, the best SSL model with just 8% labeled data matched 96% of the SL model's full-data performance. This shows an annotation-efficient framework for modeling videoconference experience. △ Less

Submitted 31 May, 2025; originally announced June 2025.

Comments: Interspeech 2025

Lightweight Transformer via Unrolling of Mixed Graph Algorithms for Traffic Forecast

Authors: Ji Qi, Tam Thuc Do, Mingxiao Liu, Zhuoshi Pan, Yuzhe Li, Gene Cheung, H. Vicky Zhao

Abstract: To forecast traffic with both spatial and temporal dimensions, we unroll a mixed-graph-based optimization algorithm into a lightweight and interpretable transformer-like neural net. Specifically, we construct two graphs: an undirected graph $\mathcal{G}^u$ capturing spatial correlations across geography, and a directed graph $\mathcal{G}^d$ capturing sequential relationships over time. We formulat… ▽ More To forecast traffic with both spatial and temporal dimensions, we unroll a mixed-graph-based optimization algorithm into a lightweight and interpretable transformer-like neural net. Specifically, we construct two graphs: an undirected graph $\mathcal{G}^u$ capturing spatial correlations across geography, and a directed graph $\mathcal{G}^d$ capturing sequential relationships over time. We formulate a prediction problem for the future samples of signal $\mathbf{x}$, assuming it is "smooth" with respect to both $\mathcal{G}^u$ and $\mathcal{G}^d$, where we design new $\ell_2$ and $\ell_1$-norm variational terms to quantify and promote signal smoothness (low-frequency reconstruction) on a directed graph. We construct an iterative algorithm based on alternating direction method of multipliers (ADMM), and unroll it into a feed-forward network for data-driven parameter learning. We insert graph learning modules for $\mathcal{G}^u$ and $\mathcal{G}^d$, which are akin to the self-attention mechanism in classical transformers. Experiments show that our unrolled networks achieve competitive traffic forecast performance as state-of-the-art prediction schemes, while reducing parameter counts drastically. Our code is available in https://github.com/SingularityUndefined/Unrolling-GSP-STForecast. △ Less

Submitted 19 May, 2025; originally announced May 2025.

Comments: 19 pages, 5 figures, 8 tables

Efficient Information Updates in Compute-First Networking via Reinforcement Learning with Joint AoI and VoI

Authors: Jianpeng Qi, Chao Liu, Chengxiang Xu, Rui Wang, Junyu Dong, Yanwei Yu

Abstract: Timely and efficient dissemination of service information is critical in compute-first networking systems, where user requests arrive dynamically and computing resources are constrained. In such systems, the access point (AP) plays a key role in forwarding user requests to a server based on its latest received service information. This paper considers a single-source, single-destination system and… ▽ More Timely and efficient dissemination of service information is critical in compute-first networking systems, where user requests arrive dynamically and computing resources are constrained. In such systems, the access point (AP) plays a key role in forwarding user requests to a server based on its latest received service information. This paper considers a single-source, single-destination system and introduces an Age-and-Value-Aware (AVA) metric that jointly captures both the timeliness and the task relevance of service information. Unlike traditional freshness-based metrics, AVA explicitly incorporates variations in server-side service capacity and AP forwarding decisions, allowing more context-aware update evaluation. Building upon AVA, we propose a reinforcement learning-based update policy that learns to selectively transmit service information updates to the AP. It aims to maximize overall task success while minimizing unnecessary communications. Extensive simulations under diverse user request patterns and varying service capacities demonstrate that AVA reduces the update frequency by over 90% on average compared to baselines, with reductions reaching 98% in certain configurations. Crucially, this reduction is achieved without compromising the accuracy of task execution or the quality of decision making. △ Less

Submitted 9 May, 2025; originally announced May 2025.

Comments: 11pages, 40 figures

STF-GCN: A Multi-Domain Graph Convolution Network Method for Automatic Modulation Recognition via Adaptive Correlation

Authors: Mingyuan Shao, Zhengqiu Fu, Dingzhao Li, Fuqing Zhang, Yilin Cai, Shaohua Hong, Lin Cao, Yuan Peng, Jie Qi

Abstract: Automatic Modulation Recognition (AMR) is an essential part of Intelligent Transportation System (ITS) dynamic spectrum allocation. However, current deep learning-based AMR (DL-AMR) methods are challenged to extract discriminative and robust features at low signal-to-noise ratios (SNRs), where the representation of modulation symbols is highly interfered by noise. Furthermore, current research on… ▽ More Automatic Modulation Recognition (AMR) is an essential part of Intelligent Transportation System (ITS) dynamic spectrum allocation. However, current deep learning-based AMR (DL-AMR) methods are challenged to extract discriminative and robust features at low signal-to-noise ratios (SNRs), where the representation of modulation symbols is highly interfered by noise. Furthermore, current research on GNN methods for AMR tasks generally suffers from issues related to graph structure construction and computational complexity. In this paper, we propose a Spatial-Temporal-Frequency Graph Convolution Network (STF-GCN) framework, with the temporal domain as the anchor point, to fuse spatial and frequency domain features embedded in the graph structure nodes. On this basis, an adaptive correlation-based adjacency matrix construction method is proposed, which significantly enhances the graph structure's capacity to aggregate local information into individual nodes. In addition, a PoolGAT layer is proposed to coarsen and compress the global key features of the graph, significantly reducing the computational complexity. The results of the experiments confirm that STF-GCN is able to achieve recognition performance far beyond the state-of-the-art DL-AMR algorithms, with overall accuracies of 64.35%, 66.04% and 70.95% on the RML2016.10a, RML2016.10b and RML22 datasets, respectively. Furthermore, the average recognition accuracies under low SNR conditions from -14dB to 0dB outperform the state-of-the-art (SOTA) models by 1.20%, 1.95% and 1.83%, respectively. △ Less

Submitted 11 April, 2025; originally announced April 2025.

NCF: Neural Correspondence Field for Medical Image Registration

Authors: Lei Zhou, Nimu Yuan, Katjana Ehrlich, Jinyi Qi

Abstract: Deformable image registration is a fundamental task in medical image processing. Traditional optimization-based methods often struggle with accuracy in dealing with complex deformation. Recently, learning-based methods have achieved good performance on public datasets, but the scarcity of medical image data makes it challenging to build a generalizable model to handle diverse real-world scenarios.… ▽ More Deformable image registration is a fundamental task in medical image processing. Traditional optimization-based methods often struggle with accuracy in dealing with complex deformation. Recently, learning-based methods have achieved good performance on public datasets, but the scarcity of medical image data makes it challenging to build a generalizable model to handle diverse real-world scenarios. To address this, we propose a training-data-free learning-based method, Neural Correspondence Field (NCF), which can learn from just one data pair. Our approach employs a compact neural network to model the correspondence field and optimize model parameters for each individual image pair. Consequently, each pair has a unique set of network weights. Notably, our model is highly efficient, utilizing only 0.06 million parameters. Evaluation results showed that the proposed method achieved superior performance on a public Lung CT dataset and outperformed a traditional method on a head and neck dataset, demonstrating both its effectiveness and efficiency. △ Less

Submitted 2 March, 2025; originally announced March 2025.

Neural Operator based Reinforcement Learning for Control of first-order PDEs with Spatially-Varying State Delay

Authors: Jiaqi Hu, Jie Qi, Jing Zhang

Abstract: Control of distributed parameter systems affected by delays is a challenging task, particularly when the delays depend on spatial variables. The idea of integrating analytical control theory with learning-based control within a unified control scheme is becoming increasingly promising and advantageous. In this paper, we address the problem of controlling an unstable first-order hyperbolic PDE with… ▽ More Control of distributed parameter systems affected by delays is a challenging task, particularly when the delays depend on spatial variables. The idea of integrating analytical control theory with learning-based control within a unified control scheme is becoming increasingly promising and advantageous. In this paper, we address the problem of controlling an unstable first-order hyperbolic PDE with spatially-varying delays by combining PDE backstepping control strategies and deep reinforcement learning (RL). To eliminate the assumption on the delay function required for the backstepping design, we propose a soft actor-critic (SAC) architecture incorporating a DeepONet to approximate the backstepping controller. The DeepONet extracts features from the backstepping controller and feeds them into the policy network. In simulations, our algorithm outperforms the baseline SAC without prior backstepping knowledge and the analytical controller. △ Less

Submitted 30 January, 2025; originally announced January 2025.

Comments: 6 Pages, 7 Figures

A wideband amplifying and filtering reconfigurable intelligent surface for wireless relay

Authors: Lijie Wu, Qun Yan Zhou, Jun Yan Dai, Siran Wang, Junwei Zhang, Zhen Jie Qi, Hanqing Yang, Ruizhe Jiang, Zheng Xing Wang, Huidong Li, Zhen Zhang, Jiang Luo, Qiang Cheng, Tie Jun Cui

Abstract: Programmable metasurfaces have garnered significant attention due to their exceptional ability to manipulate electromagnetic (EM) waves in real time, leading to the emergence of a prominent area in wireless communication, namely reconfigurable intelligent surfaces (RISs), to control the signal propagation and coverage. However, the existing RISs usually suffer from limited operating distance and b… ▽ More Programmable metasurfaces have garnered significant attention due to their exceptional ability to manipulate electromagnetic (EM) waves in real time, leading to the emergence of a prominent area in wireless communication, namely reconfigurable intelligent surfaces (RISs), to control the signal propagation and coverage. However, the existing RISs usually suffer from limited operating distance and band interference, which hinder their practical applications in wireless relay and communication systems. To overcome the limitations, we propose an amplifying and filtering RIS (AF-RIS) to enhance the in-band signal energy and filter the out-of-band signal of the incident EM waves, ensuring the miniaturization of the RIS array and enabling its anti-interference ability. In addition, each AF-RIS element is equipped with a 2-bit phase control capability, further endowing the entire array with great beamforming performance. An elaborately designed 4*8 AF-RIS array is presented by integrating the power dividing and combining networks, which substantially reduces the number of amplifiers and filters, thereby reducing the hardware costs and power consumption. Experimental results showcase the powerful capabilities of AF-RIS in beam-steering, frequency selectivity, and signal amplification. Therefore, the proposed AF-RIS holds significant promise for critical applications in wireless relay systems by offering an efficient solution to improve frequency selectivity, enhance signal coverage, and reduce hardware size. △ Less

Submitted 31 December, 2024; originally announced January 2025.

Swin-X2S: Reconstructing 3D Shape from 2D Biplanar X-ray with Swin Transformers

Authors: Kuan Liu, Zongyuan Ying, Jie Jin, Dongyan Li, Ping Huang, Wenjian Wu, Zhe Chen, Jin Qi, Yong Lu, Lianfu Deng, Bo Chen

Abstract: The conversion from 2D X-ray to 3D shape holds significant potential for improving diagnostic efficiency and safety. However, existing reconstruction methods often rely on hand-crafted features, manual intervention, and prior knowledge, resulting in unstable shape errors and additional processing costs. In this paper, we introduce Swin-X2S, an end-to-end deep learning method for directly reconstru… ▽ More The conversion from 2D X-ray to 3D shape holds significant potential for improving diagnostic efficiency and safety. However, existing reconstruction methods often rely on hand-crafted features, manual intervention, and prior knowledge, resulting in unstable shape errors and additional processing costs. In this paper, we introduce Swin-X2S, an end-to-end deep learning method for directly reconstructing 3D segmentation and labeling from 2D biplanar orthogonal X-ray images. Swin-X2S employs an encoder-decoder architecture: the encoder leverages 2D Swin Transformer for X-ray information extraction, while the decoder employs 3D convolution with cross-attention to integrate structural features from orthogonal views. A dimension-expanding module is introduced to bridge the encoder and decoder, ensuring a smooth conversion from 2D pixels to 3D voxels. We evaluate proposed method through extensive qualitative and quantitative experiments across nine publicly available datasets covering four anatomies (femur, hip, spine, and rib), with a total of 54 categories. Significant improvements over previous methods have been observed not only in the segmentation and labeling metrics but also in the clinically relevant parameters that are of primary concern in practical applications, which demonstrates the promise of Swin-X2S to provide an effective option for anatomical shape reconstruction in clinical scenarios. Code implementation is available at: \url{https://github.com/liukuan5625/Swin-X2S}. △ Less

Submitted 10 January, 2025; originally announced January 2025.

Neural Operator Feedback for a First-Order PIDE with Spatially-Varying State Delay

Authors: Jie Qi, Jiaqi Hu, Jing Zhang, Miroslav Krstic

Abstract: A transport PDE with a spatial integral and recirculation with constant delay has been a benchmark for neural operator approximations of PDE backstepping controllers. Introducing a spatially-varying delay into the model gives rise to a gain operator defined through integral equations which the operator's input -- the varying delay function -- enters in previously unencountered manners, including i… ▽ More A transport PDE with a spatial integral and recirculation with constant delay has been a benchmark for neural operator approximations of PDE backstepping controllers. Introducing a spatially-varying delay into the model gives rise to a gain operator defined through integral equations which the operator's input -- the varying delay function -- enters in previously unencountered manners, including in the limits of integration and as the inverse of the `delayED time' function. This, in turn, introduces novel mathematical challenges in estimating the operator's Lipschitz constant. The backstepping kernel function having two branches endows the feedback law with a two-branch structure, where only one of the two feedback branches depends on both of the kernel branches. For this rich feedback structure, we propose a neural operator approximation of such a two-branch feedback law and prove the approximator to be semiglobally practically stabilizing. With numerical results we illustrate the training of the neural operator and its stabilizing capability. △ Less

Submitted 14 December, 2024; v1 submitted 11 December, 2024; originally announced December 2024.

Comments: This 14 page paper contains 1 table and 20 figures

A Joint Representation Using Continuous and Discrete Features for Cardiovascular Diseases Risk Prediction on Chest CT Scans

Authors: Minfeng Xu, Chen-Chen Fan, Yan-Jie Zhou, Wenchao Guo, Pan Liu, Jing Qi, Le Lu, Hanqing Chao, Kunlun He

Abstract: Cardiovascular diseases (CVD) remain a leading health concern and contribute significantly to global mortality rates. While clinical advancements have led to a decline in CVD mortality, accurately identifying individuals who could benefit from preventive interventions remains an unsolved challenge in preventive cardiology. Current CVD risk prediction models, recommended by guidelines, are based on… ▽ More Cardiovascular diseases (CVD) remain a leading health concern and contribute significantly to global mortality rates. While clinical advancements have led to a decline in CVD mortality, accurately identifying individuals who could benefit from preventive interventions remains an unsolved challenge in preventive cardiology. Current CVD risk prediction models, recommended by guidelines, are based on limited traditional risk factors or use CT imaging to acquire quantitative biomarkers, and still have limitations in predictive accuracy and applicability. On the other hand, end-to-end trained CVD risk prediction methods leveraging deep learning on CT images often fail to provide transparent and explainable decision grounds for assisting physicians. In this work, we proposed a novel joint representation that integrates discrete quantitative biomarkers and continuous deep features extracted from chest CT scans. Our approach initiated with a deep CVD risk classification model by capturing comprehensive continuous deep learning features while jointly obtaining currently clinical-established quantitative biomarkers via segmentation models. In the feature joint representation stage, we use an instance-wise feature-gated mechanism to align the continuous and discrete features, followed by a soft instance-wise feature interaction mechanism fostering independent and effective feature interaction for the final CVD risk prediction. Our method substantially improves CVD risk predictive performance and offers individual contribution analysis of each biomarker, which is important in assisting physicians' decision-making processes. We validated our method on a public chest low-dose CT dataset and a private external chest standard-dose CT patient cohort of 17,207 CT volumes from 6,393 unique subjects, and demonstrated superior predictive performance, achieving AUCs of 0.875 and 0.843, respectively. △ Less

Submitted 15 November, 2024; v1 submitted 24 October, 2024; originally announced October 2024.

Comments: 23 pages, 9 figures

A physics-based perspective for understanding and utilizing spatial resources of wireless channels

Authors: Hui Xu, Jun Wei Wu, Zhen Jie Qi, Hao Tian Wu, Rui Wen Shao, Qiang Cheng, Jieao Zhu, Linglong Dai, Tie Jun Cui

Abstract: To satisfy the increasing demands for transmission rates of wireless communications, it is necessary to use spatial resources of electromagnetic (EM) waves. In this context, EM information theory (EIT) has become a hot topic by integrating the theoretical framework of deterministic mathematics and stochastic statistics to explore the transmission mechanisms of continuous EM waves. However, the pre… ▽ More To satisfy the increasing demands for transmission rates of wireless communications, it is necessary to use spatial resources of electromagnetic (EM) waves. In this context, EM information theory (EIT) has become a hot topic by integrating the theoretical framework of deterministic mathematics and stochastic statistics to explore the transmission mechanisms of continuous EM waves. However, the previous studies were primarily focused on frame analysis, with limited exploration of practical applications and a comprehensive understanding of its essential physical characteristics. In this paper, we present a three-dimensional (3-D) line-of-sight channel capacity formula that captures the vector EM physics and accommodates both near- and far-field scenes. Based on the rigorous mathematical equation and the physical mechanism of fast multipole expansion, a channel model is established, and the finite angular spectral bandwidth feature of scattered waves is revealed. To adapt to the feature of the channel, an optimization problem is formulated for determining the mode currents on the transmitter, aiming to obtain the optimal design of the precoder and combiner. We make comprehensive analyses to investigate the relationship among the spatial degree of freedom, noise, and transmitted power, thereby establishing a rigorous upper bound of channel capacity. A series of simulations are conducted to validate the theoretical model and numerical method. This work offers a novel perspective and methodology for understanding and leveraging EIT, and provides a theoretical foundation for the design and optimization of future wireless communications. △ Less

Submitted 8 October, 2024; originally announced October 2024.

Comments: 31pages, 8 figures

Computational Ghost Imaging with Low-Density Parity-Check Code

Authors: Shuang Liu, Yunkai Hu, Jinquan Qi, Shensheng Han, Zihuai Lin

Abstract: Ghost imaging (GI) is a high-resolution imaging technology that has been a subject of interest to many fields in the past 20 years. Most GI researchers focus on the reconstruction of signal under-sampling, nevertheless, how to use information redundancy to improve the result's belief in a complex environment has hardly been studied. Motivated by this, we propose a computational GI system based on… ▽ More Ghost imaging (GI) is a high-resolution imaging technology that has been a subject of interest to many fields in the past 20 years. Most GI researchers focus on the reconstruction of signal under-sampling, nevertheless, how to use information redundancy to improve the result's belief in a complex environment has hardly been studied. Motivated by this, we propose a computational GI system based on the low-density parity-check (LDPC) coded radiation field by exploiting the signal redundancy. The non-ideal factors generated within the imaging process can be eliminated by setting up the matching fading channel model. We have derived the analytical lower bound on the bit error rate for the proposed LDPC-coded GI system. The effectiveness and performance of the LDPC-coded GI system are further validated through numerical and experiment results. △ Less

Submitted 23 September, 2024; originally announced September 2024.

Spatio-Temporal Similarity Measure based Multi-Task Learning for Predicting Alzheimer's Disease Progression using MRI Data

Authors: Xulong Wang, Yu Zhang, Menghui Zhou, Tong Liu, Jun Qi, Po Yang

Abstract: Identifying and utilising various biomarkers for tracking Alzheimer's disease (AD) progression have received many recent attentions and enable helping clinicians make the prompt decisions. Traditional progression models focus on extracting morphological biomarkers in regions of interest (ROIs) from MRI/PET images, such as regional average cortical thickness and regional volume. They are effective… ▽ More Identifying and utilising various biomarkers for tracking Alzheimer's disease (AD) progression have received many recent attentions and enable helping clinicians make the prompt decisions. Traditional progression models focus on extracting morphological biomarkers in regions of interest (ROIs) from MRI/PET images, such as regional average cortical thickness and regional volume. They are effective but ignore the relationships between brain ROIs over time, which would lead to synergistic deterioration. For exploring the synergistic deteriorating relationship between these biomarkers, in this paper, we propose a novel spatio-temporal similarity measure based multi-task learning approach for effectively predicting AD progression and sensitively capturing the critical relationships between biomarkers. Specifically, we firstly define a temporal measure for estimating the magnitude and velocity of biomarker change over time, which indicate a changing trend(temporal). Converting this trend into the vector, we then compare this variability between biomarkers in a unified vector space(spatial). The experimental results show that compared with directly ROI based learning, our proposed method is more effective in predicting disease progression. Our method also enables performing longitudinal stability selection to identify the changing relationships between biomarkers, which play a key role in disease progression. We prove that the synergistic deteriorating biomarkers between cortical volumes or surface areas have a significant effect on the cognitive prediction. △ Less

Submitted 6 November, 2023; originally announced November 2023.

Neural Operators for PDE Backstepping Control of First-Order Hyperbolic PIDE with Recycle and Delay

Authors: Jie Qi, Jing Zhang, Miroslav Krstic

Abstract: The recently introduced DeepONet operator-learning framework for PDE control is extended from the results for basic hyperbolic and parabolic PDEs to an advanced hyperbolic class that involves delays on both the state and the system output or input. The PDE backstepping design produces gain functions that are outputs of a nonlinear operator, mapping functions on a spatial domain into functions on a… ▽ More The recently introduced DeepONet operator-learning framework for PDE control is extended from the results for basic hyperbolic and parabolic PDEs to an advanced hyperbolic class that involves delays on both the state and the system output or input. The PDE backstepping design produces gain functions that are outputs of a nonlinear operator, mapping functions on a spatial domain into functions on a spatial domain, and where this gain-generating operator's inputs are the PDE's coefficients. The operator is approximated with a DeepONet neural network to a degree of accuracy that is provably arbitrarily tight. Once we produce this approximation-theoretic result in infinite dimension, with it we establish stability in closed loop under feedback that employs approximate gains. In addition to supplying such results under full-state feedback, we also develop DeepONet-approximated observers and output-feedback laws and prove their own stabilizing properties under neural operator approximations. With numerical simulations we illustrate the theoretical results and quantify the numerical effort savings, which are of two orders of magnitude, thanks to replacing the numerical PDE solving with the DeepONet. △ Less

Submitted 14 June, 2024; v1 submitted 21 July, 2023; originally announced July 2023.

Comments: 20 pages

Journal ref: Systems & Control Letters, 2024

Robust stabilization of $2 \times 2$ first-order hyperbolic PDEs with uncertain input delay

Authors: Jing Zhang, Jie Qi

Abstract: A backstepping-based compensator design is developed for a system of $2\times2$ first-order linear hyperbolic partial differential equations (PDE) in the presence of an uncertain long input delay at boundary. We introduce a transport PDE to represent the delayed input, which leads to three coupled first-order hyperbolic PDEs. A novel backstepping transformation, composed of two Volterra transforma… ▽ More A backstepping-based compensator design is developed for a system of $2\times2$ first-order linear hyperbolic partial differential equations (PDE) in the presence of an uncertain long input delay at boundary. We introduce a transport PDE to represent the delayed input, which leads to three coupled first-order hyperbolic PDEs. A novel backstepping transformation, composed of two Volterra transformations and an affine Volterra transformation, is introduced for the predictive control design. The resulting kernel equations from the affine Volterra transformation are two coupled first-order PDEs and each with two boundary conditions, which brings challenges to the well-posedness analysis. We solve the challenge by using the method of characteristics and the successive approximation. To analyze the sensitivity of the closed-loop system to uncertain input delay, we introduce a neutral system which captures the control effect resulted from the delay uncertainty. It is proved that the proposed control is robust to small delay variations. Numerical examples illustrate the performance of the proposed compensator. △ Less

Submitted 21 July, 2023; originally announced July 2023.

Delay-Adaptive Control of First-order Hyperbolic PIDEs

Authors: Shanshan Wang, Jie Qi, Miroslav Krstic

Abstract: We develop a delay-adaptive controller for a class of first-order hyperbolic partial integro-differential equations (PIDEs) with an unknown input delay. By employing a transport PDE to represent delayed actuator states, the system is transformed into a transport partial differential equation (PDE) with unknown propagation speed cascaded with a PIDE. A parameter update law is designed using a Lyapu… ▽ More We develop a delay-adaptive controller for a class of first-order hyperbolic partial integro-differential equations (PIDEs) with an unknown input delay. By employing a transport PDE to represent delayed actuator states, the system is transformed into a transport partial differential equation (PDE) with unknown propagation speed cascaded with a PIDE. A parameter update law is designed using a Lyapunov argument and the infinite-dimensional backstepping technique to establish global stability results. Furthermore, the well-posedness of the closed-loop system is analyzed. Finally, the effectiveness of the proposed method was validated through numerical simulations △ Less

Submitted 9 July, 2023; originally announced July 2023.

Bilateral boundary control of an input delayed 2-D reaction-diffusion equation

Authors: Dandan Guan, Yanmei Chen, Jie Qi, Linglong Du

Abstract: In this paper, a delay compensation design method based on PDE backstepping is developed for a two-dimensional reaction-diffusion partial differential equation (PDE) with bilateral input delays. The PDE is defined in a rectangular domain, and the bilateral control is imposed on a pair of opposite sides of the rectangle. To represent the delayed bilateral inputs, we introduce two 2-D transport PDEs… ▽ More In this paper, a delay compensation design method based on PDE backstepping is developed for a two-dimensional reaction-diffusion partial differential equation (PDE) with bilateral input delays. The PDE is defined in a rectangular domain, and the bilateral control is imposed on a pair of opposite sides of the rectangle. To represent the delayed bilateral inputs, we introduce two 2-D transport PDEs that form a cascade system with the original PDE. A novel set of backstepping transformations is proposed for delay compensator design, including one Volterra integral transformation and two affine Volterra integral transformations. Unlike the kernel equation for 1-D PDE systems with delayed boundary input, the resulting kernel equations for the 2-D system have singular initial conditions governed by the Dirac Delta function. Consequently, the kernel solutions are written as a double trigonometric series with singularities. To address the challenge of stability analysis posed by the singularities, we prove a set of inequalities by using the Cauchy-Schwarz inequality, the 2-D Fourier series, and the Parseval's theorem. A numerical simulation illustrates the effectiveness of the proposed delay-compensation method. △ Less

Submitted 7 July, 2023; originally announced July 2023.

Comments: 11 pages, 3 figures(including 8 sub-figures)

Parameter-efficient Dysarthric Speech Recognition Using Adapter Fusion and Householder Transformation

Authors: Jinzi Qi, Hugo Van hamme

Abstract: In dysarthric speech recognition, data scarcity and the vast diversity between dysarthric speakers pose significant challenges. While finetuning has been a popular solution, it can lead to overfitting and low parameter efficiency. Adapter modules offer a better solution, with their small size and easy applicability. Additionally, Adapter Fusion can facilitate knowledge transfer from multiple learn… ▽ More In dysarthric speech recognition, data scarcity and the vast diversity between dysarthric speakers pose significant challenges. While finetuning has been a popular solution, it can lead to overfitting and low parameter efficiency. Adapter modules offer a better solution, with their small size and easy applicability. Additionally, Adapter Fusion can facilitate knowledge transfer from multiple learned adapters, but may employ more parameters. In this work, we apply Adapter Fusion for target speaker adaptation and speech recognition, achieving acceptable accuracy with significantly fewer speaker-specific trainable parameters than classical finetuning methods. We further improve the parameter efficiency of the fusion layer by reducing the size of query and key layers and using Householder transformation to reparameterize the value linear layer. Our proposed fusion layer achieves comparable recognition results to the original method with only one third of the parameters. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Comments: Accepted by Interspeech 2023

An Enhanced Res2Net with Local and Global Feature Fusion for Speaker Verification

Authors: Yafeng Chen, Siqi Zheng, Hui Wang, Luyao Cheng, Qian Chen, Jiajun Qi

Abstract: Effective fusion of multi-scale features is crucial for improving speaker verification performance. While most existing methods aggregate multi-scale features in a layer-wise manner via simple operations, such as summation or concatenation. This paper proposes a novel architecture called Enhanced Res2Net (ERes2Net), which incorporates both local and global feature fusion techniques to improve the… ▽ More Effective fusion of multi-scale features is crucial for improving speaker verification performance. While most existing methods aggregate multi-scale features in a layer-wise manner via simple operations, such as summation or concatenation. This paper proposes a novel architecture called Enhanced Res2Net (ERes2Net), which incorporates both local and global feature fusion techniques to improve the performance. The local feature fusion (LFF) fuses the features within one single residual block to extract the local signal. The global feature fusion (GFF) takes acoustic features of different scales as input to aggregate global signal. To facilitate effective feature fusion in both LFF and GFF, an attentional feature fusion module is employed in the ERes2Net architecture, replacing summation or concatenation operations. A range of experiments conducted on the VoxCeleb datasets demonstrate the superiority of the ERes2Net in speaker verification. Code has been made publicly available at https://github.com/alibaba-damo-academy/3D-Speaker. △ Less

Submitted 3 August, 2023; v1 submitted 22 May, 2023; originally announced May 2023.

Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learning

Authors: Jiaju Qi, Lei Lei, Kan Zheng, Simon X. Yang, Xuemin, Shen

Abstract: In this paper, we investigate the scheduling issue of diesel generators (DGs) in an Internet of Things (IoT)-Driven isolated microgrid (MG) by deep reinforcement learning (DRL). The renewable energy is fully exploited under the uncertainty of renewable generation and load demand. The DRL agent learns an optimal policy from history renewable and load data of previous days, where the policy can gene… ▽ More In this paper, we investigate the scheduling issue of diesel generators (DGs) in an Internet of Things (IoT)-Driven isolated microgrid (MG) by deep reinforcement learning (DRL). The renewable energy is fully exploited under the uncertainty of renewable generation and load demand. The DRL agent learns an optimal policy from history renewable and load data of previous days, where the policy can generate real-time decisions based on observations of past renewable and load data of previous hours collected by connected sensors. The goal is to reduce operating cost on the premise of ensuring supply-demand balance. In specific, a novel finite-horizon partial observable Markov decision process (POMDP) model is conceived considering the spinning reserve. In order to overcome the challenge of discrete-continuous hybrid action space due to the binary DG switching decision and continuous energy dispatch (ED) decision, a DRL algorithm, namely the hybrid action finite-horizon RDPG (HAFH-RDPG), is proposed. HAFH-RDPG seamlessly integrates two classical DRL algorithms, i.e., deep Q-network (DQN) and recurrent deterministic policy gradient (RDPG), based on a finite-horizon dynamic programming (DP) framework. Extensive experiments are performed with real-world data in an IoT-driven MG to evaluate the capability of the proposed algorithm in handling the uncertainty due to inter-hour and inter-day power fluctuation and to compare its performance with those of the benchmark algorithms. △ Less

Submitted 28 April, 2023; originally announced May 2023.

Enhancing Light Extraction of Organic Light Emitting Diodes by Deep-Groove High-index Dielectric Nanomesh Using Large-area Nanoimprint

Authors: Ji Qi, Wei Ding, Qi Zhang, Yuxuan Wang, Hao Chen, Stephen Y. Chou

Abstract: To solve the conventional conflict between maintaining good charge transport property and achieving high light extraction efficiency when using micro/nanostructure patterned substrates to extract light from organic light emitting diodes (OLEDs), we developed a novel OLED structure, termed High-index Deep-Groove Dielectric Nanomesh OLED (HDNM-OLED), fabricated by large-area nanoimprint lithography… ▽ More To solve the conventional conflict between maintaining good charge transport property and achieving high light extraction efficiency when using micro/nanostructure patterned substrates to extract light from organic light emitting diodes (OLEDs), we developed a novel OLED structure, termed High-index Deep-Groove Dielectric Nanomesh OLED (HDNM-OLED), fabricated by large-area nanoimprint lithography (NIL). The key component is a nanostructure-patterned substrate embedded with a high-index deep-groove nanomesh and capped with a low-index planarization layer. The high-index and deep-groove nanomesh efficiently releases the tapped photons to achieve significantly enhanced light extraction. And the planarization layer helps to maintain the good charge transport property of the organic active layers deposited on top of it. For a green phosphorescent OLED in our demonstration, with the HDNM-OLED structure, compared to planar conventional ITO-OLED structure, the external quantum efficiency (EQE) was enhanced by 1.85-fold from 26% to 48% and power efficiency was enhanced by 1.86-fold from 42lm/W to 79lm/W. Besides green OELDs, the HDNM-OLED structure was also shown to be able to work for red and blue-emitting OELDs and achieved enhanced light extraction efficiency (1.58-fold for red light, 1.86-fold for blue light) without further structure modification, which demonstrated the light extraction enhancement by the HDNM-OLED is broadband. △ Less

Submitted 31 January, 2023; originally announced February 2023.

Comments: arXiv admin note: text overlap with arXiv:2302.00044

Efficient Incremental Text-to-Speech on GPUs

Authors: Muyang Du, Chuan Liu, Jiaxing Qi, Junjie Lai

Abstract: Incremental text-to-speech, also known as streaming TTS, has been increasingly applied to online speech applications that require ultra-low response latency to provide an optimal user experience. However, most of the existing speech synthesis pipelines deployed on GPU are still non-incremental, which uncovers limitations in high-concurrency scenarios, especially when the pipeline is built with end… ▽ More Incremental text-to-speech, also known as streaming TTS, has been increasingly applied to online speech applications that require ultra-low response latency to provide an optimal user experience. However, most of the existing speech synthesis pipelines deployed on GPU are still non-incremental, which uncovers limitations in high-concurrency scenarios, especially when the pipeline is built with end-to-end neural network models. To address this issue, we present a highly efficient approach to perform real-time incremental TTS on GPUs with Instant Request Pooling and Module-wise Dynamic Batching. Experimental results demonstrate that the proposed method is capable of producing high-quality speech with a first-chunk latency lower than 80ms under 100 QPS on a single NVIDIA A10 GPU and significantly outperforms the non-incremental twin in both concurrency and latency. Our work reveals the effectiveness of high-performance incremental TTS on GPUs. △ Less

Submitted 5 December, 2022; v1 submitted 25 November, 2022; originally announced November 2022.

Comments: 5 pages, 4 figures

Weak-Supervised Dysarthria-invariant Features for Spoken Language Understanding using an FHVAE and Adversarial Training

Authors: Jinzi Qi, Hugo Van hamme

Abstract: The scarcity of training data and the large speaker variation in dysarthric speech lead to poor accuracy and poor speaker generalization of spoken language understanding systems for dysarthric speech. Through work on the speech features, we focus on improving the model generalization ability with limited dysarthric data. Factorized Hierarchical Variational Auto-Encoders (FHVAE) trained unsupervise… ▽ More The scarcity of training data and the large speaker variation in dysarthric speech lead to poor accuracy and poor speaker generalization of spoken language understanding systems for dysarthric speech. Through work on the speech features, we focus on improving the model generalization ability with limited dysarthric data. Factorized Hierarchical Variational Auto-Encoders (FHVAE) trained unsupervisedly have shown their advantage in disentangling content and speaker representations. Earlier work showed that the dysarthria shows in both feature vectors. Here, we add adversarial training to bridge the gap between the control and dysarthric speech data domains. We extract dysarthric and speaker invariant features using weak supervision. The extracted features are evaluated on a Spoken Language Understanding task and yield a higher accuracy on unseen speakers with more severe dysarthria compared to features from the basic FHVAE model or plain filterbanks. △ Less

Submitted 24 October, 2022; originally announced October 2022.

An Ensemble Teacher-Student Learning Approach with Poisson Sub-sampling to Differential Privacy Preserving Speech Recognition

Authors: Chao-Han Huck Yang, Jun Qi, Sabato Marco Siniscalchi, Chin-Hui Lee

Abstract: We propose an ensemble learning framework with Poisson sub-sampling to effectively train a collection of teacher models to issue some differential privacy (DP) guarantee for training data. Through boosting under DP, a student model derived from the training data suffers little model degradation from the models trained with no privacy protection. Our proposed solution leverages upon two mechanisms,… ▽ More We propose an ensemble learning framework with Poisson sub-sampling to effectively train a collection of teacher models to issue some differential privacy (DP) guarantee for training data. Through boosting under DP, a student model derived from the training data suffers little model degradation from the models trained with no privacy protection. Our proposed solution leverages upon two mechanisms, namely: (i) a privacy budget amplification via Poisson sub-sampling to train a target prediction model that requires less noise to achieve a same level of privacy budget, and (ii) a combination of the sub-sampling technique and an ensemble teacher-student learning framework that introduces DP-preserving noise at the output of the teacher models and transfers DP-preserving properties via noisy labels. Privacy-preserving student models are then trained with the noisy labels to learn the knowledge with DP-protection from the teacher model ensemble. Experimental evidences on spoken command recognition and continuous speech recognition of Mandarin speech show that our proposed framework greatly outperforms existing DP-preserving algorithms in both speech processing tasks. △ Less

Submitted 12 October, 2022; originally announced October 2022.

Comments: Accepted to ISCA, ISCSLP 2022, Singapore. 5 Pages

A CNN with Noise Inclined Module and Denoise Framework for Hyperspectral Image Classification

Authors: Zhiqiang Gong, Ping Zhong, Jiahao Qi, Panhe Hu

Abstract: Deep Neural Networks have been successfully applied in hyperspectral image classification. However, most of prior works adopt general deep architectures while ignore the intrinsic structure of the hyperspectral image, such as the physical noise generation. This would make these deep models unable to generate discriminative features and provide impressive classification performance. To leverage suc… ▽ More Deep Neural Networks have been successfully applied in hyperspectral image classification. However, most of prior works adopt general deep architectures while ignore the intrinsic structure of the hyperspectral image, such as the physical noise generation. This would make these deep models unable to generate discriminative features and provide impressive classification performance. To leverage such intrinsic information, this work develops a novel deep learning framework with the noise inclined module and denoise framework for hyperspectral image classification. First, we model the spectral signature of hyperspectral image with the physical noise model to describe the high intraclass variance of each class and great overlapping between different classes in the image. Then, a noise inclined module is developed to capture the physical noise within each object and a denoise framework is then followed to remove such noise from the object. Finally, the CNN with noise inclined module and the denoise framework is developed to obtain discriminative features and provides good classification performance of hyperspectral image. Experiments are conducted over two commonly used real-world datasets and the experimental results show the effectiveness of the proposed method. The implementation of the proposed method and other compared methods could be accessed at https://github.com/shendu-sw/noise-physical-framework. △ Less

Submitted 24 May, 2022; originally announced May 2022.

Journal ref: IET Image Processing, 2022

Model Predictive Manipulation of Compliant Objects with Multi-Objective Optimizer and Adversarial Network for Occlusion Compensation

Authors: Jiaming Qi, Dongyu Li, Yufeng Gao, Peng Zhou, David Navarro-Alarcon

Abstract: The robotic manipulation of compliant objects is currently one of the most active problems in robotics due to its potential to automate many important applications. Despite the progress achieved by the robotics community in recent years, the 3D shaping of these types of materials remains an open research problem. In this paper, we propose a new vision-based controller to automatically regulate the… ▽ More The robotic manipulation of compliant objects is currently one of the most active problems in robotics due to its potential to automate many important applications. Despite the progress achieved by the robotics community in recent years, the 3D shaping of these types of materials remains an open research problem. In this paper, we propose a new vision-based controller to automatically regulate the shape of compliant objects with robotic arms. Our method uses an efficient online surface/curve fitting algorithm that quantifies the object's geometry with a compact vector of features; This feedback-like vector enables to establish an explicit shape servo-loop. To coordinate the motion of the robot with the computed shape features, we propose a receding-time estimator that approximates the system's sensorimotor model while satisfying various performance criteria. A deep adversarial network is developed to robustly compensate for visual occlusions in the camera's field of view, which enables to guide the shaping task even with partial observations of the object. Model predictive control is utilized to compute the robot's shaping motions subject to workspace and saturation constraints. A detailed experimental study is presented to validate the effectiveness of the proposed control framework. △ Less

Submitted 20 May, 2022; originally announced May 2022.

Breast Cancer Molecular Subtypes Prediction on Pathological Images with Discriminative Patch Selecting and Multi-Instance Learning

Authors: Hong Liu, Wen-Dong Xu, Zi-Hao Shang, Xiang-Dong Wang, Hai-Yan Zhou, Ke-Wen Ma, Huan Zhou, Jia-Lin Qi, Jia-Rui Jiang, Li-Lan Tan, Hui-Min Zeng, Hui-Juan Cai, Kuan-Song Wang, Yue-Liang Qian

Abstract: Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient's paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable sampling error is risky due to tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from con… ▽ More Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient's paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable sampling error is risky due to tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&E pathological whole slide images (WSI) using AI method is useful and critical to assist pathologists pre-screen proper paraffin block for IHC. It's a challenging task since only WSI level labels of molecular subtypes can be obtained from IHC. Gigapixel WSIs are divided into a huge number of patches to be computationally feasible for deep learning. While with coarse slide-level labels, patch-based methods may suffer from abundant noise patches, such as folds, overstained regions, or non-tumor tissues. A weakly supervised learning framework based on discriminative patch selecting and multi-instance learning was proposed for breast cancer molecular subtype prediction from H&E WSIs. Firstly, co-teaching strategy was adopted to learn molecular subtype representations and filter out noise patches. Then, a balanced sampling strategy was used to handle the imbalance in subtypes in the dataset. In addition, a noise patch filtering algorithm that used local outlier factor based on cluster centers was proposed to further select discriminative patches. Finally, a loss function integrating patch with slide constraint information was used to finetune MIL framework on obtained discriminative patches and further improve the performance of molecular subtyping. The experimental results confirmed the effectiveness of the proposed method and our models outperformed even senior pathologists, with potential to assist pathologists to pre-screen paraffin blocks for IHC in clinic. △ Less

Submitted 15 March, 2022; originally announced March 2022.

Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing

Authors: Jun Qi, Chao-Han Huck Yang, Pin-Yu Chen, Javier Tejedor

Abstract: This work focuses on designing low complexity hybrid tensor networks by considering trade-offs between the model complexity and practical performance. Firstly, we exploit a low-rank tensor-train deep neural network (TT-DNN) to build an end-to-end deep learning pipeline, namely LR-TT-DNN. Secondly, a hybrid model combining LR-TT-DNN with a convolutional neural network (CNN), which is denoted as CNN… ▽ More This work focuses on designing low complexity hybrid tensor networks by considering trade-offs between the model complexity and practical performance. Firstly, we exploit a low-rank tensor-train deep neural network (TT-DNN) to build an end-to-end deep learning pipeline, namely LR-TT-DNN. Secondly, a hybrid model combining LR-TT-DNN with a convolutional neural network (CNN), which is denoted as CNN+(LR-TT-DNN), is set up to boost the performance. Instead of randomly assigning large TT-ranks for TT-DNN, we leverage Riemannian gradient descent to determine a TT-DNN associated with small TT-ranks. Furthermore, CNN+(LR-TT-DNN) consists of convolutional layers at the bottom for feature extraction and several TT layers at the top to solve regression and classification problems. We separately assess the LR-TT-DNN and CNN+(LR-TT-DNN) models on speech enhancement and spoken command recognition tasks. Our empirical evidence demonstrates that the LR-TT-DNN and CNN+(LR-TT-DNN) models with fewer model parameters can outperform the TT-DNN and CNN+(TT-DNN) counterparts. △ Less

Submitted 11 March, 2022; originally announced March 2022.

Comments: 10 pages, 10 Figures

When BERT Meets Quantum Temporal Convolution Learning for Text Classification in Heterogeneous Computing

Authors: Chao-Han Huck Yang, Jun Qi, Samuel Yen-Chi Chen, Yu Tsao, Pin-Yu Chen

Abstract: The rapid development of quantum computing has demonstrated many unique characteristics of quantum advantages, such as richer feature representation and more secured protection on model parameters. This work proposes a vertical federated learning architecture based on variational quantum circuits to demonstrate the competitive performance of a quantum-enhanced pre-trained BERT model for text class… ▽ More The rapid development of quantum computing has demonstrated many unique characteristics of quantum advantages, such as richer feature representation and more secured protection on model parameters. This work proposes a vertical federated learning architecture based on variational quantum circuits to demonstrate the competitive performance of a quantum-enhanced pre-trained BERT model for text classification. In particular, our proposed hybrid classical-quantum model consists of a novel random quantum temporal convolution (QTC) learning framework replacing some layers in the BERT-based decoder. Our experiments on intent classification show that our proposed BERT-QTC model attains competitive experimental results in the Snips and ATIS spoken language datasets. Particularly, the BERT-QTC boosts the performance of the existing quantum circuit-based language model in two text classification datasets by 1.57% and 1.52% relative improvements. Furthermore, BERT-QTC can be feasibly deployed on both existing commercial-accessible quantum computation hardware and CPU-based interface for ensuring data isolation. △ Less

Submitted 17 February, 2022; originally announced March 2022.

Comments: Accepted to ICASSP 2022

STG-GAN: A spatiotemporal graph generative adversarial networks for short-term passenger flow prediction in urban rail transit systems

Authors: Jinlei Zhang, Hua Li, Lixing Yang, Guangyin Jin, Jianguo Qi, Ziyou Gao

Abstract: Short-term passenger flow prediction is an important but challenging task for better managing urban rail transit (URT) systems. Some emerging deep learning models provide good insights to improve short-term prediction accuracy. However, there exist many complex spatiotemporal dependencies in URT systems. Most previous methods only consider the absolute error between ground truth and predictions as… ▽ More Short-term passenger flow prediction is an important but challenging task for better managing urban rail transit (URT) systems. Some emerging deep learning models provide good insights to improve short-term prediction accuracy. However, there exist many complex spatiotemporal dependencies in URT systems. Most previous methods only consider the absolute error between ground truth and predictions as the optimization objective, which fails to account for spatial and temporal constraints on the predictions. Furthermore, a large number of existing prediction models introduce complex neural network layers to improve accuracy while ignoring their training efficiency and memory occupancy, decreasing the chances to be applied to the real world. To overcome these limitations, we propose a novel deep learning-based spatiotemporal graph generative adversarial network (STG-GAN) model with higher prediction accuracy, higher efficiency, and lower memory occupancy to predict short-term passenger flows of the URT network. Our model consists of two major parts, which are optimized in an adversarial learning manner: (1) a generator network including gated temporal conventional networks (TCN) and weight sharing graph convolution networks (GCN) to capture structural spatiotemporal dependencies and generate predictions with a relatively small computational burden; (2) a discriminator network including a spatial discriminator and a temporal discriminator to enhance the spatial and temporal constraints of the predictions. The STG-GAN is evaluated on two large-scale real-world datasets from Beijing Subway. A comparison with those of several state-of-the-art models illustrates its superiority and robustness. This study can provide critical experience in conducting short-term passenger flow predictions, especially from the perspective of real-world applications. △ Less

Submitted 16 August, 2023; v1 submitted 10 February, 2022; originally announced February 2022.

Comments: There are some errors that might mislead readers for this version. There is no new version right now

ACM Class: E.0

Exploiting Hybrid Models of Tensor-Train Networks for Spoken Command Recognition

Authors: Jun Qi, Javier Tejedor

Abstract: This work aims to design a low complexity spoken command recognition (SCR) system by considering different trade-offs between the number of model parameters and classification accuracy. More specifically, we exploit a deep hybrid architecture of a tensor-train (TT) network to build an end-to-end SRC pipeline. Our command recognition system, namely CNN+(TT-DNN), is composed of convolutional layers… ▽ More This work aims to design a low complexity spoken command recognition (SCR) system by considering different trade-offs between the number of model parameters and classification accuracy. More specifically, we exploit a deep hybrid architecture of a tensor-train (TT) network to build an end-to-end SRC pipeline. Our command recognition system, namely CNN+(TT-DNN), is composed of convolutional layers at the bottom for spectral feature extraction and TT layers at the top for command classification. Compared with a traditional end-to-end CNN baseline for SCR, our proposed CNN+(TT-DNN) model replaces fully connected (FC) layers with TT ones and it can substantially reduce the number of model parameters while maintaining the baseline performance of the CNN model. We initialize the CNN+(TT-DNN) model in a randomized manner or based on a well-trained CNN+DNN, and assess the CNN+(TT-DNN) models on the Google Speech Command Dataset. Our experimental results show that the proposed CNN+(TT-DNN) model attains a competitive accuracy of 96.31% with 4 times fewer model parameters than the CNN model. Furthermore, the CNN+(TT-DNN) model can obtain a 97.2% accuracy when the number of parameters is increased. △ Less

Submitted 11 January, 2022; originally announced January 2022.

Comments: Accepted in Proc. ICASSP 2022

Neural KEM: A Kernel Method with Deep Coefficient Prior for PET Image Reconstruction

Authors: Siqi Li, Kuang Gong, Ramsey D. Badawi, Edward J. Kim, Jinyi Qi, Guobao Wang

Abstract: Image reconstruction of low-count positron emission tomography (PET) data is challenging. Kernel methods address the challenge by incorporating image prior information in the forward model of iterative PET image reconstruction. The kernelized expectation-maximization (KEM) algorithm has been developed and demonstrated to be effective and easy to implement. A common approach for a further improveme… ▽ More Image reconstruction of low-count positron emission tomography (PET) data is challenging. Kernel methods address the challenge by incorporating image prior information in the forward model of iterative PET image reconstruction. The kernelized expectation-maximization (KEM) algorithm has been developed and demonstrated to be effective and easy to implement. A common approach for a further improvement of the kernel method would be adding an explicit regularization, which however leads to a complex optimization problem. In this paper, we propose an implicit regularization for the kernel method by using a deep coefficient prior, which represents the kernel coefficient image in the PET forward model using a convolutional neural-network. To solve the maximum-likelihood neural network-based reconstruction problem, we apply the principle of optimization transfer to derive a neural KEM algorithm. Each iteration of the algorithm consists of two separate steps: a KEM step for image update from the projection data and a deep-learning step in the image domain for updating the kernel coefficient image using the neural network. This optimization algorithm is guaranteed to monotonically increase the data likelihood. The results from computer simulations and real patient data have demonstrated that the neural KEM can outperform existing KEM and deep image prior methods. △ Less

Submitted 24 October, 2022; v1 submitted 4 January, 2022; originally announced January 2022.

Comments: arXiv admin note: text overlap with arXiv:2110.01174

A Deep-Learning Framework for Improving COVID-19 CT Image Quality and Diagnostic Accuracy

Authors: Garvit Goel, Jingyuan Qi, Wu-chun Feng, Guohua Cao

Abstract: We present a deep-learning based computing framework for fast-and-accurate CT (DL-FACT) testing of COVID-19. Our CT-based DL framework was developed to improve the testing speed and accuracy of COVID-19 (plus its variants) via a DL-based approach for CT image enhancement and classification. The image enhancement network is adapted from DDnet, short for DenseNet and Deconvolution based network. To… ▽ More We present a deep-learning based computing framework for fast-and-accurate CT (DL-FACT) testing of COVID-19. Our CT-based DL framework was developed to improve the testing speed and accuracy of COVID-19 (plus its variants) via a DL-based approach for CT image enhancement and classification. The image enhancement network is adapted from DDnet, short for DenseNet and Deconvolution based network. To demonstrate its speed and accuracy, we evaluated DL-FACT across several sources of COVID-19 CT images. Our results show that DL-FACT can significantly shorten the turnaround time from days to minutes and improve the COVID-19 testing accuracy up to 91%. DL-FACT could be used as a software tool for medical professionals in diagnosing and monitoring COVID-19. △ Less

Submitted 16 December, 2021; originally announced December 2021.

Comments: 10 pages

LMR-CBT: Learning Modality-fused Representations with CB-Transformer for Multimodal Emotion Recognition from Unaligned Multimodal Sequences

Authors: Ziwang Fu, Feng Liu, Hanyang Wang, Siyuan Shen, Jiahao Zhang, Jiayin Qi, Xiangling Fu, Aimin Zhou

Abstract: Learning modality-fused representations and processing unaligned multimodal sequences are meaningful and challenging in multimodal emotion recognition. Existing approaches use directional pairwise attention or a message hub to fuse language, visual, and audio modalities. However, those approaches introduce information redundancy when fusing features and are inefficient without considering the comp… ▽ More Learning modality-fused representations and processing unaligned multimodal sequences are meaningful and challenging in multimodal emotion recognition. Existing approaches use directional pairwise attention or a message hub to fuse language, visual, and audio modalities. However, those approaches introduce information redundancy when fusing features and are inefficient without considering the complementarity of modalities. In this paper, we propose an efficient neural network to learn modality-fused representations with CB-Transformer (LMR-CBT) for multimodal emotion recognition from unaligned multimodal sequences. Specifically, we first perform feature extraction for the three modalities respectively to obtain the local structure of the sequences. Then, we design a novel transformer with cross-modal blocks (CB-Transformer) that enables complementary learning of different modalities, mainly divided into local temporal learning,cross-modal feature fusion and global self-attention representations. In addition, we splice the fused features with the original features to classify the emotions of the sequences. Finally, we conduct word-aligned and unaligned experiments on three challenging datasets, IEMOCAP, CMU-MOSI, and CMU-MOSEI. The experimental results show the superiority and efficiency of our proposed method in both settings. Compared with the mainstream methods, our approach reaches the state-of-the-art with a minimum number of parameters. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Comments: 9 pages ,Figure 2, Table 5

Delay-Compensated Distributed PDE Control of Traffic with Connected/Automated Vehicles

Authors: Jie Qi, Shurong Mo, Miroslav Krstic

Abstract: We develop an input delay-compensating design for stabilization of an Aw-Rascle-Zhang (ARZ) traffic model in congested regime which is governed by a $2\times 2$ first-order hyperbolic nonlinear PDE. The traffic flow consists of both adaptive cruise control-equipped (ACC-equipped) and manually-driven vehicles. The control input is the time gap of ACC-equipped and connected vehicles, which is subjec… ▽ More We develop an input delay-compensating design for stabilization of an Aw-Rascle-Zhang (ARZ) traffic model in congested regime which is governed by a $2\times 2$ first-order hyperbolic nonlinear PDE. The traffic flow consists of both adaptive cruise control-equipped (ACC-equipped) and manually-driven vehicles. The control input is the time gap of ACC-equipped and connected vehicles, which is subject to delays resulting from communication lag. For the linearized system, a novel three-branch bakcstepping transformation with explicit kernel functions is introduced to compensate the input delay. The transformation is proved æto be bounded, continuous and invertible, with explicit inverse transformation derived. Based on the transformation, we obtain the explicit predictor-feedback controller. We prove exponential stability of the closed-loop system with the delay compensator in $L_2$ norm. The performance improvement of the closed-loop system under the proposed controller is illustrated in simulation. △ Less

Submitted 2 September, 2022; v1 submitted 19 July, 2021; originally announced July 2021.

Direct Reconstruction of Linear Parametric Images from Dynamic PET Using Nonlocal Deep Image Prior

Authors: Kuang Gong, Ciprian Catana, Jinyi Qi, Quanzheng Li

Abstract: Direct reconstruction methods have been developed to estimate parametric images directly from the measured PET sinograms by combining the PET imaging model and tracer kinetics in an integrated framework. Due to limited counts received, signal-to-noise-ratio (SNR) and resolution of parametric images produced by direct reconstruction frameworks are still limited. Recently supervised deep learning me… ▽ More Direct reconstruction methods have been developed to estimate parametric images directly from the measured PET sinograms by combining the PET imaging model and tracer kinetics in an integrated framework. Due to limited counts received, signal-to-noise-ratio (SNR) and resolution of parametric images produced by direct reconstruction frameworks are still limited. Recently supervised deep learning methods have been successfully applied to medical imaging denoising/reconstruction when large number of high-quality training labels are available. For static PET imaging, high-quality training labels can be acquired by extending the scanning time. However, this is not feasible for dynamic PET imaging, where the scanning time is already long enough. In this work, we proposed an unsupervised deep learning framework for direct parametric reconstruction from dynamic PET, which was tested on the Patlak model and the relative equilibrium Logan model. The patient's anatomical prior image, which is readily available from PET/CT or PET/MR scans, was supplied as the network input to provide a manifold constraint, and also utilized to construct a kernel layer to perform non-local feature denoising. The linear kinetic model was embedded in the network structure as a 1x1 convolution layer. The training objective function was based on the PET statistical model. Evaluations based on dynamic datasets of 18F-FDG and 11C-PiB tracers show that the proposed framework can outperform the traditional and the kernel method-based direct reconstruction methods. △ Less

Submitted 18 June, 2021; originally announced June 2021.

Comments: 10 pages, 10 figures

Speech Disorder Classification Using Extended Factorized Hierarchical Variational Auto-encoders

Authors: Jinzi Qi, Hugo Van hamme

Abstract: Objective speech disorder classification for speakers with communication difficulty is desirable for diagnosis and administering therapy. With the current state of speech technology, it is evident to propose neural networks for this application. But neural network model training is hampered by a lack of labeled disordered speech data. In this research, we apply an extended version of Factorized Hi… ▽ More Objective speech disorder classification for speakers with communication difficulty is desirable for diagnosis and administering therapy. With the current state of speech technology, it is evident to propose neural networks for this application. But neural network model training is hampered by a lack of labeled disordered speech data. In this research, we apply an extended version of Factorized Hierarchical Variational Auto-encoders (FHVAE) for representation learning on disordered speech. The FHVAE model extracts both content-related and sequence-related latent variables from speech data, and we utilize the extracted variables to explore how disorder type information is represented in the latent variables. For better classification performance, the latent variables are aggregated at the word and sentence level. We show that an extension of the FHVAE model succeeds in the better disentanglement of the content-related and sequence-related related representations, but both representations are still required for best results on disorder type classification. △ Less

Submitted 14 June, 2021; originally announced June 2021.

Comments: 5 pages, 2 figures, submitted to INTERSPEECH2021

Contour Moments Based Manipulation of Composite Rigid-Deformable Objects with Finite Time Model Estimation and Shape/Position Control

Authors: Jiaming Qi, Guangfu Ma, Jihong Zhu, Peng Zhou, Yueyong Lyu, Haibo Zhang, David Navarro-Alarcon

Abstract: The robotic manipulation of composite rigid-deformable objects (i.e. those with mixed non-homogeneous stiffness properties) is a challenging problem with clear practical applications that, despite the recent progress in the field, it has not been sufficiently studied in the literature. To deal with this issue, in this paper we propose a new visual servoing method that has the capability to manipul… ▽ More The robotic manipulation of composite rigid-deformable objects (i.e. those with mixed non-homogeneous stiffness properties) is a challenging problem with clear practical applications that, despite the recent progress in the field, it has not been sufficiently studied in the literature. To deal with this issue, in this paper we propose a new visual servoing method that has the capability to manipulate this broad class of objects (which varies from soft to rigid) with the same adaptive strategy. To quantify the object's infinite-dimensional configuration, our new approach computes a compact feedback vector of 2D contour moments features. A sliding mode control scheme is then designed to simultaneously ensure the finite-time convergence of both the feedback shape error and the model estimation error. The stability of the proposed framework (including the boundedness of all the signals) is rigorously proved with Lyapunov theory. Detailed simulations and experiments are presented to validate the effectiveness of the proposed approach. To the best of the author's knowledge, this is the first time that contour moments along with finite-time control have been used to solve this difficult manipulation problem. △ Less

Submitted 4 June, 2021; originally announced June 2021.

Bidirectional Multiscale Feature Aggregation for Speaker Verification

Authors: Jiajun Qi, Wu Guo, Bin Gu

Abstract: In this paper, we propose a novel bidirectional multiscale feature aggregation (BMFA) network with attentional fusion modules for text-independent speaker verification. The feature maps from different stages of the backbone network are iteratively combined and refined in both a bottom-up and top-down manner. Furthermore, instead of simple concatenation or element-wise addition of feature maps from… ▽ More In this paper, we propose a novel bidirectional multiscale feature aggregation (BMFA) network with attentional fusion modules for text-independent speaker verification. The feature maps from different stages of the backbone network are iteratively combined and refined in both a bottom-up and top-down manner. Furthermore, instead of simple concatenation or element-wise addition of feature maps from different stages, an attentional fusion module is designed to compute the fusion weights. Experiments are conducted on the NIST SRE16 and VoxCeleb1 datasets. The experimental results demonstrate the effectiveness of the bidirectional aggregation strategy and show that the proposed attentional fusion module can further improve the performance. △ Less

Submitted 31 March, 2021; originally announced April 2021.

Generator Parameter Estimation by Q-Learning Based on PMU Measurements

Authors: Seyyed Rashid Khazeiynasab, Junjian Qi, Issa Batarseh

Abstract: In this paper, a novel Q-learning based approach is proposed for estimating the parameters of synchronous generators using PMU measurements. Event playback is used to generate model outputs under different parameters for training the agent in Q-learning. We assume that the exact values of some parameters in the model are not known by the agent in Q-learning. Then, an optimal history-dependent poli… ▽ More In this paper, a novel Q-learning based approach is proposed for estimating the parameters of synchronous generators using PMU measurements. Event playback is used to generate model outputs under different parameters for training the agent in Q-learning. We assume that the exact values of some parameters in the model are not known by the agent in Q-learning. Then, an optimal history-dependent policy for the exploration-exploitation trade-off is planned. With given prior knowledge, the parameter vector can be viewed as states with a specific reward, which is a function of the fitting error compared with the measurements. The agent takes an action (either increasing or decreasing the parameter) and the estimated parameter will move to a new state. Based on the reward function, the optimal action policy will move the parameter set to a state with the highest reward. If multiple events are available, they will be used sequentially so that the updated $\mathbfcal{Q}$-value can be utilized to improve the computational efficiency. The effectiveness of the proposed approach is validated by estimating the parameters of the dynamic model of a synchronous generator. △ Less

Submitted 1 December, 2020; originally announced December 2020.

Decentralizing Feature Extraction with Quantum Convolutional Neural Network for Automatic Speech Recognition

Authors: Chao-Han Huck Yang, Jun Qi, Samuel Yen-Chi Chen, Pin-Yu Chen, Sabato Marco Siniscalchi, Xiaoli Ma, Chin-Hui Lee

Abstract: We propose a novel decentralized feature extraction approach in federated learning to address privacy-preservation issues for speech recognition. It is built upon a quantum convolutional neural network (QCNN) composed of a quantum circuit encoder for feature extraction, and a recurrent neural network (RNN) based end-to-end acoustic model (AM). To enhance model parameter protection in a decentraliz… ▽ More We propose a novel decentralized feature extraction approach in federated learning to address privacy-preservation issues for speech recognition. It is built upon a quantum convolutional neural network (QCNN) composed of a quantum circuit encoder for feature extraction, and a recurrent neural network (RNN) based end-to-end acoustic model (AM). To enhance model parameter protection in a decentralized architecture, an input speech is first up-streamed to a quantum computing server to extract Mel-spectrogram, and the corresponding convolutional features are encoded using a quantum circuit algorithm with random parameters. The encoded features are then down-streamed to the local RNN model for the final recognition. The proposed decentralized framework takes advantage of the quantum learning progress to secure models and to avoid privacy leakage attacks. Testing on the Google Speech Commands Dataset, the proposed QCNN encoder attains a competitive accuracy of 95.12% in a decentralized model, which is better than the previous architectures using centralized RNN models with convolutional features. We also conduct an in-depth study of different quantum circuit encoder architectures to provide insights into designing QCNN-based feature extractors. Neural saliency analyses demonstrate a correlation between the proposed QCNN features, class activation maps, and input spectrograms. We provide an implementation for future studies. △ Less

Submitted 12 February, 2021; v1 submitted 25 October, 2020; originally announced October 2020.

Comments: Accepted to IEEE ICASSP 2021. Code is available: https://github.com/huckiyang/QuantumSpeech-QCNN

Journal ref: 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Multi-task Metric Learning for Text-independent Speaker Verification

Authors: Yafeng Chen, Wu Guo, Jingjing Shi, Jiajun Qi, Tan Liu

Abstract: In this work, we introduce metric learning (ML) to enhance the deep embedding learning for text-independent speaker verification (SV). Specifically, the deep speaker embedding network is trained with conventional cross entropy loss and auxiliary pair-based ML loss function. For the auxiliary ML task, training samples of a mini-batch are first arranged into pairs, then positive and negative pairs a… ▽ More In this work, we introduce metric learning (ML) to enhance the deep embedding learning for text-independent speaker verification (SV). Specifically, the deep speaker embedding network is trained with conventional cross entropy loss and auxiliary pair-based ML loss function. For the auxiliary ML task, training samples of a mini-batch are first arranged into pairs, then positive and negative pairs are selected and weighted through their own and relative similarities, and finally the auxiliary ML loss is calculated by the similarity of the selected pairs. To evaluate the proposed method, we conduct experiments on the Speaker in the Wild (SITW) dataset. The results demonstrate the effectiveness of the proposed method. △ Less

Submitted 22 March, 2023; v1 submitted 21 October, 2020; originally announced October 2020.

Comments: Not a particularly high-quality work, so we request withdrawal

Multi-Objective PMU Allocation for Resilient Power System Monitoring

Authors: Hamed Haggi, Wei Sun, Junjian Qi

Abstract: Phasor measurement units (PMUs) enable better system monitoring and security enhancement in smart grids. In order to enhance power system resilience against outages and blackouts caused by extreme weather events or man-made attacks, it remains a major challenge to determine the optimal number and location of PMUs. In this paper, a multi-objective resilient PMU placement (MORPP) problem is formulat… ▽ More Phasor measurement units (PMUs) enable better system monitoring and security enhancement in smart grids. In order to enhance power system resilience against outages and blackouts caused by extreme weather events or man-made attacks, it remains a major challenge to determine the optimal number and location of PMUs. In this paper, a multi-objective resilient PMU placement (MORPP) problem is formulated, and solved by a modified Teaching-Learning-Based Optimization (MO-TLBO) algorithm. Three objectives are considered in the MORPP problem, minimizing the number of PMUs, maximizing the system observability, and minimizing the voltage stability index. The effectiveness of the proposed method is validated through testing on IEEE 14-bus, 30-bus, and 118-bus test systems. The advantage of the MO-TLBO-based MORPP is demonstrated through the comparison with other methods in the literature, in terms of iteration number, optimality and time of convergence. △ Less

Submitted 15 October, 2020; originally announced October 2020.

Comments: IEEE PES General Meeting 2020

Exploring Universal Speech Attributes for Speaker Verification with an Improved Cross-stitch Network

Authors: Jiajun Qi, Wu Guo, Jingjing Shi, Yafeng Chen, Tan Liu

Abstract: The universal speech attributes for x-vector based speaker verification (SV) are addressed in this paper. The manner and place of articulation form the fundamental speech attribute unit (SAU), and then new speech attribute (NSA) units for acoustic modeling are generated by tied tri-SAU states. An improved cross-stitch network is adopted as a multitask learning (MTL) framework for integrating these… ▽ More The universal speech attributes for x-vector based speaker verification (SV) are addressed in this paper. The manner and place of articulation form the fundamental speech attribute unit (SAU), and then new speech attribute (NSA) units for acoustic modeling are generated by tied tri-SAU states. An improved cross-stitch network is adopted as a multitask learning (MTL) framework for integrating these universal speech attributes into the x-vector network training process. Experiments are conducted on common condition 5 (CC5) of the core-core and the 10 s-10 s tests of the NIST SRE10 evaluation set, and the proposed algorithm can achieve consistent improvements over the baseline x-vector on both these tasks. △ Less

Submitted 31 May, 2023; v1 submitted 13 October, 2020; originally announced October 2020.

Comments: Not a particularly high-quality work, so we request withdrawal

Resilience Analysis and Cascading FailureModeling of Power Systems under Extreme Temperatures

Authors: Seyyed Rashid Khazeiynasab, Junjian Qi

Abstract: In this paper, we propose an AC power flow based cascading failure model that explicitly considers external weather conditions, extreme temperatures in particular, and evaluates the impact of extreme temperature on the initiation and propagation of cascading blackouts. Specifically, load and dynamic line rating changes are modeled due to temperature disturbance, the probabilities for transmission… ▽ More In this paper, we propose an AC power flow based cascading failure model that explicitly considers external weather conditions, extreme temperatures in particular, and evaluates the impact of extreme temperature on the initiation and propagation of cascading blackouts. Specifically, load and dynamic line rating changes are modeled due to temperature disturbance, the probabilities for transmission line and generator outages are evaluated, and the timing for each type of events is carefully calculated to decide the actual event sequence. It should be emphasized that the correlated events, in the advent of external temperature changes, could together contribute to voltage instability. Besides, we model undervoltage load shedding and operator re-dispatch as control strategies for preventing the propagation of cascading failures. The effectiveness of the proposed model is verified by simulation results on the RTS-96 3-area system and it is found that temperature disturbances can lead to correlated load change and line/generator tripping, which together will greatly increase the risk of cascading and voltage instability. Critical temperature change, critical area with temperature disturbance, identification of most vulnerable buses, and comparison of different control strategies are also carefully investigated. △ Less

Submitted 29 September, 2020; originally announced September 2020.

Variational Inference-Based Dropout in Recurrent Neural Networks for Slot Filling in Spoken Language Understanding

Authors: Jun Qi, Xu Liu, Javier Tejedor

Abstract: This paper proposes to generalize the variational recurrent neural network (RNN) with variational inference (VI)-based dropout regularization employed for the long short-term memory (LSTM) cells to more advanced RNN architectures like gated recurrent unit (GRU) and bi-directional LSTM/GRU. The new variational RNNs are employed for slot filling, which is an intriguing but challenging task in spoken… ▽ More This paper proposes to generalize the variational recurrent neural network (RNN) with variational inference (VI)-based dropout regularization employed for the long short-term memory (LSTM) cells to more advanced RNN architectures like gated recurrent unit (GRU) and bi-directional LSTM/GRU. The new variational RNNs are employed for slot filling, which is an intriguing but challenging task in spoken language understanding. The experiments on the ATIS dataset suggest that the variational RNNs with the VI-based dropout regularization can significantly improve the naive dropout regularization RNNs-based baseline systems in terms of F-measure. Particularly, the variational RNN with bi-directional LSTM/GRU obtains the best F-measure score. △ Less

Submitted 23 August, 2020; originally announced September 2020.

Comments: conference paper, 5 pages

On Mean Absolute Error for Deep Neural Network Based Vector-to-Vector Regression

Authors: Jun Qi, Jun Du, Sabato Marco Siniscalchi, Xiaoli Ma, Chin-Hui Lee

Abstract: In this paper, we exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression. The goal of this work is two-fold: (i) presenting performance bounds of MAE, and (ii) demonstrating new properties of MAE that make it more appropriate than mean squared error (MSE) as a loss function for DNN based vector-to-vector regression… ▽ More In this paper, we exploit the properties of mean absolute error (MAE) as a loss function for the deep neural network (DNN) based vector-to-vector regression. The goal of this work is two-fold: (i) presenting performance bounds of MAE, and (ii) demonstrating new properties of MAE that make it more appropriate than mean squared error (MSE) as a loss function for DNN based vector-to-vector regression. First, we show that a generalized upper-bound for DNN-based vector- to-vector regression can be ensured by leveraging the known Lipschitz continuity property of MAE. Next, we derive a new generalized upper bound in the presence of additive noise. Finally, in contrast to conventional MSE commonly adopted to approximate Gaussian errors for regression, we show that MAE can be interpreted as an error modeled by Laplacian distribution. Speech enhancement experiments are conducted to corroborate our proposed theorems and validate the performance advantages of MAE over MSE for DNN based regression. △ Less

Submitted 12 August, 2020; originally announced August 2020.

Journal ref: IEEE Signal Processing Letters, 2020

Adaptive Shape Servoing of Elastic Rods using Parameterized Regression Features and Auto-Tuning Motion Controls

Authors: Jiaming Qi, Guangtao Ran, Bohui Wang, Jian Liu, Wanyu Ma, Peng Zhou, David Navarro-Alarcon

Abstract: The robotic manipulation of deformable linear objects has shown great potential in a wide range of real-world applications. However, it presents many challenges due to the objects' complex nonlinearity and high-dimensional configuration. In this paper, we propose a new shape servoing framework to automatically manipulate elastic rods through visual feedback. Our new method uses parameterized regre… ▽ More The robotic manipulation of deformable linear objects has shown great potential in a wide range of real-world applications. However, it presents many challenges due to the objects' complex nonlinearity and high-dimensional configuration. In this paper, we propose a new shape servoing framework to automatically manipulate elastic rods through visual feedback. Our new method uses parameterized regression features to compute a compact (low-dimensional) feature vector that quantifies the object's shape, thus, enabling to establish an explicit shape servo-loop. To automatically deform the rod into a desired shape, the proposed adaptive controller iteratively estimates the differential transformation between the robot's motion and the relative shape changes; This valuable capability allows to effectively manipulate objects with unknown mechanical models. An auto-tuning algorithm is introduced to adjust the robot's shaping motions in real-time based on optimal performance criteria. To validate the proposed framework, a detailed experimental study with vision-guided robotic manipulators is presented. △ Less

Submitted 9 September, 2023; v1 submitted 16 August, 2020; originally announced August 2020.

Comments: 8 pages, 12 figures

Analyzing Upper Bounds on Mean Absolute Errors for Deep Neural Network Based Vector-to-Vector Regression

Authors: Jun Qi, Jun Du, Sabato Marco Siniscalchi, Xiaoli Ma, Chin-Hui Lee

Abstract: In this paper, we show that, in vector-to-vector regression utilizing deep neural networks (DNNs), a generalized loss of mean absolute error (MAE) between the predicted and expected feature vectors is upper bounded by the sum of an approximation error, an estimation error, and an optimization error. Leveraging upon error decomposition techniques in statistical learning theory and non-convex optimi… ▽ More In this paper, we show that, in vector-to-vector regression utilizing deep neural networks (DNNs), a generalized loss of mean absolute error (MAE) between the predicted and expected feature vectors is upper bounded by the sum of an approximation error, an estimation error, and an optimization error. Leveraging upon error decomposition techniques in statistical learning theory and non-convex optimization theory, we derive upper bounds for each of the three aforementioned errors and impose necessary constraints on DNN models. Moreover, we assess our theoretical results through a set of image de-noising and speech enhancement experiments. Our proposed upper bounds of MAE for DNN based vector-to-vector regression are corroborated by the experimental results and the upper bounds are valid with and without the "over-parametrization" technique. △ Less

Submitted 4 August, 2020; originally announced August 2020.

Journal ref: IEEE Transactions on Signal Processing, Vol 68, pp. 3411-3422, 2020

Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement

Authors: Jun Qi, Hu Hu, Yannan Wang, Chao-Han Huck Yang, Sabato Marco Siniscalchi, Chin-Hui Lee

Abstract: This paper investigates different trade-offs between the number of model parameters and enhanced speech qualities by employing several deep tensor-to-vector regression models for speech enhancement. We find that a hybrid architecture, namely CNN-TT, is capable of maintaining a good quality performance with a reduced model parameter size. CNN-TT is composed of several convolutional layers at the bo… ▽ More This paper investigates different trade-offs between the number of model parameters and enhanced speech qualities by employing several deep tensor-to-vector regression models for speech enhancement. We find that a hybrid architecture, namely CNN-TT, is capable of maintaining a good quality performance with a reduced model parameter size. CNN-TT is composed of several convolutional layers at the bottom for feature extraction to improve speech quality and a tensor-train (TT) output layer on the top to reduce model parameters. We first derive a new upper bound on the generalization power of the convolutional neural network (CNN) based vector-to-vector regression models. Then, we provide experimental evidence on the Edinburgh noisy speech corpus to demonstrate that, in single-channel speech enhancement, CNN outperforms DNN at the expense of a small increment of model sizes. Besides, CNN-TT slightly outperforms the CNN counterpart by utilizing only 32\% of the CNN model parameters. Besides, further performance improvement can be attained if the number of CNN-TT parameters is increased to 44\% of the CNN model size. Finally, our experiments of multi-channel speech enhancement on a simulated noisy WSJ0 corpus demonstrate that our proposed hybrid CNN-TT architecture achieves better results than both DNN and CNN models in terms of better-enhanced speech qualities and smaller parameter sizes. △ Less

Submitted 2 August, 2020; v1 submitted 25 July, 2020; originally announced July 2020.

Comments: Accepted to InterSpeech 2020