-
A Control-Oriented Simplified Single Particle Model with Grouped Parameter and Sensitivity Analysis for Lithium-Ion Batteries
Authors:
Feng Guo,
Luis D. Couto
Abstract:
Lithium-ion batteries are widely used in transportation, energy storage, and consumer electronics, driving the need for reliable battery management systems (BMS) for state estimation and control. The Single Particle Model (SPM) balances computational efficiency and accuracy but faces challenges in parameter estimation due to numerous parameters. Current SPM models using parabolic approximation int…
▽ More
Lithium-ion batteries are widely used in transportation, energy storage, and consumer electronics, driving the need for reliable battery management systems (BMS) for state estimation and control. The Single Particle Model (SPM) balances computational efficiency and accuracy but faces challenges in parameter estimation due to numerous parameters. Current SPM models using parabolic approximation introduce intermediate variables and hard to do parameter grouping. This study presents a control-oriented SPM reformulation that employs parameter grouping and parabolic approximation to simplify model parameters while using average and surface lithium-ion concentrations as model output. By parameter grouping, the original 17 parameters were reduced to 9 grouped parameters. The reformulated model achieves a reduced-order ordinary differential equation form while maintaining mathematical accuracy equivalent to the pre-grouped discretized SPM. Through Sobol sensitivity analysis under various current profiles, the grouped parameters were reduced from 9 to 6 highly sensitive parameters. Results demonstrate that estimating these 6 parameters achieves comparable practical accuracy to estimating all 9 parameters, with faster convergence. This control-oriented SPM enhances BMS applications by facilitating state estimation and control while reducing parameter estimation requirements.
△ Less
Submitted 26 May, 2025; v1 submitted 8 April, 2025;
originally announced April 2025.
-
Optimizing Parameter Estimation for Electrochemical Battery Model: A Comparative Analysis of Operating Profiles on Computational Efficiency and Accuracy
Authors:
Feng Guo,
Luis D. Couto,
Khiem Trad,
Grietus Mulder,
Keivan Haghverdi,
Guillaume Thenaisie
Abstract:
Parameter estimation in electrochemical models remains a significant challenge in their application. This study investigates the impact of different operating profiles on electrochemical model parameter estimation to identify the optimal conditions. In particular, the present study is focused on Nickel Manganese Cobalt Oxide(NMC) lithium-ion batteries. Based on five fundamental current profiles (C…
▽ More
Parameter estimation in electrochemical models remains a significant challenge in their application. This study investigates the impact of different operating profiles on electrochemical model parameter estimation to identify the optimal conditions. In particular, the present study is focused on Nickel Manganese Cobalt Oxide(NMC) lithium-ion batteries. Based on five fundamental current profiles (C/5, C/2, 1C, Pulse, DST), 31 combinations of conditions were generated and used for parameter estimation and validation, resulting in 961 evaluation outcomes. The Particle Swarm Optimization is employed for parameter identification in electrochemical models, specifically using the Single Particle Model (SPM). The analysis considered three dimensions: model voltage output error, parameter estimation error, and time cost. Results show that using all five profiles (C/5, C/2, 1C, Pulse, DST) minimizes voltage output error, while {C/5, C/2, Pulse, DST} minimizes parameter estimation error. The shortest time cost is achieved with {1C}. When considering both model voltage output and parameter errors, {C/5, C/2, 1C, DST} is optimal. For minimizing model voltage output error and time cost, {C/2, 1C} is best, while {1C} is ideal for parameter error and time cost. The comprehensive optimal condition is {C/5, C/2, 1C, DST}. These findings provide guidance for selecting current conditions tailored to specific needs.
△ Less
Submitted 4 March, 2025; v1 submitted 1 March, 2025;
originally announced March 2025.
-
Comparative Performance Analysis of Numerical Discretization Methods for Electrochemical Models of Lithium-ion Batteries
Authors:
Feng Guo,
Luis D. Couto
Abstract:
This study evaluates numerical discretization methods for the Single Particle Model (SPM) used in electrochemical modeling. The methods include the Finite Difference Method (FDM), spectral methods, Padé approximation, and parabolic approximation. Evaluation criteria are accuracy, execution time, and memory usage, aiming to guide method selection for electrochemical models. Under constant current c…
▽ More
This study evaluates numerical discretization methods for the Single Particle Model (SPM) used in electrochemical modeling. The methods include the Finite Difference Method (FDM), spectral methods, Padé approximation, and parabolic approximation. Evaluation criteria are accuracy, execution time, and memory usage, aiming to guide method selection for electrochemical models. Under constant current conditions, the FDM explicit Euler and Runge-Kutta methods show significant errors, while the FDM implicit Euler method improves accuracy with more nodes. The spectral method achieves the best accuracy and convergence with as few as five nodes. The Padé approximation exhibits increasing errors with higher current, and the parabolic approximation shows higher errors than the converged spectral and FDM implicit Euler methods. Under dynamic conditions, frequency domain analysis indicates that the FDM, spectral, and Padé approximation methods improve high-frequency response by increasing node count or method order. In terms of execution time, the parabolic method is fastest, followed by the Padé approximation. The spectral method is faster than FDM, while the FDM implicit Euler method is the slowest. Memory usage is lowest for the parabolic and Padé methods, moderate for FDM, and highest for the spectral method. These findings provide practical guidance for selecting discretization methods under different operating scenarios.
△ Less
Submitted 2 June, 2025; v1 submitted 27 February, 2025;
originally announced February 2025.
-
Enhanced Masked Image Modeling to Avoid Model Collapse on Multi-modal MRI Datasets
Authors:
Linxuan Han,
Sa Xiao,
Zimeng Li,
Haidong Li,
Xiuchao Zhao,
Yeqing Han,
Fumin Guo,
Xin Zhou
Abstract:
Multi-modal magnetic resonance imaging (MRI) provides information of lesions for computer-aided diagnosis from different views. Deep learning algorithms are suitable for identifying specific anatomical structures, segmenting lesions, and classifying diseases. Manual labels are limited due to the high expense, which hinders further improvement of accuracy. Self-supervised learning, particularly mas…
▽ More
Multi-modal magnetic resonance imaging (MRI) provides information of lesions for computer-aided diagnosis from different views. Deep learning algorithms are suitable for identifying specific anatomical structures, segmenting lesions, and classifying diseases. Manual labels are limited due to the high expense, which hinders further improvement of accuracy. Self-supervised learning, particularly masked image modeling (MIM), has shown promise in utilizing unlabeled data. However, we spot model collapse when applying MIM to multi-modal MRI datasets. The performance of downstream tasks does not see any improvement following the collapsed model. To solve model collapse, we analyze and address it in two types: complete collapse and dimensional collapse. We find complete collapse occurs because the collapsed loss value in multi-modal MRI datasets falls below the normally converged loss value. Based on this, the hybrid mask pattern (HMP) masking strategy is introduced to elevate the collapsed loss above the normally converged loss value and avoid complete collapse. Additionally, we reveal that dimensional collapse stems from insufficient feature uniformity in MIM. We mitigate dimensional collapse by introducing the pyramid barlow twins (PBT) module as an explicit regularization method. Overall, we construct the enhanced MIM (E-MIM) with HMP and PBT module to avoid model collapse multi-modal MRI. Experiments are conducted on three multi-modal MRI datasets to validate the effectiveness of our approach in preventing both types of model collapse. By preventing model collapse, the training of the model becomes more stable, resulting in a decent improvement in performance for segmentation and classification tasks. The code is available at https://github.com/LinxuanHan/E-MIM.
△ Less
Submitted 15 January, 2025; v1 submitted 14 July, 2024;
originally announced July 2024.
-
Distributed Least-Squares Optimization Solvers with Differential Privacy
Authors:
Weijia Liu,
Lei Wang,
Fanghong Guo,
Zhengguang Wu,
Hongye Su
Abstract:
This paper studies the distributed least-squares optimization problem with differential privacy requirement of local cost functions, for which two differentially private distributed solvers are proposed. The first is established on the distributed gradient tracking algorithm, by appropriately perturbing the initial values and parameters that contain the privacy-sensitive data with Gaussian and tru…
▽ More
This paper studies the distributed least-squares optimization problem with differential privacy requirement of local cost functions, for which two differentially private distributed solvers are proposed. The first is established on the distributed gradient tracking algorithm, by appropriately perturbing the initial values and parameters that contain the privacy-sensitive data with Gaussian and truncated Laplacian noises, respectively. Rigorous proofs are established to show the achievable trade-off between the (ε, δ)-differential privacy and the computation accuracy. The second solver is established on the combination of the distributed shuffling mechanism and the average consensus algorithm, which enables each agent to obtain a noisy version of parameters characterizing the global gradient. As a result, the least-squares optimization problem can be eventually solved by each agent locally in such a way that any given (ε, δ)-differential privacy requirement can be preserved while the solution may be computed with the accuracy independent of the network size, which makes the latter more suitable for large-scale distributed least-squares problems. Numerical simulations are presented to show the effectiveness of both solvers.
△ Less
Submitted 3 March, 2024;
originally announced March 2024.
-
Waveform-Domain Complementary Signal Sets for Interrupted Sampling Repeater Jamming Suppression
Authors:
Hanning Su,
Qinglong Bao,
Jiameng Pan,
Fucheng Guo,
Weidong Hu
Abstract:
The interrupted-sampling repeater jamming (ISRJ) is coherent and has the characteristic of suppression and deception to degrade the radar detection capabilities. The study focuses on anti-ISRJ techniques in the waveform domain, primarily capitalizing on waveform design and and anti-jamming signal processing methods in the waveform domain. By exploring the relationship between waveform-domain adapt…
▽ More
The interrupted-sampling repeater jamming (ISRJ) is coherent and has the characteristic of suppression and deception to degrade the radar detection capabilities. The study focuses on anti-ISRJ techniques in the waveform domain, primarily capitalizing on waveform design and and anti-jamming signal processing methods in the waveform domain. By exploring the relationship between waveform-domain adaptive matched filtering (WD-AMF) output and waveform-domain signals, we demonstrate that ISRJ can be effectively suppressed when the transmitted waveform exhibits waveform-domain complementarity. We introduce a phase-coded (PC) waveform set with waveform-domain complementarity and propose a method for generating such waveform sets of arbitrary code lengths. The performance of WD-AMF are further developed due to the designed waveforms, and simulations affirm the superior adaptive anti-jamming capabilities of the designed waveforms compared to traditional ones. Remarkably, this improved performance is achieved without the need for prior knowledge of ISRJ interference parameters at either the transmitter or receiver stages.
△ Less
Submitted 18 January, 2024;
originally announced January 2024.
-
Reputation-Based Federated Learning Defense to Mitigate Threats in EEG Signal Classification
Authors:
Zhibo Zhang,
Pengfei Li,
Ahmed Y. Al Hammadi,
Fusen Guo,
Ernesto Damiani,
Chan Yeob Yeun
Abstract:
This paper presents a reputation-based threat mitigation framework that defends potential security threats in electroencephalogram (EEG) signal classification during model aggregation of Federated Learning. While EEG signal analysis has attracted attention because of the emergence of brain-computer interface (BCI) technology, it is difficult to create efficient learning models for EEG analysis bec…
▽ More
This paper presents a reputation-based threat mitigation framework that defends potential security threats in electroencephalogram (EEG) signal classification during model aggregation of Federated Learning. While EEG signal analysis has attracted attention because of the emergence of brain-computer interface (BCI) technology, it is difficult to create efficient learning models for EEG analysis because of the distributed nature of EEG data and related privacy and security concerns. To address these challenges, the proposed defending framework leverages the Federated Learning paradigm to preserve privacy by collaborative model training with localized data from dispersed sources and introduces a reputation-based mechanism to mitigate the influence of data poisoning attacks and identify compromised participants. To assess the efficiency of the proposed reputation-based federated learning defense framework, data poisoning attacks based on the risk level of training data derived by Explainable Artificial Intelligence (XAI) techniques are conducted on both publicly available EEG signal datasets and the self-established EEG signal dataset. Experimental results on the poisoned datasets show that the proposed defense methodology performs well in EEG signal classification while reducing the risks associated with security threats.
△ Less
Submitted 22 October, 2023;
originally announced January 2024.
-
On the Distribution of SINR for Cell-Free Massive MIMO Systems
Authors:
Baolin Chong,
Fengqian Guo,
Hancheng Lu,
Langtian Qin
Abstract:
Cell-free (CF) massive multiple-input multiple-output (mMIMO) has been considered as a potential technology for Beyond 5G communication systems. However, the performance of CF mMIMO systems has not been well studied. Most existing analytical work on CF mMIMO systems is based on the expected signal-to-interference-plus-noise ratio (SINR). The statistical characteristics of the SINR, which is critic…
▽ More
Cell-free (CF) massive multiple-input multiple-output (mMIMO) has been considered as a potential technology for Beyond 5G communication systems. However, the performance of CF mMIMO systems has not been well studied. Most existing analytical work on CF mMIMO systems is based on the expected signal-to-interference-plus-noise ratio (SINR). The statistical characteristics of the SINR, which is critical for emerging applications that focus on extreme events, have not been investigated. To address this issue, in this paper, we attempt to obtain the distribution of SINR in CF mMIMO systems. Considering a downlink CF mMIMO system with pilot contamination, we first give the closed-form expression of the SINR. Based on our analytical work on the two components of the SINR, i.e., desired signal and interference-plus-noise, we then derive the probability density function and cumulative distribution function of the SINR under maximum ratio transmission (MRT) and full-pilot zero-forcing (FZF) precoding, respectively. Subsequently, the closed-form expressions for two more sophisticated performance metrics, i.e., achievable rate and outage probability, can be obtained. Finally, we perform Monte Carlo simulations to validate our analytical work. The results demonstrate the effectiveness of the derived SINR distribution, achievable rate, and outage probability.
△ Less
Submitted 4 October, 2023;
originally announced October 2023.
-
Differentially Private Average Consensus with Improved Accuracy-Privacy Trade-off
Authors:
Lei Wang,
Weijia Liu,
Fanghong Guo,
Zixin Qiao,
Zhengguang Wu
Abstract:
This paper studies the average consensus problem with differential privacy of initial states, for which it is widely recognized that there is a trade-off between the mean-square computation accuracy and privacy level. Considering the trade-off gap between the average consensus algorithm and the centralized averaging approach with differential privacy, we propose a distributed shuffling mechanism b…
▽ More
This paper studies the average consensus problem with differential privacy of initial states, for which it is widely recognized that there is a trade-off between the mean-square computation accuracy and privacy level. Considering the trade-off gap between the average consensus algorithm and the centralized averaging approach with differential privacy, we propose a distributed shuffling mechanism based on the Paillier cryptosystem to generate correlated zero-sum randomness. By randomizing each local privacy-sensitive initial state with an i.i.d. Gaussian noise and the output of the mechanism using Gaussian noises, it is shown that the resulting average consensus algorithm can eliminate the gap in the sense that the accuracy-privacy trade-off of the centralized averaging approach with differential privacy can be almost recovered by appropriately designing the variances of the added noises. We also extend such a design framework with Gaussian noises to the one using Laplace noises, and show that the improved privacy-accuracy trade-off is preserved.
△ Less
Submitted 5 May, 2024; v1 submitted 15 September, 2023;
originally announced September 2023.
-
Waveform-Domain Adaptive Matched Filtering for Suppressing Interrupted-Sampling Repeater Jamming
Authors:
Hanning Su,
Qinglong Bao,
Jiameng Pan,
Fucheng Guo,
Weidong Hu
Abstract:
The inadequate adaptability to flexible interference scenarios remains an unresolved challenge in the majority of techniques utilized for mitigating interrupted-sampling repeater jamming (ISRJ). Matched filtering system based methods is desirable to incorporate anti-ISRJ measures based on prior ISRJ modeling, either preceding or succeeding the matched filtering. Due to the partial matching nature…
▽ More
The inadequate adaptability to flexible interference scenarios remains an unresolved challenge in the majority of techniques utilized for mitigating interrupted-sampling repeater jamming (ISRJ). Matched filtering system based methods is desirable to incorporate anti-ISRJ measures based on prior ISRJ modeling, either preceding or succeeding the matched filtering. Due to the partial matching nature of ISRJ, its characteristics are revealed during the process of matched filtering. Therefore, this paper introduces an extended domain called the waveform domain within the matched filtering process. On this domain, an adaptive matched filtering model, known as the waveform-domain adaptive matched filtering (WD-AMF), is established to tackle the problem of ISRJ suppression without relying on a pre-existing ISRJ model. The output of the WD-AMF encompasses an adaptive filtering term and a compensation term. The adaptive filtering term encompasses the adaptive integration outcomes in the waveform domain, which are determined by an adaptive weighted function. This function, akin to a collection of bandpass filters, decomposes the integrated function into multiple components, some of which contain interference while others do not. The compensation term adheres to an integrated guideline for discerning the presence of signal components or noise within the integrated function. The integration results are then concatenated to reconstruct a compensated matched filter signal output. Simulations are conducted to showcase the exceptional capability of the proposed method in suppressing ISRJ in diverse interference scenarios, even in the absence of a pre-existing ISRJ model.
△ Less
Submitted 13 November, 2023; v1 submitted 6 July, 2023;
originally announced July 2023.
-
HODINet: High-Order Discrepant Interaction Network for RGB-D Salient Object Detection
Authors:
Kang Yi,
Jing Xu,
Xiao Jin,
Fu Guo,
Yan-Feng Wu
Abstract:
RGB-D salient object detection (SOD) aims to detect the prominent regions by jointly modeling RGB and depth information. Most RGB-D SOD methods apply the same type of backbones and fusion modules to identically learn the multimodality and multistage features. However, these features contribute differently to the final saliency results, which raises two issues: 1) how to model discrepant characteri…
▽ More
RGB-D salient object detection (SOD) aims to detect the prominent regions by jointly modeling RGB and depth information. Most RGB-D SOD methods apply the same type of backbones and fusion modules to identically learn the multimodality and multistage features. However, these features contribute differently to the final saliency results, which raises two issues: 1) how to model discrepant characteristics of RGB images and depth maps; 2) how to fuse these cross-modality features in different stages. In this paper, we propose a high-order discrepant interaction network (HODINet) for RGB-D SOD. Concretely, we first employ transformer-based and CNN-based architectures as backbones to encode RGB and depth features, respectively. Then, the high-order representations are delicately extracted and embedded into spatial and channel attentions for cross-modality feature fusion in different stages. Specifically, we design a high-order spatial fusion (HOSF) module and a high-order channel fusion (HOCF) module to fuse features of the first two and the last two stages, respectively. Besides, a cascaded pyramid reconstruction network is adopted to progressively decode the fused features in a top-down pathway. Extensive experiments are conducted on seven widely used datasets to demonstrate the effectiveness of the proposed approach. We achieve competitive performance against 24 state-of-the-art methods under four evaluation metrics.
△ Less
Submitted 3 July, 2023;
originally announced July 2023.
-
Encoding Enhanced Complex CNN for Accurate and Highly Accelerated MRI
Authors:
Zimeng Li,
Sa Xiao,
Cheng Wang,
Haidong Li,
Xiuchao Zhao,
Caohui Duan,
Qian Zhou,
Qiuchen Rao,
Yuan Fang,
Junshuai Xie,
Lei Shi,
Fumin Guo,
Chaohui Ye,
Xin Zhou
Abstract:
Magnetic resonance imaging (MRI) using hyperpolarized noble gases provides a way to visualize the structure and function of human lung, but the long imaging time limits its broad research and clinical applications. Deep learning has demonstrated great potential for accelerating MRI by reconstructing images from undersampled data. However, most existing deep conventional neural networks (CNN) direc…
▽ More
Magnetic resonance imaging (MRI) using hyperpolarized noble gases provides a way to visualize the structure and function of human lung, but the long imaging time limits its broad research and clinical applications. Deep learning has demonstrated great potential for accelerating MRI by reconstructing images from undersampled data. However, most existing deep conventional neural networks (CNN) directly apply square convolution to k-space data without considering the inherent properties of k-space sampling, limiting k-space learning efficiency and image reconstruction quality. In this work, we propose an encoding enhanced (EN2) complex CNN for highly undersampled pulmonary MRI reconstruction. EN2 employs convolution along either the frequency or phase-encoding direction, resembling the mechanisms of k-space sampling, to maximize the utilization of the encoding correlation and integrity within a row or column of k-space. We also employ complex convolution to learn rich representations from the complex k-space data. In addition, we develop a feature-strengthened modularized unit to further boost the reconstruction performance. Experiments demonstrate that our approach can accurately reconstruct hyperpolarized 129Xe and 1H lung MRI from 6-fold undersampled k-space data and provide lung function measurements with minimal biases compared with fully-sampled image. These results demonstrate the effectiveness of the proposed algorithmic components and indicate that the proposed approach could be used for accelerated pulmonary MRI in research and clinical lung disease patient care.
△ Less
Submitted 13 November, 2023; v1 submitted 20 June, 2023;
originally announced June 2023.
-
Achievable Sum Rate Optimization on NOMA-aided Cell-Free Massive MIMO with Finite Blocklength Coding
Authors:
Baolin Chong,
Hancheng Lu,
Yuang Chen,
Langtian Qin,
Fengqian Guo
Abstract:
Non-orthogonal multiple access (NOMA)-aided cell-free massive multiple-input multiple-output (CFmMIMO) has been considered as a promising technology to fulfill strict quality of service requirements for ultra-reliable low-latency communications (URLLC). However, finite blocklength coding (FBC) in URLLC makes it challenging to achieve the optimal performance in the NOMA-aided CFmMIMO system. In thi…
▽ More
Non-orthogonal multiple access (NOMA)-aided cell-free massive multiple-input multiple-output (CFmMIMO) has been considered as a promising technology to fulfill strict quality of service requirements for ultra-reliable low-latency communications (URLLC). However, finite blocklength coding (FBC) in URLLC makes it challenging to achieve the optimal performance in the NOMA-aided CFmMIMO system. In this paper, we investigate the performance of the NOMA-aided CFmMIMO system with FBC in terms of achievable sum rate (ASR). Firstly, we derive a lower bound (LB) on the ergodic data rate. Then, we formulate an ASR maximization problem by jointly considering power allocation and user equipment (UE) clustering. To tackle such an intractable problem, we decompose it into two sub-problems, i.e., the power allocation problem and the UE clustering problem. A successive convex approximation (SCA) algorithm is proposed to solve the power allocation problem by transforming it into a series of geometric programming problems. Meanwhile, two algorithms based on graph theory are proposed to solve the UE clustering problem by identifying negative loops. Finally, alternative optimization is performed to find the maximum ASR of the NOMA-aided CFmMIMO system with FBC. The simulation results demonstrate that the proposed algorithms significantly outperform the benchmark algorithms in terms of ASR under various scenarios.
△ Less
Submitted 25 March, 2024; v1 submitted 3 June, 2023;
originally announced June 2023.
-
Towards the Universal Defense for Query-Based Audio Adversarial Attacks
Authors:
Feng Guo,
Zheng Sun,
Yuxuan Chen,
Lei Ju
Abstract:
Recently, studies show that deep learning-based automatic speech recognition (ASR) systems are vulnerable to adversarial examples (AEs), which add a small amount of noise to the original audio examples. These AE attacks pose new challenges to deep learning security and have raised significant concerns about deploying ASR systems and devices. The existing defense methods are either limited in appli…
▽ More
Recently, studies show that deep learning-based automatic speech recognition (ASR) systems are vulnerable to adversarial examples (AEs), which add a small amount of noise to the original audio examples. These AE attacks pose new challenges to deep learning security and have raised significant concerns about deploying ASR systems and devices. The existing defense methods are either limited in application or only defend on results, but not on process. In this work, we propose a novel method to infer the adversary intent and discover audio adversarial examples based on the AEs generation process. The insight of this method is based on the observation: many existing audio AE attacks utilize query-based methods, which means the adversary must send continuous and similar queries to target ASR models during the audio AE generation process. Inspired by this observation, We propose a memory mechanism by adopting audio fingerprint technology to analyze the similarity of the current query with a certain length of memory query. Thus, we can identify when a sequence of queries appears to be suspectable to generate audio AEs. Through extensive evaluation on four state-of-the-art audio AE attacks, we demonstrate that on average our defense identify the adversary intent with over 90% accuracy. With careful regard for robustness evaluations, we also analyze our proposed defense and its strength to withstand two adaptive attacks. Finally, our scheme is available out-of-the-box and directly compatible with any ensemble of ASR defense models to uncover audio AE attacks effectively without model retraining.
△ Less
Submitted 20 April, 2023;
originally announced April 2023.
-
Towards the Transferable Audio Adversarial Attack via Ensemble Methods
Authors:
Feng Guo,
Zheng Sun,
Yuxuan Chen,
Lei Ju
Abstract:
In recent years, deep learning (DL) models have achieved significant progress in many domains, such as autonomous driving, facial recognition, and speech recognition. However, the vulnerability of deep learning models to adversarial attacks has raised serious concerns in the community because of their insufficient robustness and generalization. Also, transferable attacks have become a prominent me…
▽ More
In recent years, deep learning (DL) models have achieved significant progress in many domains, such as autonomous driving, facial recognition, and speech recognition. However, the vulnerability of deep learning models to adversarial attacks has raised serious concerns in the community because of their insufficient robustness and generalization. Also, transferable attacks have become a prominent method for black-box attacks. In this work, we explore the potential factors that impact adversarial examples (AEs) transferability in DL-based speech recognition. We also discuss the vulnerability of different DL systems and the irregular nature of decision boundaries. Our results show a remarkable difference in the transferability of AEs between speech and images, with the data relevance being low in images but opposite in speech recognition. Motivated by dropout-based ensemble approaches, we propose random gradient ensembles and dynamic gradient-weighted ensembles, and we evaluate the impact of ensembles on the transferability of AEs. The results show that the AEs created by both approaches are valid for transfer to the black box API.
△ Less
Submitted 18 April, 2023;
originally announced April 2023.
-
Automatic Song Translation for Tonal Languages
Authors:
Fenfei Guo,
Chen Zhang,
Zhirui Zhang,
Qixin He,
Kejun Zhang,
Jun Xie,
Jordan Boyd-Graber
Abstract:
This paper develops automatic song translation (AST) for tonal languages and addresses the unique challenge of aligning words' tones with melody of a song in addition to conveying the original meaning. We propose three criteria for effective AST -- preserving meaning, singability and intelligibility -- and design metrics for these criteria. We develop a new benchmark for English--Mandarin song tra…
▽ More
This paper develops automatic song translation (AST) for tonal languages and addresses the unique challenge of aligning words' tones with melody of a song in addition to conveying the original meaning. We propose three criteria for effective AST -- preserving meaning, singability and intelligibility -- and design metrics for these criteria. We develop a new benchmark for English--Mandarin song translation and develop an unsupervised AST system, Guided AliGnment for Automatic Song Translation (GagaST), which combines pre-training with three decoding constraints. Both automatic and human evaluations show GagaST successfully balances semantics and singability.
△ Less
Submitted 24 March, 2022;
originally announced March 2022.
-
A Distributed Parallel Optimization Algorithm via Alternating Direction Method of Multipliers
Authors:
Ziye Liu,
Fanghong Guo,
Wei Wang,
Xiaoqun Wu
Abstract:
Alternating Direction Method of Multipliers (ADMM) algorithm has been widely adopted for solving the distributed optimization problem (DOP).
In this paper, a new distributed parallel ADMM algorithm is proposed, which allows the agents to update their local states and dual variables in a completely distributed and parallel manner by modifying the existing distributed sequential ADMM.
Moreover,…
▽ More
Alternating Direction Method of Multipliers (ADMM) algorithm has been widely adopted for solving the distributed optimization problem (DOP).
In this paper, a new distributed parallel ADMM algorithm is proposed, which allows the agents to update their local states and dual variables in a completely distributed and parallel manner by modifying the existing distributed sequential ADMM.
Moreover, the updating rules and storage method for variables are illustrated.
It is shown that all the agents can reach a consensus by asymptotically converging to the optimal solution.
Besides, the global cost function will converge to the optimal value at a rate of O(1/k).
Simulation results on a numerical example are given to show the effectiveness of the proposed algorithm.
△ Less
Submitted 19 November, 2021;
originally announced November 2021.
-
NOMA Assisted Multi-MEC Offloading for IoVT Networks
Authors:
Fengqian Guo,
Hancheng Lu,
Bo Li,
Dingxuan Li,
Chang Wen Chen
Abstract:
Nowadays, Internet of Video Things (IoVT) grows rapidly in terms of quantity and computation demands. In spite of the higher local computation capability on visual processing compared with conventional Internet of Things devices, IoVT devices need to offload partial visual processing tasks to the mobile edge computing (MEC) server wirelessly due to its larger computation demands. However, visual p…
▽ More
Nowadays, Internet of Video Things (IoVT) grows rapidly in terms of quantity and computation demands. In spite of the higher local computation capability on visual processing compared with conventional Internet of Things devices, IoVT devices need to offload partial visual processing tasks to the mobile edge computing (MEC) server wirelessly due to its larger computation demands. However, visual processing task offloading is limited by uplink throughput and computation capability of the MEC server. To break through these limitations, a novel non-orthogonal multiple access (NOMA) assisted IoVT framework with multiple MEC servers is proposed, where NOMA is exploited to improve uplink throughput and MEC servers are co-located with base stations to provide enough computation capability for offloading. In the proposed framework, the association strategy, uplink visual data transmission assisted by NOMA and division of the visual processing tasks as well as computation resource allocation at the MEC servers are jointly optimized to minimize the total delay of all visual processing tasks, while meeting the delay requirements of all IoVT devices. Simulation results demonstrate that significant performance gains can be achieved by proposed joint optimization with NOMA transmission and multi-MEC offloading in the heterogeneous IoVT network.
△ Less
Submitted 28 July, 2021;
originally announced August 2021.
-
Visible Watermark Removal via Self-calibrated Localization and Background Refinement
Authors:
Jing Liang,
Li Niu,
Fengjun Guo,
Teng Long,
Liqing Zhang
Abstract:
Superimposing visible watermarks on images provides a powerful weapon to cope with the copyright issue. Watermark removal techniques, which can strengthen the robustness of visible watermarks in an adversarial way, have attracted increasing research interest. Modern watermark removal methods perform watermark localization and background restoration simultaneously, which could be viewed as a multi-…
▽ More
Superimposing visible watermarks on images provides a powerful weapon to cope with the copyright issue. Watermark removal techniques, which can strengthen the robustness of visible watermarks in an adversarial way, have attracted increasing research interest. Modern watermark removal methods perform watermark localization and background restoration simultaneously, which could be viewed as a multi-task learning problem. However, existing approaches suffer from incomplete detected watermark and degraded texture quality of restored background. Therefore, we design a two-stage multi-task network to address the above issues. The coarse stage consists of a watermark branch and a background branch, in which the watermark branch self-calibrates the roughly estimated mask and passes the calibrated mask to background branch to reconstruct the watermarked area. In the refinement stage, we integrate multi-level features to improve the texture quality of watermarked area. Extensive experiments on two datasets demonstrate the effectiveness of our proposed method.
△ Less
Submitted 8 August, 2021;
originally announced August 2021.
-
QoS-aware User Grouping Strategy for Downlink Multi-Cell NOMA Systems
Authors:
Fengqian Guo,
Hancheng Lu,
Xiaoda Jiang,
Ming Zhang,
Jun Wu,
Chang Wen Chen
Abstract:
In multi-cell non-orthogonal multiple access (NOMA) systems, designing an appropriate user grouping strategy is an open problem due to diverse quality of service (QoS) requirements and inter-cell interference. In this paper, we exploit both game theory and graph theory to study QoS-aware user grouping strategies, aiming at minimizing power consumption in downlink multi-cell NOMA systems. Under dif…
▽ More
In multi-cell non-orthogonal multiple access (NOMA) systems, designing an appropriate user grouping strategy is an open problem due to diverse quality of service (QoS) requirements and inter-cell interference. In this paper, we exploit both game theory and graph theory to study QoS-aware user grouping strategies, aiming at minimizing power consumption in downlink multi-cell NOMA systems. Under different QoS requirements, we derive the optimal successive interference cancellation (SIC) decoding order with inter-cell interference, which is different from existing SIC decoding order of increasing channel gains, and obtain the corresponding power allocation strategy. Based on this, the exact potential game model of the user grouping strategies adopted by multiple cells is formulated. We prove that, in this game, the problem for each player to find a grouping strategy can be converted into the problem of searching for specific negative loops in the graph composed of users. Bellman-Ford algorithm is expanded to find these negative loops. Furthermore, we design a greedy based suboptimal strategy to approach the optimal solution with polynomial time. Extensive simulations confirm the effectiveness of grouping users with consideration of QoS and inter-cell interference, and show that the proposed strategies can considerably reduce total power consumption comparing with reference strategies.
△ Less
Submitted 27 July, 2021;
originally announced July 2021.
-
Algorithm to Compilation Co-design: An Integrated View of Neural Network Sparsity
Authors:
Fu-Ming Guo,
Austin Huang
Abstract:
Reducing computation cost, inference latency, and memory footprint of neural networks are frequently cited as research motivations for pruning and sparsity. However, operationalizing those benefits and understanding the end-to-end effect of algorithm design and regularization on the runtime execution is not often examined in depth.
Here we apply structured and unstructured pruning to attention w…
▽ More
Reducing computation cost, inference latency, and memory footprint of neural networks are frequently cited as research motivations for pruning and sparsity. However, operationalizing those benefits and understanding the end-to-end effect of algorithm design and regularization on the runtime execution is not often examined in depth.
Here we apply structured and unstructured pruning to attention weights of transformer blocks of the BERT language model, while also expanding block sparse representation (BSR) operations in the TVM compiler. Integration of BSR operations enables the TVM runtime execution to leverage structured pattern sparsity induced by model regularization.
This integrated view of pruning algorithms enables us to study relationships between modeling decisions and their direct impact on sparsity-enhanced execution. Our main findings are: 1) we validate that performance benefits of structured sparsity block regularization must be enabled by the BSR augmentations to TVM, with 4x speedup relative to vanilla PyTorch and 2.2x speedup relative to standard TVM compilation (without expanded BSR support). 2) for BERT attention weights, the end-to-end optimal block sparsity shape in this CPU inference context is not a square block (as in \cite{gray2017gpu}) but rather a linear 32x1 block 3) the relationship between performance and block size / shape is is suggestive of how model regularization parameters interact with task scheduler optimizations resulting in the observed end-to-end performance.
△ Less
Submitted 17 June, 2021; v1 submitted 16 June, 2021;
originally announced June 2021.
-
Detecting False Data Injection Attacks in Smart Grids with Modeling Errors: A Deep Transfer Learning Based Approach
Authors:
Bowen Xu,
Fanghong Guo,
Changyun Wen,
Ruilong Deng,
Wen-An Zhang
Abstract:
Most traditional false data injection attack (FDIA) detection approaches rely on a key assumption, i.e., the power system can be accurately modeled. However, the transmission line parameters are dynamic and cannot be accurately known during operation and thus the involved modeling errors should not be neglected. In this paper, an illustrative case has revealed that modeling errors in transmission…
▽ More
Most traditional false data injection attack (FDIA) detection approaches rely on a key assumption, i.e., the power system can be accurately modeled. However, the transmission line parameters are dynamic and cannot be accurately known during operation and thus the involved modeling errors should not be neglected. In this paper, an illustrative case has revealed that modeling errors in transmission lines significantly weaken the detection effectiveness of conventional FDIA approaches. To tackle this issue, we propose an FDIA detection mechanism from the perspective of transfer learning. Specifically, the simulated power system is treated as a source domain, which provides abundant simulated normal and attack data. The real world's running system whose transmission line parameters are unknown is taken as a target domain where sufficient real normal data are collected for tracking the latest system states online. The designed transfer strategy that aims at making full use of data in hand is divided into two optimization stages. In the first stage, a deep neural network (DNN) is built by simultaneously optimizing several well-designed objective terms with both simulated data and real data, and then it is fine-tuned via real data in the second stage. Several case studies on the IEEE 14-bus and 118-bus systems verify the effectiveness of the proposed mechanism.
△ Less
Submitted 7 September, 2021; v1 submitted 9 April, 2021;
originally announced April 2021.
-
Estimating Uncertainty in Neural Networks for Cardiac MRI Segmentation: A Benchmark Study
Authors:
Matthew Ng,
Fumin Guo,
Labonny Biswas,
Steffen E. Petersen,
Stefan K. Piechnik,
Stefan Neubauer,
Graham Wright
Abstract:
Objective: Convolutional neural networks (CNNs) have demonstrated promise in automated cardiac magnetic resonance image segmentation. However, when using CNNs in a large real-world dataset, it is important to quantify segmentation uncertainty and identify segmentations which could be problematic. In this work, we performed a systematic study of Bayesian and non-Bayesian methods for estimating unce…
▽ More
Objective: Convolutional neural networks (CNNs) have demonstrated promise in automated cardiac magnetic resonance image segmentation. However, when using CNNs in a large real-world dataset, it is important to quantify segmentation uncertainty and identify segmentations which could be problematic. In this work, we performed a systematic study of Bayesian and non-Bayesian methods for estimating uncertainty in segmentation neural networks.
Methods: We evaluated Bayes by Backprop, Monte Carlo Dropout, Deep Ensembles, and Stochastic Segmentation Networks in terms of segmentation accuracy, probability calibration, uncertainty on out-of-distribution images, and segmentation quality control.
Results: We observed that Deep Ensembles outperformed the other methods except for images with heavy noise and blurring distortions. We showed that Bayes by Backprop is more robust to noise distortions while Stochastic Segmentation Networks are more resistant to blurring distortions. For segmentation quality control, we showed that segmentation uncertainty is correlated with segmentation accuracy for all the methods. With the incorporation of uncertainty estimates, we were able to reduce the percentage of poor segmentation to 5% by flagging 31--48% of the most uncertain segmentations for manual review, substantially lower than random review without using neural network uncertainty (reviewing 75--78% of all images).
Conclusion: This work provides a comprehensive evaluation of uncertainty estimation methods and showed that Deep Ensembles outperformed other methods in most cases.
Significance: Neural network uncertainty measures can help identify potentially inaccurate segmentations and alert users for manual review.
△ Less
Submitted 30 December, 2022; v1 submitted 31 December, 2020;
originally announced December 2020.
-
MindReading: An Ultra-Low-Power Photonic Accelerator for EEG-based Human Intention Recognition
Authors:
Qian Lou,
Wenyang Liu,
Weichen Liu,
Feng Guo,
Lei Jiang
Abstract:
A scalp-recording electroencephalography (EEG)-based brain-computer interface (BCI) system can greatly improve the quality of life for people who suffer from motor disabilities. Deep neural networks consisting of multiple convolutional, LSTM and fully-connected layers are created to decode EEG signals to maximize the human intention recognition accuracy. However, prior FPGA, ASIC, ReRAM and photon…
▽ More
A scalp-recording electroencephalography (EEG)-based brain-computer interface (BCI) system can greatly improve the quality of life for people who suffer from motor disabilities. Deep neural networks consisting of multiple convolutional, LSTM and fully-connected layers are created to decode EEG signals to maximize the human intention recognition accuracy. However, prior FPGA, ASIC, ReRAM and photonic accelerators cannot maintain sufficient battery lifetime when processing real-time intention recognition. In this paper, we propose an ultra-low-power photonic accelerator, MindReading, for human intention recognition by only low bit-width addition and shift operations. Compared to prior neural network accelerators, to maintain the real-time processing throughput, MindReading reduces the power consumption by 62.7\% and improves the throughput per Watt by 168\%.
△ Less
Submitted 30 January, 2020;
originally announced January 2020.
-
Generalized Richardson-Lucy (GRL) for analyzing multi-shell diffusion MRI data
Authors:
Fenghua Guo,
Alexander Leemans,
Max A. Viergever,
Flavio Dell'Acqua,
Alberto De Luca
Abstract:
Spherical deconvolution is a widely used approach to quantify fiber orientation distribution from diffusion MRI data. The damped Richardson-Lucy (dRL) is developed to perform robust spherical deconvolution on single shell diffusion MRI data. While the dRL algorithm could in theory be directly applied to multi-shell data, it is not optimised to model the signal from multiple tissue types. In this w…
▽ More
Spherical deconvolution is a widely used approach to quantify fiber orientation distribution from diffusion MRI data. The damped Richardson-Lucy (dRL) is developed to perform robust spherical deconvolution on single shell diffusion MRI data. While the dRL algorithm could in theory be directly applied to multi-shell data, it is not optimised to model the signal from multiple tissue types. In this work, we introduce a new framework based on dRL - dubbed Generalized Richardson Lucy (GRL) - that uses multi-shell data in combination with user-chosen tissue models to disentangle partial volume effects and increase the accuracy in FOD estimation. The optimal weighting of multi-shell data in the fit and the robustness to noise and partial volume effects of GRL was studied with synthetic data. Subsequently, we investigated the performances of GRL in comparison to dRL on a high-resolution diffusion MRI dataset from the Human Connectome Project and on an MRI dataset acquired at 3T on a clinical scanner. The feasibility of including intra-voxel incoherent motion (IVIM) effects in the modelling was studied on a third dataset. Results of simulations show that GRL can robustly disentangle different tissue types at SNR above 20 and improves the angular accuracy of the FOD estimation. On real data, GRL provides signal fraction maps that are physiologically plausible and consistent between datasets. When considering IVIM effects, high blood pseudo-diffusion fraction is observed in the medial temporal lobe and in the sagittal sinus. In comparison to dRL, GRL provides sharper FODs and less spurious peaks in presence of partial volume effects and results in a better tract termination at the grey/white matter interface or at the outer cortical surface. In conclusion, GRL offers a new modular and flexible framework to perform spherical deconvolution of multi-shell data.
△ Less
Submitted 20 April, 2020; v1 submitted 11 October, 2019;
originally announced October 2019.
-
Deep-Waveform: A Learned OFDM Receiver Based on Deep Complex-valued Convolutional Networks
Authors:
Zhongyuan Zhao,
Mehmet C. Vuran,
Fujuan Guo,
Stephen D. Scott
Abstract:
The (inverse) discrete Fourier transform (DFT/IDFT) is often perceived as essential to orthogonal frequency-division multiplexing (OFDM) systems. In this paper, a deep complex-valued convolutional network (DCCN) is developed to recover bits from time-domain OFDM signals without relying on any explicit DFT/IDFT. The DCCN can exploit the cyclic prefix (CP) of OFDM waveform for increased SNR by repla…
▽ More
The (inverse) discrete Fourier transform (DFT/IDFT) is often perceived as essential to orthogonal frequency-division multiplexing (OFDM) systems. In this paper, a deep complex-valued convolutional network (DCCN) is developed to recover bits from time-domain OFDM signals without relying on any explicit DFT/IDFT. The DCCN can exploit the cyclic prefix (CP) of OFDM waveform for increased SNR by replacing DFT with a learned linear transform, and has the advantage of combining CP-exploitation, channel estimation, and intersymbol interference (ISI) mitigation, with a complexity of $\mathcal{O}(N^2)$. Numerical tests show that the DCCN receiver can outperform the legacy channel estimators based on ideal and approximate linear minimum mean square error (LMMSE) estimation and a conventional CP-enhanced technique in Rayleigh fading channels with various delay spreads and mobility. The proposed approach benefits from the expressive nature of complex-valued neural networks, which, however, currently lack support from popular deep learning platforms. In response, guidelines of exact and approximate implementations of a complex-valued convolutional layer are provided for the design and analysis of convolutional networks for wireless PHY. Furthermore, a suite of novel training techniques are developed to improve the convergence and generalizability of the trained model in fading channels. This work demonstrates the capability of deep neural networks in processing OFDM waveforms and the results suggest that the FFT processor in OFDM receivers can be replaced by a hardware AI accelerator.
△ Less
Submitted 5 May, 2021; v1 submitted 16 October, 2018;
originally announced October 2018.
-
SDPTools: High Precision SDP Solver in Maple
Authors:
Feng Guo
Abstract:
Semidefinite programs are an important class of convex optimization problems. It can be solved efficiently by SDP solvers in Matlab, such as SeDuMi, SDPT3, DSDP. However, since we are running fixed precision SDP solvers in Matlab, for some applications, due to the numerical error, we can not get good results. SDPTools is a Maple package to solve SDP in high precision. We apply SDPTools to the cert…
▽ More
Semidefinite programs are an important class of convex optimization problems. It can be solved efficiently by SDP solvers in Matlab, such as SeDuMi, SDPT3, DSDP. However, since we are running fixed precision SDP solvers in Matlab, for some applications, due to the numerical error, we can not get good results. SDPTools is a Maple package to solve SDP in high precision. We apply SDPTools to the certification of the global optimum of rational functions. For the Rumps Model Problem, we obtain the best numerical results so far.
△ Less
Submitted 21 December, 2011;
originally announced December 2011.
