Spectral Introspection Identifies Group Training Dynamics in Deep Neural Networks for Neuroimaging
Authors:
Bradley T. Baker,
Vince D. Calhoun,
Sergey M. Plis
Abstract:
Neural networks, whice have had a profound effect on how researchers study complex phenomena, do so through a complex, nonlinear mathematical structure which can be difficult for human researchers to interpret. This obstacle can be especially salient when researchers want to better understand the emergence of particular model behaviors such as bias, overfitting, overparametrization, and more. In N…
▽ More
Neural networks, whice have had a profound effect on how researchers study complex phenomena, do so through a complex, nonlinear mathematical structure which can be difficult for human researchers to interpret. This obstacle can be especially salient when researchers want to better understand the emergence of particular model behaviors such as bias, overfitting, overparametrization, and more. In Neuroimaging, the understanding of how such phenomena emerge is fundamental to preventing and informing users of the potential risks involved in practice. In this work, we present a novel introspection framework for Deep Learning on Neuroimaging data, which exploits the natural structure of gradient computations via the singular value decomposition of gradient components during reverse-mode auto-differentiation. Unlike post-hoc introspection techniques, which require fully-trained models for evaluation, our method allows for the study of training dynamics on the fly, and even more interestingly, allow for the decomposition of gradients based on which samples belong to particular groups of interest. We demonstrate how the gradient spectra for several common deep learning models differ between schizophrenia and control participants from the COBRE study, and illustrate how these trajectories may reveal specific training dynamics helpful for further analysis.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
Energy Efficiency Maximization in IRS-enabled Phase Cooperative PS-SWIPT based Self-sustainable IoT Network
Authors:
Haleema Sadia,
Ahmad Kamal Hassan,
Ziaul Haq Abbas,
Ghulam Abbas,
Thar Baker
Abstract:
Power splitting based simultaneous wireless information and power transfer (PS-SWIPT) appears to be a promising solution to support future self-sustainable Internet of Things (SS-IoT) networks. However, the performance of these networks is constrained by radio frequency signal strength and channel impairments. To address this challenge, intelligent reflecting surfaces (IRSs) are introduced in PS-S…
▽ More
Power splitting based simultaneous wireless information and power transfer (PS-SWIPT) appears to be a promising solution to support future self-sustainable Internet of Things (SS-IoT) networks. However, the performance of these networks is constrained by radio frequency signal strength and channel impairments. To address this challenge, intelligent reflecting surfaces (IRSs) are introduced in PS-SWIPT based SS-IoT networks to improve network efficiency by controlling signal reflections. In this article, an IRS-enabled phase cooperative framework is proposed to improve energy efficiency (EE) of the IoT network $({\mathtt {I}}^{net})$ using phase shifts of the user network $({\mathtt {U}^{net})}$, without constraining hardware resources at ${\mathtt {U}^{net}}$. We exploit transmit beamforming (BF) at access points (APs) and phase shifts optimization at the IRS end with phase effective cooperation between APs to enhance ${\mathtt {I}}^{net}$ EE performance. The maximization problem turns out to be NP-hard, so first, an alternating optimization (AO) is solved for the ${\mathtt {U}^{net}}$ using low computational complexity heuristic BF approaches, namely, transmit minimum-mean-square-error and zero-forcing BF, and phase optimization is performed using semidefinite relaxation (SDR) approach. To combat the computational complexity of AO, we also propose an alternative solution by exploiting heuristic BF schemes and an iterative algorithm, i.e., the element-wise block-coordinate descent method for phase shifts optimization. Next, EE maximization is solved for the ${\mathtt {I}^{net}}$ by optimizing the PS ratio and active BF vectors by exploiting optimal phase shifts of the ${\mathtt {U}}^{net}$. Simulation results confirm that employing IRS phase cooperation in PS-SWIPT based SS-IoT networks can significantly improve EE performance of ${\mathtt {I}^{net}}$ without constraining resources.
△ Less
Submitted 10 January, 2024;
originally announced January 2024.
