-
A Novel Approach to Classify Power Quality Signals Using Vision Transformers
Authors:
Ahmad Mohammad Saber,
Alaa Selim,
Mohamed M. Hammad,
Amr Youssef,
Deepa Kundur,
Ehab El-Saadany
Abstract:
With the rapid integration of electronically interfaced renewable energy resources and loads into smart grids, there is increasing interest in power quality disturbances (PQD) classification to enhance the security and efficiency of these grids. This paper introduces a new approach to PQD classification based on the Vision Transformer (ViT) model. When a PQD occurs, the proposed approach first con…
▽ More
With the rapid integration of electronically interfaced renewable energy resources and loads into smart grids, there is increasing interest in power quality disturbances (PQD) classification to enhance the security and efficiency of these grids. This paper introduces a new approach to PQD classification based on the Vision Transformer (ViT) model. When a PQD occurs, the proposed approach first converts the power quality signal into an image and then utilizes a pre-trained ViT to accurately determine the class of the PQD. Unlike most previous works, which were limited to a few disturbance classes or small datasets, the proposed method is trained and tested on a large dataset with 17 disturbance classes. Our experimental results show that the proposed ViT-based approach achieves PQD classification precision and recall of 98.28% and 97.98%, respectively, outperforming recently proposed techniques applied to the same dataset.
△ Less
Submitted 3 September, 2024; v1 submitted 16 August, 2024;
originally announced September 2024.
-
Artificial Neural Network and Deep Learning: Fundamentals and Theory
Authors:
M. M. Hammad
Abstract:
"Artificial Neural Network and Deep Learning: Fundamentals and Theory" offers a comprehensive exploration of the foundational principles and advanced methodologies in neural networks and deep learning. This book begins with essential concepts in descriptive statistics and probability theory, laying a solid groundwork for understanding data and probability distributions. As the reader progresses, t…
▽ More
"Artificial Neural Network and Deep Learning: Fundamentals and Theory" offers a comprehensive exploration of the foundational principles and advanced methodologies in neural networks and deep learning. This book begins with essential concepts in descriptive statistics and probability theory, laying a solid groundwork for understanding data and probability distributions. As the reader progresses, they are introduced to matrix calculus and gradient optimization, crucial for training and fine-tuning neural networks. The book delves into multilayer feed-forward neural networks, explaining their architecture, training processes, and the backpropagation algorithm. Key challenges in neural network optimization, such as activation function saturation, vanishing and exploding gradients, and weight initialization, are thoroughly discussed. The text covers various learning rate schedules and adaptive algorithms, providing strategies to optimize the training process. Techniques for generalization and hyperparameter tuning, including Bayesian optimization and Gaussian processes, are also presented to enhance model performance and prevent overfitting. Advanced activation functions are explored in detail, categorized into sigmoid-based, ReLU-based, ELU-based, miscellaneous, non-standard, and combined types. Each activation function is examined for its properties and applications, offering readers a deep understanding of their impact on neural network behavior. The final chapter introduces complex-valued neural networks, discussing complex numbers, functions, and visualizations, as well as complex calculus and backpropagation algorithms. This book equips readers with the knowledge and skills necessary to design, and optimize advanced neural network models, contributing to the ongoing advancements in artificial intelligence.
△ Less
Submitted 12 August, 2024;
originally announced August 2024.
-
Comprehensive Survey of Complex-Valued Neural Networks: Insights into Backpropagation and Activation Functions
Authors:
M. M. Hammad
Abstract:
Artificial neural networks (ANNs), particularly those employing deep learning models, have found widespread application in fields such as computer vision, signal processing, and wireless communications, where complex numbers are crucial. Despite the prevailing use of real-number implementations in current ANN frameworks, there is a growing interest in developing ANNs that utilize complex numbers.…
▽ More
Artificial neural networks (ANNs), particularly those employing deep learning models, have found widespread application in fields such as computer vision, signal processing, and wireless communications, where complex numbers are crucial. Despite the prevailing use of real-number implementations in current ANN frameworks, there is a growing interest in developing ANNs that utilize complex numbers. This paper presents a comprehensive survey of recent advancements in complex-valued neural networks (CVNNs), focusing on their activation functions (AFs) and learning algorithms. We delve into the extension of the backpropagation algorithm to the complex domain, which enables the training of neural networks with complex-valued inputs, weights, AFs, and outputs. This survey considers three complex backpropagation algorithms: the complex derivative approach, the partial derivatives approach, and algorithms incorporating the Cauchy-Riemann equations. A significant challenge in CVNN design is the identification of suitable nonlinear Complex Valued Activation Functions (CVAFs), due to the conflict between boundedness and differentiability over the entire complex plane as stated by Liouville theorem. We examine both fully complex AFs, which strive for boundedness and differentiability, and split AFs, which offer a practical compromise despite not preserving analyticity. This review provides an in-depth analysis of various CVAFs essential for constructing effective CVNNs. Moreover, this survey not only offers a comprehensive overview of the current state of CVNNs but also contributes to ongoing research and development by introducing a new set of CVAFs (fully complex, split and complex amplitude-phase AFs).
△ Less
Submitted 27 July, 2024;
originally announced July 2024.
-
Deep Learning Activation Functions: Fixed-Shape, Parametric, Adaptive, Stochastic, Miscellaneous, Non-Standard, Ensemble
Authors:
M. M. Hammad
Abstract:
In the architecture of deep learning models, inspired by biological neurons, activation functions (AFs) play a pivotal role. They significantly influence the performance of artificial neural networks. By modulating the non-linear properties essential for learning complex patterns, AFs are fundamental in both classification and regression tasks. This paper presents a comprehensive review of various…
▽ More
In the architecture of deep learning models, inspired by biological neurons, activation functions (AFs) play a pivotal role. They significantly influence the performance of artificial neural networks. By modulating the non-linear properties essential for learning complex patterns, AFs are fundamental in both classification and regression tasks. This paper presents a comprehensive review of various types of AFs, including fixed-shape, parametric, adaptive, stochastic/probabilistic, non-standard, and ensemble/combining types. We begin with a systematic taxonomy and detailed classification frameworks that delineates the principal characteristics of AFs and organizes them based on their structural and functional distinctions. Our in-depth analysis covers primary groups such as sigmoid-based, ReLU-based, and ELU-based AFs, discussing their theoretical foundations, mathematical formulations, and specific benefits and limitations in different contexts. We also highlight key attributes of AFs such as output range, monotonicity, and smoothness. Furthermore, we explore miscellaneous AFs that do not conform to these categories but have shown unique advantages in specialized applications. Non-standard AFs are also explored, showcasing cutting-edge variations that challenge traditional paradigms and offer enhanced adaptability and model performance. We examine strategies for combining multiple AFs to leverage complementary properties. The paper concludes with a comparative evaluation of 12 state-of-the-art AFs, using rigorous statistical and experimental methodologies to assess their efficacy. This analysis not only aids practitioners in selecting and designing the most appropriate AFs for their specific deep learning tasks but also encourages continued innovation in AF development within the machine learning community.
△ Less
Submitted 14 July, 2024;
originally announced July 2024.
-
Conformable Fractional Bohr Hamiltonian with Bonatsos and Double-Well Sextic Potentials
Authors:
M. M. Hammad
Abstract:
Using the conformable fractional calculus, a new formulation of the Bohr Hamiltonian is introduced. The conformable fractional energy spectra of free- and two- parameters anharmonic oscillator potentials are investigated. The energy eigenvalues and wave functions are calculated utilizing the finite-difference discretization method. It is proved that the conformable fractional spectra of the free-p…
▽ More
Using the conformable fractional calculus, a new formulation of the Bohr Hamiltonian is introduced. The conformable fractional energy spectra of free- and two- parameters anharmonic oscillator potentials are investigated. The energy eigenvalues and wave functions are calculated utilizing the finite-difference discretization method. It is proved that the conformable fractional spectra of the free-parameter Bonatsos potentials, $\frac{β^{2 n}}{2}$, close completely the gaps between the classical spectra of the vibrational U(5) dynamical symmetry, the $E(5)-β^{2 n}$ models, and the E(5) critical point symmetry. The ground effective sextic potential, which generates both the ground state and the $β$ excited states $0^+$, is considered to have two degenerate minima. In this case, the conformable fractional spectra of sextic potentials show a change, as a function of barrier height, from $γ$-unstable O(6) energy level sequence to the spectrum of $E(5)-β^{6}$ model and simultaneously provide new features. The shape coexistence phenomena, in the ground band states, are identified. The energy spectrum and shape coexistence with mixing phenomena in $\, ^{96}\text{MO}$ nucleus are discussed in the framework of the conformable fractional Bohr Hamiltonian.
△ Less
Submitted 23 November, 2023;
originally announced November 2023.
-
Algebraic solutions for $o(12) {\leftrightarrow} u(2) \otimes u(10)$ quantum phase transitions in the proton-neutron interacting boson model
Authors:
M. M. Hammad,
Andriana Martinou,
Dennis Bonatsos
Abstract:
A simple systematic procedure to construct the proton-neutron unitary, $u_{\text{sd}}^{πν}{(12)}$, orthogonal, $o_{\text{sd}}^{πν}{(12)}$, and quasi-spin $\text{su}_{\text{sd}}^{πν}{(1,1)}$ algebras of the sd bosonic system is presented. New algebraic substructures of these algebras are discussed and the explicit formulae for their generators and Casimir operators are given in the spherical tensor…
▽ More
A simple systematic procedure to construct the proton-neutron unitary, $u_{\text{sd}}^{πν}{(12)}$, orthogonal, $o_{\text{sd}}^{πν}{(12)}$, and quasi-spin $\text{su}_{\text{sd}}^{πν}{(1,1)}$ algebras of the sd bosonic system is presented. New algebraic substructures of these algebras are discussed and the explicit formulae for their generators and Casimir operators are given in the spherical tensor form. The complementarity relationship of the Casimir operators of the $\text{su}_{\text{sd}}^{πν}{(1,1)}$ and $o_{\text{sd}}^{πν}{(12)}$ is derived. The exact algebraic solutions of the quantum phase transition Hamiltonian between the $o_{\text{sd}}^{πν}{(12)}$ and $u_s^{πν}{(2)} \otimes u_d^{πν}{(10)}$ limits has been considered, for the first time, in the framework of affine $\text{su}_{\text{sd}}^{πν}{(1,1)}$ Lie algebra. The low lying energy spectra of the $\, ^{70}\text{Ge},\, ^{76-78}\text{Se},\, ^{96-98}\text{Mo},\text{and}\, ^{100-102}\text{Ru}$ isotopes are calculated using the $o_{\text{sd}}^{πν} {(12)} {\leftrightarrow} u_s^{πν}{(2)} \otimes u_d^{πν}{(10)}$ transition Hamiltonian. The good agreement of our computation with empirical result in these isotopes emphasizes the importance of $u_s^{πν}{(2)} \otimes u_d^{πν}{(10)}$ limit. With this addition, symmetry can be extended to many nuclei.
△ Less
Submitted 7 November, 2023;
originally announced November 2023.
-
Triaxial nuclei and analytical solutions of the conformable fractional Bohr Hamiltonian with some exponential-type potentials
Authors:
M. M. Hammad,
M. M. Yahia,
Dennis Bonatsos
Abstract:
New approximate analytical solutions have been obtained for the conformable fractional collective Bohr Hamiltonian suitable for triaxial nuclei, with the harmonic oscillator in γ-part of the collective potential and different exponential-type potentials (namely Morse, Tietz-Hua, and multi-parameter exponential-type potential) in \b{eta}-part. The conformable fractional Nikiforov-Uvarov approach is…
▽ More
New approximate analytical solutions have been obtained for the conformable fractional collective Bohr Hamiltonian suitable for triaxial nuclei, with the harmonic oscillator in γ-part of the collective potential and different exponential-type potentials (namely Morse, Tietz-Hua, and multi-parameter exponential-type potential) in \b{eta}-part. The conformable fractional Nikiforov-Uvarov approach is used to derive closed analytical formulas for energy spectra and wave functions. The relation between the conformable fractional spectra of the three potentials and the Z(5) spectrum has been studied. The evolution of the energy spectra as a function of the potential parameters has been investigated for the three potentials. The normalized B(E2) transition rates and spectra have been calculated and compared with the experimental data and theoretical predictions of Kratzer potential. The predictions can describe well the experimental results for 114,116Pd, 126,128Xe, and 192,194Pt isotopes.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
Statistics for Machine Learning with Mathematica Applications
Authors:
M. M. Hammad
Abstract:
In recent years, the field of statistics has experienced a surge in interest and application, largely due to significant advances in computer technology. This progress has led to remarkable developments in statistics methods and algorithms, enabling their widespread adoption across various disciplines. Key areas benefiting from these advancements include machine learning, economics, finance, geoph…
▽ More
In recent years, the field of statistics has experienced a surge in interest and application, largely due to significant advances in computer technology. This progress has led to remarkable developments in statistics methods and algorithms, enabling their widespread adoption across various disciplines. Key areas benefiting from these advancements include machine learning, economics, finance, geophysics, molecular modeling, computational systems biology, operations research, and engineering. For example, in machine learning, statistics forms the foundation for algorithms used in regression, classification, clustering, and deep learning to analyze vast datasets and make predictions. Mathematica, among other tools, has played a significant role in enabling the integration of statistics and computer technology, facilitating deeper exploration of data-driven insights and groundbreaking discoveries across diverse domains. With a rich library of functions, Mathematica allows users to calculate measures of central tendency, dispersion, and correlation, as well as perform hypothesis testing and estimation. Moreover, it supports probability distributions, making simulations and probabilistic modeling tasks more accessible. This monograph presents the main theorems in mathematical statistics, ranging from basic descriptive statistics to sophisticated inferential techniques. In addition, we have created, more than 200 manipulates cover different scenarios in statistics, more than 500 light Mathematica codes (examples) and 25 programs (procedures) that follow the principles of testing hypotheses and estimation theory. The code will run as-is with no code from prior algorithms or third parties required beyond the installation of Mathematica.
△ Less
Submitted 1 August, 2023;
originally announced October 2023.
-
Mathematics for Machine Learning and Data Science: Optimization with Mathematica Applications
Authors:
M. M. Hammad,
M. M. Yahia
Abstract:
The field of optimization has gotten a lot of interest in recent years owing to significant advances in computer technology. Numerous issues in machine learning, economics, finance, geophysics, molecular modeling, computational systems biology, operations research, and all areas of engineering are now being resolved owing to the rapid growth of optimization methods and algorithms. This monograph p…
▽ More
The field of optimization has gotten a lot of interest in recent years owing to significant advances in computer technology. Numerous issues in machine learning, economics, finance, geophysics, molecular modeling, computational systems biology, operations research, and all areas of engineering are now being resolved owing to the rapid growth of optimization methods and algorithms. This monograph presents the main theorems in linear algebra, convex sets, convex functions, single variable optimization, multivariable optimization, and their corresponding algorithms. We also briefly touch upon the constrained nonlinear optimization. We have found the Wolfram language to be ideal for specifying algorithms in human readable form. To minimize nonlinear objective functions, we have created 27 Mathematica functions that follow the principles of 18 algorithms. The code examples were carefully designed to demonstrate the purpose of given algorithm. The code for each algorithm will run as is with no code from prior algorithms or third parties required beyond the installation of Mathematica.
△ Less
Submitted 12 February, 2023;
originally announced February 2023.
