-
Quantitative Analysis of Deeply Quantized Tiny Neural Networks Robust to Adversarial Attacks
Authors:
Idris Zakariyya,
Ferheen Ayaz,
Mounia Kharbouche-Harrari,
Jeremy Singer,
Sye Loong Keoh,
Danilo Pau,
José Cano
Abstract:
Reducing the memory footprint of Machine Learning (ML) models, especially Deep Neural Networks (DNNs), is imperative to facilitate their deployment on resource-constrained edge devices. However, a notable drawback of DNN models lies in their susceptibility to adversarial attacks, wherein minor input perturbations can deceive them. A primary challenge revolves around the development of accurate, re…
▽ More
Reducing the memory footprint of Machine Learning (ML) models, especially Deep Neural Networks (DNNs), is imperative to facilitate their deployment on resource-constrained edge devices. However, a notable drawback of DNN models lies in their susceptibility to adversarial attacks, wherein minor input perturbations can deceive them. A primary challenge revolves around the development of accurate, resilient, and compact DNN models suitable for deployment on resource-constrained edge devices. This paper presents the outcomes of a compact DNN model that exhibits resilience against both black-box and white-box adversarial attacks. This work has achieved this resilience through training with the QKeras quantization-aware training framework. The study explores the potential of QKeras and an adversarial robustness technique, Jacobian Regularization (JR), to co-optimize the DNN architecture through per-layer JR methodology. As a result, this paper has devised a DNN model employing this co-optimization strategy based on Stochastic Ternary Quantization (STQ). Its performance was compared against existing DNN models in the face of various white-box and black-box attacks. The experimental findings revealed that, the proposed DNN model had small footprint and on average, it exhibited better performance than Quanos and DS-CNN MLCommons/TinyML (MLC/T) benchmarks when challenged with white-box and black-box attacks, respectively, on the CIFAR-10 image and Google Speech Commands audio datasets.
△ Less
Submitted 11 March, 2025;
originally announced March 2025.
-
Green UAV-enabled Internet-of-Things Network with AI-assisted NOMA for Disaster Management
Authors:
Muhammad Ali Jamshed,
Ferheen Ayaz,
Aryan Kaushik,
Carlo Fischione,
Masood Ur-Rehman
Abstract:
Unmanned aerial vehicle (UAV)-assisted communication is becoming a streamlined technology in providing improved coverage to the internet-of-things (IoT) based devices. Rapid deployment, portability, and flexibility are some of the fundamental characteristics of UAVs, which make them ideal for effectively managing emergency-based IoT applications. This paper studies a UAV-assisted wireless IoT netw…
▽ More
Unmanned aerial vehicle (UAV)-assisted communication is becoming a streamlined technology in providing improved coverage to the internet-of-things (IoT) based devices. Rapid deployment, portability, and flexibility are some of the fundamental characteristics of UAVs, which make them ideal for effectively managing emergency-based IoT applications. This paper studies a UAV-assisted wireless IoT network relying on non-orthogonal multiple access (NOMA) to facilitate uplink connectivity for devices spread over a disaster region. The UAV setup is capable of relaying the information to the cellular base station (BS) using decode and forward relay protocol. By jointly utilizing the concepts of unsupervised machine learning (ML) and solving the resulting non-convex problem, we can maximize the total energy efficiency (EE) of IoT devices spread over a disaster region. Our proposed approach uses a combination of k-medoids and Silhouette analysis to perform resource allocation, whereas, power optimization is performed using iterative methods. In comparison to the exhaustive search method, our proposed scheme solves the EE maximization problem with much lower complexity and at the same time improves the overall energy consumption of the IoT devices. Moreover, in comparison to a modified version of greedy algorithm, our proposed approach improves the total EE of the system by 19% for a fixed 50k target number of bits.
△ Less
Submitted 14 June, 2024; v1 submitted 26 April, 2023;
originally announced April 2023.
-
Improving Robustness Against Adversarial Attacks with Deeply Quantized Neural Networks
Authors:
Ferheen Ayaz,
Idris Zakariyya,
José Cano,
Sye Loong Keoh,
Jeremy Singer,
Danilo Pau,
Mounia Kharbouche-Harrari
Abstract:
Reducing the memory footprint of Machine Learning (ML) models, particularly Deep Neural Networks (DNNs), is essential to enable their deployment into resource-constrained tiny devices. However, a disadvantage of DNN models is their vulnerability to adversarial attacks, as they can be fooled by adding slight perturbations to the inputs. Therefore, the challenge is how to create accurate, robust, an…
▽ More
Reducing the memory footprint of Machine Learning (ML) models, particularly Deep Neural Networks (DNNs), is essential to enable their deployment into resource-constrained tiny devices. However, a disadvantage of DNN models is their vulnerability to adversarial attacks, as they can be fooled by adding slight perturbations to the inputs. Therefore, the challenge is how to create accurate, robust, and tiny DNN models deployable on resource-constrained embedded devices. This paper reports the results of devising a tiny DNN model, robust to adversarial black and white box attacks, trained with an automatic quantizationaware training framework, i.e. QKeras, with deep quantization loss accounted in the learning loop, thereby making the designed DNNs more accurate for deployment on tiny devices. We investigated how QKeras and an adversarial robustness technique, Jacobian Regularization (JR), can provide a co-optimization strategy by exploiting the DNN topology and the per layer JR approach to produce robust yet tiny deeply quantized DNN models. As a result, a new DNN model implementing this cooptimization strategy was conceived, developed and tested on three datasets containing both images and audio inputs, as well as compared its performance with existing benchmarks against various white-box and black-box attacks. Experimental results demonstrated that on average our proposed DNN model resulted in 8.3% and 79.5% higher accuracy than MLCommons/Tiny benchmarks in the presence of white-box and black-box attacks on the CIFAR-10 image dataset and a subset of the Google Speech Commands audio dataset respectively. It was also 6.5% more accurate for black-box attacks on the SVHN image dataset.
△ Less
Submitted 25 April, 2023;
originally announced April 2023.
-
A Blockchain based Federated Learning for Message Dissemination in Vehicular Networks
Authors:
Ferheen Ayaz,
Zhengguo Sheng,
Daxin Tian,
Yong Liang Guan
Abstract:
Message exchange among vehicles plays an important role in ensuring road safety. Emergency message dissemination is usually carried out by broadcasting. However, high vehicle density and mobility usually lead to challenges in message dissemination such as broadcasting storm and low probability of packet reception. This paper proposes a federated learning based blockchain-assisted message dissemina…
▽ More
Message exchange among vehicles plays an important role in ensuring road safety. Emergency message dissemination is usually carried out by broadcasting. However, high vehicle density and mobility usually lead to challenges in message dissemination such as broadcasting storm and low probability of packet reception. This paper proposes a federated learning based blockchain-assisted message dissemination solution. Similar to the incentive-based Proof-of-Work consensus in blockchain, vehicles compete to become a relay node (miner) by processing the proposed Proof-of-Federated-Learning (PoFL) consensus which is embedded in the smart contract of blockchain. Both theoretical and practical analysis of the proposed solution are provided. Specifically, the proposed blockchain based federated learning results in more number of vehicles uploading their models in a given time, which can potentially lead to a more accurate model in less time as compared to the same solution without using blockchain. It also outperforms the other blockchain approaches for message dissemination by reducing 65.2% of time delay in consensus, improving at least 8.2% message delivery rate and preserving privacy of neighbor vehicle more efficiently. The economic model to incentivize vehicles participating in federated learning and message dissemination is further analyzed using Stackelberg game model.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Analysis of Optimized Threshold with SLM based Blanking Non-Linearity for Impulsive Noise Reduction in Power Line Communication Systems
Authors:
Ferheen Ayaz,
Khaled Rabie,
Bamidele Adebisi
Abstract:
High amplitude impulsive noise (IN) occurrence over power line channels severely degrades the performance of Orthogonal Frequency Division Multiplexing (OFDM)systems. One of the simplest methods to reduce IN is to precede the OFDM demodulator with a blanking non-linearity processor. In this respect, Selective Mapping (SLM) applied to an OFDM signal before the transmitter does not only reduce Peak-…
▽ More
High amplitude impulsive noise (IN) occurrence over power line channels severely degrades the performance of Orthogonal Frequency Division Multiplexing (OFDM)systems. One of the simplest methods to reduce IN is to precede the OFDM demodulator with a blanking non-linearity processor. In this respect, Selective Mapping (SLM) applied to an OFDM signal before the transmitter does not only reduce Peak-to-Average Power Ratio (PAPR) but also increases the resulting Signal-to-Noise Ratio (SNR) when blanking nonlinearity is applied at the receiver. This paper highlights another advantage of SLM based IN reduction, which is the reduced dependency on threshold used for blanking nonlinearity. The simulation results show that the optimal threshold to achieve maximum SNR is found to be constant for phase vectors greater than or equal to 64 in the SLM scheme. If the optimized threshold calculation method is used, the output SNR with SLM OFDM will result in SNR gains of up to 8.6dB compared to the unmodified system, i.e. without implementing SLM. Moreover, by using SLM, we not only get the advantage of low peak power, but also the need to calculate optimized threshold is eliminated, thereby reducing the additional computation.
△ Less
Submitted 17 July, 2018;
originally announced July 2018.
