Showing 1–15 of 15 results for author: De Masi, G

snnTrans-DHZ: A Lightweight Spiking Neural Network Architecture for Underwater Image Dehazing

Authors: Vidya Sudevan, Fakhreddine Zayer, Rizwana Kausar, Sajid Javed, Hamad Karki, Giulia De Masi, Jorge Dias

Abstract: Underwater image dehazing is critical for vision-based marine operations because light scattering and absorption can severely reduce visibility. This paper introduces snnTrans-DHZ, a lightweight Spiking Neural Network (SNN) specifically designed for underwater dehazing. By leveraging the temporal dynamics of SNNs, snnTrans-DHZ efficiently processes time-dependent raw image sequences while maintain… ▽ More Underwater image dehazing is critical for vision-based marine operations because light scattering and absorption can severely reduce visibility. This paper introduces snnTrans-DHZ, a lightweight Spiking Neural Network (SNN) specifically designed for underwater dehazing. By leveraging the temporal dynamics of SNNs, snnTrans-DHZ efficiently processes time-dependent raw image sequences while maintaining low power consumption. Static underwater images are first converted into time-dependent sequences by repeatedly inputting the same image over user-defined timesteps. These RGB sequences are then transformed into LAB color space representations and processed concurrently. The architecture features three key modules: (i) a K estimator that extracts features from multiple color space representations; (ii) a Background Light Estimator that jointly infers the background light component from the RGB-LAB images; and (iii) a soft image reconstruction module that produces haze-free, visibility-enhanced outputs. The snnTrans-DHZ model is directly trained using a surrogate gradient-based backpropagation through time (BPTT) strategy alongside a novel combined loss function. Evaluated on the UIEB benchmark, snnTrans-DHZ achieves a PSNR of 21.68 dB and an SSIM of 0.8795, and on the EUVP dataset, it yields a PSNR of 23.46 dB and an SSIM of 0.8439. With only 0.5670 million network parameters, and requiring just 7.42 GSOPs and 0.0151 J of energy, the algorithm significantly outperforms existing state-of-the-art methods in terms of efficiency. These features make snnTrans-DHZ highly suitable for deployment in underwater robotics, marine exploration, and environmental monitoring. △ Less

Submitted 13 April, 2025; originally announced April 2025.

Underwater Image Enhancement by Convolutional Spiking Neural Networks

Authors: Vidya Sudevan, Fakhreddine Zayer, Rizwana Kausar, Sajid Javed, Hamad Karki, Giulia De Masi, Jorge Dias

Abstract: Underwater image enhancement (UIE) is fundamental for marine applications, including autonomous vision-based navigation. Deep learning methods using convolutional neural networks (CNN) and vision transformers advanced UIE performance. Recently, spiking neural networks (SNN) have gained attention for their lightweight design, energy efficiency, and scalability. This paper introduces UIE-SNN, the fi… ▽ More Underwater image enhancement (UIE) is fundamental for marine applications, including autonomous vision-based navigation. Deep learning methods using convolutional neural networks (CNN) and vision transformers advanced UIE performance. Recently, spiking neural networks (SNN) have gained attention for their lightweight design, energy efficiency, and scalability. This paper introduces UIE-SNN, the first SNN-based UIE algorithm to improve visibility of underwater images. UIE-SNN is a 19- layered convolutional spiking encoder-decoder framework with skip connections, directly trained using surrogate gradient-based backpropagation through time (BPTT) strategy. We explore and validate the influence of training datasets on energy reduction, a unique advantage of UIE-SNN architecture, in contrast to the conventional learning-based architectures, where energy consumption is model-dependent. UIE-SNN optimizes the loss function in latent space representation to reconstruct clear underwater images. Our algorithm performs on par with its non-spiking counterpart methods in terms of PSNR and structural similarity index (SSIM) at reduced timesteps ($T=5$) and energy consumption of $85\%$. The algorithm is trained on two publicly available benchmark datasets, UIEB and EUVP, and tested on unseen images from UIEB, EUVP, LSUI, U45, and our custom UIE dataset. The UIE-SNN algorithm achieves PSNR of \(17.7801~dB\) and SSIM of \(0.7454\) on UIEB, and PSNR of \(23.1725~dB\) and SSIM of \(0.7890\) on EUVP. UIE-SNN achieves this algorithmic performance with fewer operators (\(147.49\) GSOPs) and energy (\(0.1327~J\)) compared to its non-spiking counterpart (GFLOPs = \(218.88\) and Energy=\(1.0068~J\)). Compared with existing SOTA UIE methods, UIE-SNN achieves an average of \(6.5\times\) improvement in energy efficiency. The source code is available at \href{https://github.com/vidya-rejul/UIE-SNN.git}{UIE-SNN}. △ Less

Submitted 26 March, 2025; originally announced March 2025.

Boundary Control Behaviors of Multiple Low-cost AUVs Using Acoustic Communication

Authors: Mohammed Tarnini, Saverio Iacoponi, Andrea Infanti, Cesare Stefanini, Giulia De Masi, Federico Renda

Abstract: This study presents acoustic-based methods for the control of multiple autonomous underwater vehicles (AUV). This study proposes two different models for implementing boundary and path control on low-cost AUVs using acoustic communication and a single central acoustic beacon. Two methods are presented: the Range Variation-Based (RVB) model completely relies on range data obtained by acoustic modem… ▽ More This study presents acoustic-based methods for the control of multiple autonomous underwater vehicles (AUV). This study proposes two different models for implementing boundary and path control on low-cost AUVs using acoustic communication and a single central acoustic beacon. Two methods are presented: the Range Variation-Based (RVB) model completely relies on range data obtained by acoustic modems, whereas the Heading Estimation-Based (HEB) model uses ranges and range rates to estimate the position of the central boundary beacon and perform assigned behaviors. The models are tested on two boundary control behaviors: Fencing and Milling. Fencing behavior ensures AUVs return within predefined boundaries, while Milling enables the AUVs to move cyclically on a predefined path around the beacon. Models are validated by successfully performing the boundary control behaviors in simulations, pool tests, including artificial underwater currents, and field tests conducted in the ocean. All tests were performed with fully autonomous platforms, and no external input or sensor was provided to the AUVs during validation. Quantitative and qualitative analyses are presented in the study, focusing on the effect and application of a multi-robot system. △ Less

Submitted 28 November, 2024; originally announced December 2024.

Comments: Submitted to Journal of Field Robotics

Journal ref: IEEE Access 13 2025 72506-72525

Dehazing-aided Multi-Rate Multi-Modal Pose Estimation Framework for Mitigating Visual Disturbances in Extreme Underwater Domain

Authors: Vidya Sudevan, Fakhreddine Zayer, Taimur Hassan, Sajid Javed, Hamad Karki, Giulia De Masi, Jorge Dias

Abstract: This paper delves into the potential of DU-VIO, a dehazing-aided hybrid multi-rate multi-modal Visual-Inertial Odometry (VIO) estimation framework, designed to thrive in the challenging realm of extreme underwater environments. The cutting-edge DU-VIO framework is incorporating a GAN-based pre-processing module and a hybrid CNN-LSTM module for precise pose estimation, using visibility-enhanced und… ▽ More This paper delves into the potential of DU-VIO, a dehazing-aided hybrid multi-rate multi-modal Visual-Inertial Odometry (VIO) estimation framework, designed to thrive in the challenging realm of extreme underwater environments. The cutting-edge DU-VIO framework is incorporating a GAN-based pre-processing module and a hybrid CNN-LSTM module for precise pose estimation, using visibility-enhanced underwater images and raw IMU data. Accurate pose estimation is paramount for various underwater robotics and exploration applications. However, underwater visibility is often compromised by suspended particles and attenuation effects, rendering visual-inertial pose estimation a formidable challenge. DU-VIO aims to overcome these limitations by effectively removing visual disturbances from raw image data, enhancing the quality of image features used for pose estimation. We demonstrate the effectiveness of DU-VIO by calculating RMSE scores for translation and rotation vectors in comparison to their reference values. These scores are then compared to those of a base model using a modified AQUALOC Dataset. This study's significance lies in its potential to revolutionize underwater robotics and exploration. DU-VIO offers a robust solution to the persistent challenge of underwater visibility, significantly improving the accuracy of pose estimation. This research contributes valuable insights and tools for advancing underwater technology, with far-reaching implications for scientific research, environmental monitoring, and industrial applications. △ Less

Submitted 21 November, 2024; originally announced November 2024.

Hybrid-Neuromorphic Approach for Underwater Robotics Applications: A Conceptual Framework

Authors: Vidya Sudevan, Fakhreddine Zayer, Sajid Javed, Hamad Karki, Giulia De Masi, Jorge Dias

Abstract: This paper introduces the concept of employing neuromorphic methodologies for task-oriented underwater robotics applications. In contrast to the increasing computational demands of conventional deep learning algorithms, neuromorphic technology, leveraging spiking neural network architectures, promises sophisticated artificial intelligence with significantly reduced computational requirements and p… ▽ More This paper introduces the concept of employing neuromorphic methodologies for task-oriented underwater robotics applications. In contrast to the increasing computational demands of conventional deep learning algorithms, neuromorphic technology, leveraging spiking neural network architectures, promises sophisticated artificial intelligence with significantly reduced computational requirements and power consumption, emulating human brain operational principles. Despite documented neuromorphic technology applications in various robotic domains, its utilization in marine robotics remains largely unexplored. Thus, this article proposes a unified framework for integrating neuromorphic technologies for perception, pose estimation, and haptic-guided conditional control of underwater vehicles, customized to specific user-defined objectives. This conceptual framework stands to revolutionize underwater robotics, enhancing efficiency and autonomy while reducing energy consumption. By enabling greater adaptability and robustness, this advancement could facilitate applications such as underwater exploration, environmental monitoring, and infrastructure maintenance, thereby contributing to significant progress in marine science and technology. △ Less

Submitted 21 November, 2024; originally announced November 2024.

Wireless teleoperation of HSURF artificial fish in complex paths

Authors: Saverio Iacoponi, Nikita Mankovskii, Mohammed El Hanbaly, Andrea Infanti, Shamma Alhajeri, Federico Renda, Cesare Stefanini, Giulia De Masi

Abstract: In this paper we show the application of the new robotic multi-platform system HSURF to a specific use case of teleoperation, aimed at monitoring and inspection. The HSURF system, consists of 3 different kinds of platforms: floater, sinker and robotic fishes. The collaborative control of the 3 platforms allows a remotely based operator to control the fish in order to visit and inspect several targ… ▽ More In this paper we show the application of the new robotic multi-platform system HSURF to a specific use case of teleoperation, aimed at monitoring and inspection. The HSURF system, consists of 3 different kinds of platforms: floater, sinker and robotic fishes. The collaborative control of the 3 platforms allows a remotely based operator to control the fish in order to visit and inspect several targets underwater following a complex trajectory. A shared autonomy solution shows to be the most suitable, in order to minimize the effect of limited bandwidth and relevant delay intrinsic to acoustic communications. The control architecture is described and preliminary results of the acoustically teleoperated visits of multiple targets in a testing pool are provided. △ Less

Submitted 6 July, 2024; originally announced July 2024.

Journal ref: OCEANS 2023 - Limerick, Ireland, 2023, pp. 1-5

Underwater Human-Robot and Human-Swarm Interaction: A Review and Perspective

Authors: Sara Aldhaheri, Federico Renda, Giulia De Masi

Abstract: There has been a growing interest in extending the capabilities of autonomous underwater vehicles (AUVs) in subsea missions, particularly in integrating underwater human-robot interaction (UHRI) for control. UHRI and its subfield,underwater gesture recognition (UGR), play a significant role in enhancing diver-robot communication for marine research. This review explores the latest developments in… ▽ More There has been a growing interest in extending the capabilities of autonomous underwater vehicles (AUVs) in subsea missions, particularly in integrating underwater human-robot interaction (UHRI) for control. UHRI and its subfield,underwater gesture recognition (UGR), play a significant role in enhancing diver-robot communication for marine research. This review explores the latest developments in UHRI and examines its promising applications for multi-robot systems. With the developments in UGR, opportunities are presented for underwater robots to work alongside human divers to increase their functionality. Human gestures creates a seamless and safe collaborative environment where divers and robots can interact more efficiently. By highlighting the state-of-the-art in this field, we can potentially encourage advancements in underwater multi-robot system (UMRS) blending the natural communication channels of human-robot interaction with the multi-faceted coordination capabilities of underwater swarms,thus enhancing robustness in complex aquatic environments. △ Less

Submitted 21 June, 2024; v1 submitted 18 June, 2024; originally announced June 2024.

Comments: OCEANS 2024-Singapore, 1-9

Dynamic Spiking Framework for Graph Neural Networks

Authors: Nan Yin, Mengzhu Wang, Zhenghan Chen, Giulia De Masi, Bin Gu, Huan Xiong

Abstract: The integration of Spiking Neural Networks (SNNs) and Graph Neural Networks (GNNs) is gradually attracting attention due to the low power consumption and high efficiency in processing the non-Euclidean data represented by graphs. However, as a common problem, dynamic graph representation learning faces challenges such as high complexity and large memory overheads. Current work often uses SNNs inst… ▽ More The integration of Spiking Neural Networks (SNNs) and Graph Neural Networks (GNNs) is gradually attracting attention due to the low power consumption and high efficiency in processing the non-Euclidean data represented by graphs. However, as a common problem, dynamic graph representation learning faces challenges such as high complexity and large memory overheads. Current work often uses SNNs instead of Recurrent Neural Networks (RNNs) by using binary features instead of continuous ones for efficient training, which would overlooks graph structure information and leads to the loss of details during propagation. Additionally, optimizing dynamic spiking models typically requires propagation of information across time steps, which increases memory requirements. To address these challenges, we present a framework named \underline{Dy}namic \underline{S}p\underline{i}king \underline{G}raph \underline{N}eural Networks (\method{}). To mitigate the information loss problem, \method{} propagates early-layer information directly to the last layer for information compensation. To accommodate the memory requirements, we apply the implicit differentiation on the equilibrium state, which does not rely on the exact reverse of the forward computation. While traditional implicit differentiation methods are usually used for static situations, \method{} extends it to the dynamic graph setting. Extensive experiments on three large-scale real-world dynamic graph datasets validate the effectiveness of \method{} on dynamic node classification tasks with lower computational costs. △ Less

Submitted 30 July, 2024; v1 submitted 15 December, 2023; originally announced January 2024.

Energy Efficient Training of SNN using Local Zeroth Order Method

Authors: Bhaskar Mukhoty, Velibor Bojkovic, William de Vazelhes, Giulia De Masi, Huan Xiong, Bin Gu

Abstract: Spiking neural networks are becoming increasingly popular for their low energy requirement in real-world tasks with accuracy comparable to the traditional ANNs. SNN training algorithms face the loss of gradient information and non-differentiability due to the Heaviside function in minimizing the model loss over model parameters. To circumvent the problem surrogate method uses a differentiable appr… ▽ More Spiking neural networks are becoming increasingly popular for their low energy requirement in real-world tasks with accuracy comparable to the traditional ANNs. SNN training algorithms face the loss of gradient information and non-differentiability due to the Heaviside function in minimizing the model loss over model parameters. To circumvent the problem surrogate method uses a differentiable approximation of the Heaviside in the backward pass, while the forward pass uses the Heaviside as the spiking function. We propose to use the zeroth order technique at the neuron level to resolve this dichotomy and use it within the automatic differentiation tool. As a result, we establish a theoretical connection between the proposed local zeroth-order technique and the existing surrogate methods and vice-versa. The proposed method naturally lends itself to energy-efficient training of SNNs on GPUs. Experimental results with neuromorphic datasets show that such implementation requires less than 1 percent neurons to be active in the backward pass, resulting in a 100x speed-up in the backward computation time. Our method offers better generalization compared to the state-of-the-art energy-efficient technique while maintaining similar efficiency. △ Less

Submitted 5 February, 2023; v1 submitted 2 February, 2023; originally announced February 2023.

Cross-inhibition leads to group consensus despite the presence of strongly opinionated minorities and asocial behaviour

Authors: Andreagiovanni Reina, Raina Zakir, Giulia De Masi, Eliseo Ferrante

Abstract: Strongly opinionated minorities can have a dramatic impact on the opinion dynamics of a large population. Two factions of inflexible minorities, polarised into two competing opinions, could lead the entire population to persistent indecision. Equivalently, populations can remain undecided when individuals sporadically change their opinion based on individual information rather than social informat… ▽ More Strongly opinionated minorities can have a dramatic impact on the opinion dynamics of a large population. Two factions of inflexible minorities, polarised into two competing opinions, could lead the entire population to persistent indecision. Equivalently, populations can remain undecided when individuals sporadically change their opinion based on individual information rather than social information. Our analysis compares the cross-inhibition model with the voter model for decisions between equally good alternatives, and with the weighted voter model for decisions among alternatives characterised by different qualities. Here we show that cross-inhibition, differently from the other two models, is a simple mechanism, ubiquitous in collective biological systems, that allows the population to reach a stable majority for one alternative even in the presence of asocial behaviour. The results predicted by the mean-field models are confirmed by experiments with swarms of 100 locally interacting robots. This work suggests an answer to the longstanding question of why inhibitory signals are widespread in natural systems of collective decision making, and, at the same time, it proposes an efficient mechanism for designing resilient swarms of minimalistic robots. △ Less

Submitted 25 July, 2023; v1 submitted 17 November, 2022; originally announced November 2022.

Underwater Image Enhancement Using Pre-trained Transformer

Authors: Abderrahmene Boudiaf, Yuhang Guo, Adarsh Ghimire, Naoufel Werghi, Giulia De Masi, Sajid Javed, Jorge Dias

Abstract: The goal of this work is to apply a denoising image transformer to remove the distortion from underwater images and compare it with other similar approaches. Automatic restoration of underwater images plays an important role since it allows to increase the quality of the images, without the need for more expensive equipment. This is a critical example of the important role of the machine learning… ▽ More The goal of this work is to apply a denoising image transformer to remove the distortion from underwater images and compare it with other similar approaches. Automatic restoration of underwater images plays an important role since it allows to increase the quality of the images, without the need for more expensive equipment. This is a critical example of the important role of the machine learning algorithms to support marine exploration and monitoring, reducing the need for human intervention like the manual processing of the images, thus saving time, effort, and cost. This paper is the first application of the image transformer-based approach called "Pre-Trained Image Processing Transformer" to underwater images. This approach is tested on the UFO-120 dataset, containing 1500 images with the corresponding clean images. △ Less

Submitted 8 April, 2022; originally announced April 2022.

Design of magnetic coupling-based anti-biofouling mechanism for underwater optical sensors

Authors: Jane Pauline Ramirez, Cesare Stefanini, Giulia De Masi, Donato Romano

Abstract: Water monitoring is crucial for environmental monitoring, transportation, energy and telecommunication. One of the main problems in aquatic environmental monitoring is biofouling. The simplest method among the current antifouling strategies is the use of wiper technologies like brushes and wipers which apply mechanical pressure. In designing built-in strategies however, manufacturers usually build… ▽ More Water monitoring is crucial for environmental monitoring, transportation, energy and telecommunication. One of the main problems in aquatic environmental monitoring is biofouling. The simplest method among the current antifouling strategies is the use of wiper technologies like brushes and wipers which apply mechanical pressure. In designing built-in strategies however, manufacturers usually build the sensor around the biofouling system. The current state-of-the-art is a fully integrated central wiper in the sensor that enables cleaning of all probes mounted on the sonde. Improvements in antifouling strategies lag rapid advancements in sensor technologies such as in miniaturization, specialization, and costs. Hence, improving built-in designs by decreasing size and complexity will decrease maintenance and overall costs. This design is targeted for the EU project Robocoenosis since bio-hybrid systems in this project incorporate living organisms. This technology targets selective proliferation of the organisms which only prevents biofilms on components where they are unwanted. Beyond this, the use of autonomous activation based on image processing may likely be advantageous for minimizing the need for human inspection and maintenance. In addition to Robocoenosis, we also aim at incorporating this design in another project entitled Heterogeneous Swarm of Underwater Autonomous Vehicles where a swarm of heterogeneous underwater robotic fish is being developed. △ Less

Submitted 28 January, 2022; originally announced January 2022.

Comments: 6 pages, 10 figures

Nukhada USV: a Robot for Autonomous Surveying and Support to Underwater Operations

Authors: Èric Pairet, Simone Spanò, Nikita Mankovskii, Paolo Pellegrino, Igor Zhilin, Jeremy Nicola, Francesco La Gala, Giulia De Masi

Abstract: The Technology Innovation Institute in Abu Dhabi, United Arab Emirates, has recently finished the production and testing of a new unmanned surface vehicle, called Nukhada, specifically designed for autonomous survey, inspection, and support to underwater operations. This manuscript describes the main characteristics of the Nukhada USV, as well as some of the trials conducted during the development… ▽ More The Technology Innovation Institute in Abu Dhabi, United Arab Emirates, has recently finished the production and testing of a new unmanned surface vehicle, called Nukhada, specifically designed for autonomous survey, inspection, and support to underwater operations. This manuscript describes the main characteristics of the Nukhada USV, as well as some of the trials conducted during the development. △ Less

Submitted 10 January, 2022; originally announced January 2022.

Comments: OCEANS 2022 - Chennai

Journal ref: OCEANS 2022 - Chennai, 2022

Underwater Robot Manipulation: Advances, Challenges and Prospective Ventures

Authors: Sara Aldhaheri, Giulia De Masi, Èric Pairet, Paola Ardón

Abstract: Underwater manipulation is one of the most remarkable ongoing research subjects in robotics. \acp{I-AUV} not only have to cope with the technical challenges associated with traditional manipulation tasks but do so while currents and waves perturb the stability of the vehicle, and low-light, turbid water conditions impede perceiving the surroundings. Certainly, the dynamic nature and our limited un… ▽ More Underwater manipulation is one of the most remarkable ongoing research subjects in robotics. \acp{I-AUV} not only have to cope with the technical challenges associated with traditional manipulation tasks but do so while currents and waves perturb the stability of the vehicle, and low-light, turbid water conditions impede perceiving the surroundings. Certainly, the dynamic nature and our limited understanding of the marine environment hinder the autonomous performance of underwater robot manipulation. This manuscript provides a discussion on previous research and the limiting factors that impose on the long-envisioned prospects of autonomous underwater manipulation to finally highlight research directions that have the potential to improve the autonomy capabilities of I-AUV. △ Less

Submitted 9 January, 2022; originally announced January 2022.

Comments: Published in IEEE OCEANS 2022

Underwater Object Classification and Detection: first results and open challenges

Authors: Andre Jesus, Claudio Zito, Claudio Tortorici, Eloy Roura, Giulia De Masi

Abstract: This work reviews the problem of object detection in underwater environments. We analyse and quantify the shortcomings of conventional state-of-the-art (SOTA) algorithms in the computer vision community when applied to this challenging environment, as well as providing insights and general guidelines for future research efforts. First, we assessed if pretraining with the conventional ImageNet is b… ▽ More This work reviews the problem of object detection in underwater environments. We analyse and quantify the shortcomings of conventional state-of-the-art (SOTA) algorithms in the computer vision community when applied to this challenging environment, as well as providing insights and general guidelines for future research efforts. First, we assessed if pretraining with the conventional ImageNet is beneficial when the object detector needs to be applied to environments that may be characterised by a different feature distribution. We then investigate whether two-stage detectors yields to better performance with respect to single-stage detectors, in terms of accuracy, intersection of union (IoU), floating operation per second (FLOPS), and inference time. Finally, we assessed the generalisation capability of each model to a lower quality dataset to simulate performance on a real scenario, in which harsher conditions ought to be expected. Our experimental results provide evidence that underwater object detection requires searching for "ad-hoc" architectures than merely training SOTA architectures on new data, and that pretraining is not beneficial. △ Less

Submitted 3 January, 2022; originally announced January 2022.

Journal ref: In Proceedings of OCEANS 2022 Chennai, February 21-24, pp. 1-6