-
Efficient FPGA-accelerated Convolutional Neural Networks for Cloud Detection on CubeSats
Authors:
Angela Cratere,
M. Salim Farissi,
Andrea Carbone,
Marcello Asciolla,
Maria Rizzi,
Francesco Dell'Olio,
Augusto Nascetti,
Dario Spiller
Abstract:
We present the implementation of four FPGA-accelerated convolutional neural network (CNN) models for onboard cloud detection in resource-constrained CubeSat missions, leveraging Xilinx's Vitis AI (VAI) framework and Deep Learning Processing Unit (DPU), a programmable engine with pre-implemented, parameterizable IP cores optimized for deep neural networks, on a Zynq UltraScale+ MPSoC. This study ex…
▽ More
We present the implementation of four FPGA-accelerated convolutional neural network (CNN) models for onboard cloud detection in resource-constrained CubeSat missions, leveraging Xilinx's Vitis AI (VAI) framework and Deep Learning Processing Unit (DPU), a programmable engine with pre-implemented, parameterizable IP cores optimized for deep neural networks, on a Zynq UltraScale+ MPSoC. This study explores both pixel-wise (Pixel-Net and Patch-Net) and image-wise (U-Net and Scene-Net) models to benchmark trade-offs in accuracy, latency, and model complexity. Applying channel pruning, we achieved substantial reductions in model parameters (up to 98.6%) and floating-point operations (up to 90.7%) with minimal accuracy loss. Furthermore, the VAI tool was used to quantize the models to 8-bit precision, ensuring optimized hardware performance with negligible impact on accuracy. All models retained high accuracy post-FPGA integration, with a cumulative maximum accuracy drop of only 0.6% after quantization and pruning. The image-wise Scene-Net and U-Net models demonstrated strong real-time inference capabilities, achieving frame rates per second of 57.14 and 37.45, respectively, with power consumption of around 2.5 W, surpassing state-of-the-art onboard cloud detection solutions. Our approach underscores the potential of DPU-based hardware accelerators to expand the processing capabilities of small satellites, enabling efficient and flexible onboard CNN-based applications.
△ Less
Submitted 4 April, 2025;
originally announced April 2025.
-
Motion Estimation and Compensation in Automotive MIMO SAR
Authors:
Marco Manzoni,
Dario Tagliaferri,
Marco Rizzi,
Stefano Tebaldini,
Andrea Virgilio Monti-Guarnieri,
Claudio Maria Prati,
Monica Nicoli,
Ivan Russo,
Sergi Duque,
Christian Mazzucco,
Umberto Spagnolini
Abstract:
With the advent of self-driving vehicles, autonomous driving systems will have to rely on a vast number of heterogeneous sensors to perform dynamic perception of the surrounding environment. Synthetic Aperture Radar (SAR) systems increase the resolution of conventional mass-market radars by exploiting the vehicle's ego-motion, requiring a very accurate knowledge of the trajectory, usually not comp…
▽ More
With the advent of self-driving vehicles, autonomous driving systems will have to rely on a vast number of heterogeneous sensors to perform dynamic perception of the surrounding environment. Synthetic Aperture Radar (SAR) systems increase the resolution of conventional mass-market radars by exploiting the vehicle's ego-motion, requiring a very accurate knowledge of the trajectory, usually not compatible with automotive-grade navigation systems. In this regard, this paper deals with the analysis, estimation and compensation of trajectory estimation errors in automotive SAR systems, proposing a complete residual motion estimation and compensation workflow. We start by defining the geometry of the acquisition and the basic processing steps of Multiple-Input Multiple-Output (MIMO) SAR systems. Then, we analytically derive the effects of typical motion errors in automotive SAR imaging. Based on the derived models, the procedure is detailed, outlining the guidelines for its practical implementation. We show the effectiveness of the proposed technique by means of experimental data gathered by a 77 GHz radar mounted in a forward looking configuration.
△ Less
Submitted 25 January, 2022;
originally announced January 2022.
-
Residual Motion Compensation in Automotive MIMO SAR Imaging
Authors:
Marco Manzoni,
Marco Rizzi,
Stefano Tebaldini,
Andrea Virgilio Monti-Guarnieri,
Claudio Maria Prati,
Dario Tagliaferri,
Monica Nicoli,
Ivan Russo,
Christian Mazzucco,
Sergi Duque Biarge,
Umberto Spagnolini
Abstract:
This paper deals with the analysis, estimation, and compensation of trajectory errors in automotive-based Synthetic Aperture Radar (SAR) systems. First of all, we define the geometry of the acquisition and the model of the received signal. We then proceed by analytically evaluating the effect of an error in the vehicle's trajectory. Based on the derived model, we introduce a motion compensation (M…
▽ More
This paper deals with the analysis, estimation, and compensation of trajectory errors in automotive-based Synthetic Aperture Radar (SAR) systems. First of all, we define the geometry of the acquisition and the model of the received signal. We then proceed by analytically evaluating the effect of an error in the vehicle's trajectory. Based on the derived model, we introduce a motion compensation (MoCo) procedure capable of estimating and compensating constant velocity motion errors leading to a well-focused and well-localized SAR image. The procedure is validated using real data gathered by a 77 GHz automotive SAR with MIMO capabilities.
△ Less
Submitted 28 October, 2021;
originally announced October 2021.
-
Multi-Beam Automotive SAR Imaging in Urban Scenarios
Authors:
Marco Rizzi,
Marco Manzoni,
Stefano Tebaldini,
Andrea Virgilio Monti-Guarnieri,
Claudio Maria Prati,
Dario Tagliaferri,
Monica Nicoli,
Ivan Russo,
Christian Mazzucco,
Simón Tejero Alfageme,
Umberto Spagnolini
Abstract:
Automotive synthetic aperture radar (SAR) systems are rapidly emerging as a candidate technological solution to enable a high-resolution environment mapping for autonomous driving. Compared to lidars and cameras, automotive-legacy radars can work in any weather condition and without an external source of illumination, but are limited in either range or angular resolution. SARs offer a relevant inc…
▽ More
Automotive synthetic aperture radar (SAR) systems are rapidly emerging as a candidate technological solution to enable a high-resolution environment mapping for autonomous driving. Compared to lidars and cameras, automotive-legacy radars can work in any weather condition and without an external source of illumination, but are limited in either range or angular resolution. SARs offer a relevant increase in angular resolution, provided that the ego-motion of the radar platform is known along the synthetic aperture. In this paper, we present the results of an experimental campaign aimed at assessing the potential of a multi-beam SAR imaging in an urban scenario, composed of various targets (buildings, cars, pedestrian, etc.), employing a 77 GHz multiple-input multiple-output (MIMO) radar platform based on a mass-market available automotive-grade technology. The results highlight a centimeter-level accuracy of the SAR images in realistic driving conditions, showing the possibility to use a multi-angle focusing approach to detect and discriminate between different targets based on their angular scattering response.
△ Less
Submitted 28 October, 2021;
originally announced October 2021.
-
A self-supervised learning strategy for postoperative brain cavity segmentation simulating resections
Authors:
Fernando Pérez-García,
Reuben Dorent,
Michele Rizzi,
Francesco Cardinale,
Valerio Frazzini,
Vincent Navarro,
Caroline Essert,
Irène Ollivier,
Tom Vercauteren,
Rachel Sparks,
John S. Duncan,
Sébastien Ourselin
Abstract:
Accurate segmentation of brain resection cavities (RCs) aids in postoperative analysis and determining follow-up treatment. Convolutional neural networks (CNNs) are the state-of-the-art image segmentation technique, but require large annotated datasets for training. Annotation of 3D medical images is time-consuming, requires highly-trained raters, and may suffer from high inter-rater variability.…
▽ More
Accurate segmentation of brain resection cavities (RCs) aids in postoperative analysis and determining follow-up treatment. Convolutional neural networks (CNNs) are the state-of-the-art image segmentation technique, but require large annotated datasets for training. Annotation of 3D medical images is time-consuming, requires highly-trained raters, and may suffer from high inter-rater variability. Self-supervised learning strategies can leverage unlabeled data for training.
We developed an algorithm to simulate resections from preoperative magnetic resonance images (MRIs). We performed self-supervised training of a 3D CNN for RC segmentation using our simulation method. We curated EPISURG, a dataset comprising 430 postoperative and 268 preoperative MRIs from 430 refractory epilepsy patients who underwent resective neurosurgery. We fine-tuned our model on three small annotated datasets from different institutions and on the annotated images in EPISURG, comprising 20, 33, 19 and 133 subjects.
The model trained on data with simulated resections obtained median (interquartile range) Dice score coefficients (DSCs) of 81.7 (16.4), 82.4 (36.4), 74.9 (24.2) and 80.5 (18.7) for each of the four datasets. After fine-tuning, DSCs were 89.2 (13.3), 84.1 (19.8), 80.2 (20.1) and 85.2 (10.8). For comparison, inter-rater agreement between human annotators from our previous study was 84.0 (9.9).
We present a self-supervised learning strategy for 3D CNNs using simulated RCs to accurately segment real RCs on postoperative MRI. Our method generalizes well to data from different institutions, pathologies and modalities. Source code, segmentation models and the EPISURG dataset are available at https://github.com/fepegar/ressegijcars .
△ Less
Submitted 24 May, 2021;
originally announced May 2021.
