-
Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml
Authors:
Nicolò Ghielmetti,
Vladimir Loncar,
Maurizio Pierini,
Marcel Roed,
Sioni Summers,
Thea Aarrestad,
Christoffer Petersson,
Hampus Linander,
Jennifer Ngadiuba,
Kelvin Lin,
Philip Harris
Abstract:
In this paper, we investigate how field programmable gate arrays can serve as hardware accelerators for real-time semantic segmentation tasks relevant for autonomous driving. Considering compressed versions of the ENet convolutional neural network architecture, we demonstrate a fully-on-chip deployment with a latency of 4.9 ms per image, using less than 30% of the available resources on a Xilinx Z…
▽ More
In this paper, we investigate how field programmable gate arrays can serve as hardware accelerators for real-time semantic segmentation tasks relevant for autonomous driving. Considering compressed versions of the ENet convolutional neural network architecture, we demonstrate a fully-on-chip deployment with a latency of 4.9 ms per image, using less than 30% of the available resources on a Xilinx ZCU102 evaluation board. The latency is reduced to 3 ms per image when increasing the batch size to ten, corresponding to the use case where the autonomous vehicle receives inputs from multiple cameras simultaneously. We show, through aggressive filter reduction and heterogeneous quantization-aware training, and an optimized implementation of convolutional layers, that the power consumption and resource utilization can be significantly reduced while maintaining accuracy on the Cityscapes dataset.
△ Less
Submitted 16 May, 2022;
originally announced May 2022.
-
The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider
Authors:
T. Aarrestad,
M. van Beekveld,
M. Bona,
A. Boveia,
S. Caron,
J. Davies,
A. De Simone,
C. Doglioni,
J. M. Duarte,
A. Farbin,
H. Gupta,
L. Hendriks,
L. Heinrich,
J. Howarth,
P. Jawahar,
A. Jueid,
J. Lastow,
A. Leinweber,
J. Mamuzic,
E. Merényi,
A. Morandini,
P. Moskvitina,
C. Nellist,
J. Ngadiuba,
B. Ostdiek
, et al. (14 additional authors not shown)
Abstract:
We describe the outcome of a data challenge conducted as part of the Dark Machines Initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims at detecting signals of new physics at the LHC using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We defin…
▽ More
We describe the outcome of a data challenge conducted as part of the Dark Machines Initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims at detecting signals of new physics at the LHC using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 Billion simulated LHC events corresponding to $10~\rm{fb}^{-1}$ of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.
△ Less
Submitted 9 December, 2021; v1 submitted 28 May, 2021;
originally announced May 2021.
-
Fast convolutional neural networks on FPGAs with hls4ml
Authors:
Thea Aarrestad,
Vladimir Loncar,
Nicolò Ghielmetti,
Maurizio Pierini,
Sioni Summers,
Jennifer Ngadiuba,
Christoffer Petersson,
Hampus Linander,
Yutaro Iiyama,
Giuseppe Di Guglielmo,
Javier Duarte,
Philip Harris,
Dylan Rankin,
Sergo Jindariani,
Kevin Pedro,
Nhan Tran,
Mia Liu,
Edward Kreinar,
Zhenbin Wu,
Duc Hoang
Abstract:
We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on FPGAs. By extending the hls4ml library, we demonstrate an inference latency of $5\,μ$s using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Num…
▽ More
We introduce an automated tool for deploying ultra low-latency, low-power deep neural networks with convolutional layers on FPGAs. By extending the hls4ml library, we demonstrate an inference latency of $5\,μ$s using convolutional architectures, targeting microsecond latency applications like those at the CERN Large Hadron Collider. Considering benchmark models trained on the Street View House Numbers Dataset, we demonstrate various methods for model compression in order to fit the computational constraints of a typical FPGA device used in trigger and data acquisition systems of particle detectors. In particular, we discuss pruning and quantization-aware training, and demonstrate how resource utilization can be significantly reduced with little to no loss in model accuracy. We show that the FPGA critical resource consumption can be reduced by 97% with zero loss in model accuracy, and by 99% when tolerating a 6% accuracy degradation.
△ Less
Submitted 29 April, 2021; v1 submitted 13 January, 2021;
originally announced January 2021.
