How Real is CARLAs Dynamic Vision Sensor? A Study on the Sim-to-Real Gap in Traffic Object Detection
Authors:
Kaiyuan Tan,
Pavan Kumar B N,
Bharatesh Chakravarthi
Abstract:
Event cameras are gaining traction in traffic monitoring applications due to their low latency, high temporal resolution, and energy efficiency, which makes them well-suited for real-time object detection at traffic intersections. However, the development of robust event-based detection models is hindered by the limited availability of annotated real-world datasets. To address this, several simula…
▽ More
Event cameras are gaining traction in traffic monitoring applications due to their low latency, high temporal resolution, and energy efficiency, which makes them well-suited for real-time object detection at traffic intersections. However, the development of robust event-based detection models is hindered by the limited availability of annotated real-world datasets. To address this, several simulation tools have been developed to generate synthetic event data. Among these, the CARLA driving simulator includes a built-in dynamic vision sensor (DVS) module that emulates event camera output. Despite its potential, the sim-to-real gap for event-based object detection remains insufficiently studied. In this work, we present a systematic evaluation of this gap by training a recurrent vision transformer model exclusively on synthetic data generated using CARLAs DVS and testing it on varying combinations of synthetic and real-world event streams. Our experiments show that models trained solely on synthetic data perform well on synthetic-heavy test sets but suffer significant performance degradation as the proportion of real-world data increases. In contrast, models trained on real-world data demonstrate stronger generalization across domains. This study offers the first quantifiable analysis of the sim-to-real gap in event-based object detection using CARLAs DVS. Our findings highlight limitations in current DVS simulation fidelity and underscore the need for improved domain adaptation techniques in neuromorphic vision for traffic monitoring.
△ Less
Submitted 16 June, 2025;
originally announced June 2025.
A Comprehensive Review of Leap Motion Controller-based Hand Gesture Datasets
Authors:
Bharatesh Chakravarthi,
Prabhu Prasad B M,
Pavan Kumar B N
Abstract:
This paper comprehensively reviews hand gesture datasets based on Ultraleap's leap motion controller, a popular device for capturing and tracking hand gestures in real-time. The aim is to offer researchers and practitioners a valuable resource for developing and evaluating gesture recognition algorithms. The review compares various datasets found in the literature, considering factors such as targ…
▽ More
This paper comprehensively reviews hand gesture datasets based on Ultraleap's leap motion controller, a popular device for capturing and tracking hand gestures in real-time. The aim is to offer researchers and practitioners a valuable resource for developing and evaluating gesture recognition algorithms. The review compares various datasets found in the literature, considering factors such as target domain, dataset size, gesture diversity, subject numbers, and data modality. The strengths and limitations of each dataset are discussed, along with the applications and research areas in which they have been utilized. An experimental evaluation of the leap motion controller 2 device is conducted to assess its capabilities in generating gesture data for various applications, specifically focusing on touchless interactive systems and virtual reality. This review serves as a roadmap for researchers, aiding them in selecting appropriate datasets for their specific gesture recognition tasks and advancing the field of hand gesture recognition using leap motion controller technology.
△ Less
Submitted 7 November, 2023;
originally announced November 2023.
