Learning Approach to Efficient Vision-based Active Tracking of a Flying Target by an Unmanned Aerial Vehicle
Authors:
Jagadeswara PKV Pothuri,
Aditya Bhatt,
Prajit KrisshnaKumar,
Manaswin Oddiraju,
Souma Chowdhury
Abstract:
Autonomous tracking of flying aerial objects has important civilian and defense applications, ranging from search and rescue to counter-unmanned aerial systems (counter-UAS). Ground based tracking requires setting up infrastructure, could be range limited, and may not be feasible in remote areas, crowded cities or in dense vegetation areas. Vision based active tracking of aerial objects from anoth…
▽ More
Autonomous tracking of flying aerial objects has important civilian and defense applications, ranging from search and rescue to counter-unmanned aerial systems (counter-UAS). Ground based tracking requires setting up infrastructure, could be range limited, and may not be feasible in remote areas, crowded cities or in dense vegetation areas. Vision based active tracking of aerial objects from another airborne vehicle, e.g., a chaser unmanned aerial vehicle (UAV), promises to fill this important gap, along with serving aerial coordination use cases. Vision-based active tracking by a UAV entails solving two coupled problems: 1) compute-efficient and accurate (target) object detection and target state estimation; and 2) maneuver decisions to ensure that the target remains in the field of view in the future time-steps and favorably positioned for continued detection. As a solution to the first problem, this paper presents a novel integration of standard deep learning based architectures with Kernelized Correlation Filter (KCF) to achieve compute-efficient object detection without compromising accuracy, unlike standalone learning or filtering approaches. The proposed perception framework is validated using a lab-scale setup. For the second problem, to obviate the linearity assumptions and background variations limiting effectiveness of the traditional controllers, we present the use of reinforcement learning to train a neuro-controller for fast computation of velocity maneuvers. New state space, action space and reward formulations are developed for this purpose, and training is performed in simulation using AirSim. The trained model is also tested in AirSim with respect to complex target maneuvers, and is found to outperform a baseline PID control in terms of tracking up-time and average distance maintained (from the target) during tracking.
△ Less
Submitted 22 June, 2025;
originally announced June 2025.
An Open-source Hardware/Software Architecture and Supporting Simulation Environment to Perform Human FPV Flight Demonstrations for Unmanned Aerial Vehicle Autonomy
Authors:
Haosong Xiao,
Prajit Krisshnakumar,
Jagadeswara P K V Pothuri,
Puru Soni,
Eric Butcher,
Souma Chowdhury
Abstract:
Small multi-rotor unmanned aerial vehicles (UAVs), mainly quadcopters, are nowadays ubiquitous in research on aerial autonomy, including serving as scaled-down models for much larger aircraft such as vertical-take-off-and-lift vehicles for urban air mobility. Among the various research use cases, first-person-view RC flight experiments allow for collecting data on how human pilots fly such aircraf…
▽ More
Small multi-rotor unmanned aerial vehicles (UAVs), mainly quadcopters, are nowadays ubiquitous in research on aerial autonomy, including serving as scaled-down models for much larger aircraft such as vertical-take-off-and-lift vehicles for urban air mobility. Among the various research use cases, first-person-view RC flight experiments allow for collecting data on how human pilots fly such aircraft, which could then be used to compare, contrast, validate, or train autonomous flight agents. While this could be uniquely beneficial, especially for studying UAV operation in contextually complex and safety-critical environments such as in human-UAV shared spaces, the lack of inexpensive and open-source hardware/software platforms that offer this capability along with low-level access to the underlying control software and data remains limited. To address this gap and significantly reduce barriers to human-guided autonomy research with UAVs, this paper presents an open-source software architecture implemented with an inexpensive in-house built quadcopter platform based on the F450 Quadcopter Frame. This setup uses two cameras to provide a dual-view FPV and an open-source flight controller, Pixhawk. The underlying software architecture, developed using the Python-based Kivy library, allows logging telemetry, GPS, control inputs, and camera frame data in a synchronized manner on the ground station computer. Since costs (time) and weather constraints typically limit numbers of physical outdoor flight experiments, this paper also presents a unique AirSim/Unreal Engine based simulation environment and graphical user interface aka digital twin, that provides a Hardware In The Loop setup via the Pixhawk flight controller. We demonstrate the usability and reliability of the overall framework through a set of diverse physical FPV flight experiments and corresponding flight tests in the digital twin.
△ Less
Submitted 5 September, 2024; v1 submitted 8 July, 2024;
originally announced July 2024.
