Object Permanence in Object Detection Leveraging Temporal Priors at Inference Time
Authors:
Michael Fürst,
Priyash Bhugra,
René Schuster,
Didier Stricker
Abstract:
Object permanence is the concept that objects do not suddenly disappear in the physical world. Humans understand this concept at young ages and know that another person is still there, even though it is temporarily occluded. Neural networks currently often struggle with this challenge. Thus, we introduce explicit object permanence into two stage detection approaches drawing inspiration from partic…
▽ More
Object permanence is the concept that objects do not suddenly disappear in the physical world. Humans understand this concept at young ages and know that another person is still there, even though it is temporarily occluded. Neural networks currently often struggle with this challenge. Thus, we introduce explicit object permanence into two stage detection approaches drawing inspiration from particle filters. At the core, our detector uses the predictions of previous frames as additional proposals for the current one at inference time. Experiments confirm the feedback loop improving detection performance by a up to 10.3 mAP with little computational overhead.
Our approach is suited to extend two-stage detectors for stabilized and reliable detections even under heavy occlusion. Additionally, the ability to apply our method without retraining an existing model promises wide application in real-world tasks.
△ Less
Submitted 28 November, 2022;
originally announced November 2022.
RapidAI4EO: Mono- and Multi-temporal Deep Learning models for Updating the CORINE Land Cover Product
Authors:
Priyash Bhugra,
Benjamin Bischke,
Christoph Werner,
Robert Syrnicki,
Carolin Packbier,
Patrick Helber,
Caglar Senaras,
Akhil Singh Rana,
Tim Davis,
Wanda De Keersmaecker,
Daniele Zanaga,
Annett Wania,
Ruben Van De Kerchove,
Giovanni Marchisio
Abstract:
In the remote sensing community, Land Use Land Cover (LULC) classification with satellite imagery is a main focus of current research activities. Accurate and appropriate LULC classification, however, continues to be a challenging task. In this paper, we evaluate the performance of multi-temporal (monthly time series) compared to mono-temporal (single time step) satellite images for multi-label cl…
▽ More
In the remote sensing community, Land Use Land Cover (LULC) classification with satellite imagery is a main focus of current research activities. Accurate and appropriate LULC classification, however, continues to be a challenging task. In this paper, we evaluate the performance of multi-temporal (monthly time series) compared to mono-temporal (single time step) satellite images for multi-label classification using supervised learning on the RapidAI4EO dataset. As a first step, we trained our CNN model on images at a single time step for multi-label classification, i.e. mono-temporal. We incorporated time-series images using a LSTM model to assess whether or not multi-temporal signals from satellites improves CLC classification. The results demonstrate an improvement of approximately 0.89% in classifying satellite imagery on 15 classes using a multi-temporal approach on monthly time series images compared to the mono-temporal approach. Using features from multi-temporal or mono-temporal images, this work is a step towards an efficient change detection and land monitoring approach.
△ Less
Submitted 26 October, 2022;
originally announced October 2022.
