Benchmarking Vision Foundation Models for Input Monitoring in Autonomous Driving
Authors:
Mert Keser,
Halil Ibrahim Orhan,
Niki Amini-Naieni,
Gesina Schwalbe,
Alois Knoll,
Matthias Rottmann
Abstract:
Deep neural networks (DNNs) remain challenged by distribution shifts in complex open-world domains like automated driving (AD): Robustness against yet unknown novel objects (semantic shift) or styles like lighting conditions (covariate shift) cannot be guaranteed. Hence, reliable operation-time monitors for identification of out-of-training-data-distribution (OOD) scenarios are imperative. Current…
▽ More
Deep neural networks (DNNs) remain challenged by distribution shifts in complex open-world domains like automated driving (AD): Robustness against yet unknown novel objects (semantic shift) or styles like lighting conditions (covariate shift) cannot be guaranteed. Hence, reliable operation-time monitors for identification of out-of-training-data-distribution (OOD) scenarios are imperative. Current approaches for OOD classification are untested for complex domains like AD, are limited in the kinds of shifts they detect, or even require supervision with OOD samples. To prepare for unanticipated shifts, we instead establish a framework around a principled, unsupervised and model-agnostic method that unifies detection of semantic and covariate shifts: Find a full model of the training data's feature distribution, to then use its density at new points as in-distribution (ID) score. To implement this, we propose to combine Vision Foundation Models (VFMs) as feature extractors with density modeling techniques. Through a comprehensive benchmark of 4 VFMs with different backbone architectures and 5 density-modeling techniques against established baselines, we provide the first systematic evaluation of OOD classification capabilities of VFMs across diverse conditions. A comparison with state-of-the-art binary OOD classification methods reveals that VFM embeddings with density estimation outperform existing approaches in identifying OOD inputs. Additionally, we show that our method detects high-risk inputs likely to cause errors in downstream tasks, thereby improving overall performance. Overall, VFMs, when coupled with robust density modeling techniques, are promising to realize model-agnostic, unsupervised, reliable safety monitors in complex vision tasks
△ Less
Submitted 4 April, 2025; v1 submitted 14 January, 2025;
originally announced January 2025.
Analytical Investigation of Long-time Diffusion Dynamics in a Synaptic Channel with Glial Cells
Authors:
Enes Oncu,
Halil U. Ozdemir,
Halil I. Orhan,
Bayram Cevdet Akdeniz,
Asaf Toprakci,
Anil Aslihak,
H. Birkan Yilmaz,
Ali E. Pusane,
Tuna Tugcu,
Fatih Dinc
Abstract:
In this letter, we first derive the analytical channel impulse response for a cylindrical synaptic channel surrounded by glial cells and validate it with particle-based simulations. Afterwards, we provide an accurate analytical approximation for the long-time decay rate of the channel impulse response by employing Taylor expansion to the characteristic equations that determine the decay rates of t…
▽ More
In this letter, we first derive the analytical channel impulse response for a cylindrical synaptic channel surrounded by glial cells and validate it with particle-based simulations. Afterwards, we provide an accurate analytical approximation for the long-time decay rate of the channel impulse response by employing Taylor expansion to the characteristic equations that determine the decay rates of the system. We validate our approximation by comparing it with the numerical decay rate obtained from the characteristic equation. Overall, we provide a fully analytical description for the long-time behavior of synaptic diffusion, e.g., the clean-up processes inside the channel after communication has long concluded.
△ Less
Submitted 21 August, 2021;
originally announced September 2021.
