-
OpenDAS: Open-Vocabulary Domain Adaptation for 2D and 3D Segmentation
Authors:
Gonca Yilmaz,
Songyou Peng,
Marc Pollefeys,
Francis Engelmann,
Hermann Blum
Abstract:
Recently, Vision-Language Models (VLMs) have advanced segmentation techniques by shifting from the traditional segmentation of a closed-set of predefined object classes to open-vocabulary segmentation (OVS), allowing users to segment novel classes and concepts unseen during training of the segmentation model. However, this flexibility comes with a trade-off: fully-supervised closed-set methods sti…
▽ More
Recently, Vision-Language Models (VLMs) have advanced segmentation techniques by shifting from the traditional segmentation of a closed-set of predefined object classes to open-vocabulary segmentation (OVS), allowing users to segment novel classes and concepts unseen during training of the segmentation model. However, this flexibility comes with a trade-off: fully-supervised closed-set methods still outperform OVS methods on base classes, that is on classes on which they have been explicitly trained. This is due to the lack of pixel-aligned training masks for VLMs (which are trained on image-caption pairs), and the absence of domain-specific knowledge, such as autonomous driving. Therefore, we propose the task of open-vocabulary domain adaptation to infuse domain-specific knowledge into VLMs while preserving their open-vocabulary nature. By doing so, we achieve improved performance in base and novel classes. Existing VLM adaptation methods improve performance on base (training) queries, but fail to fully preserve the open-set capabilities of VLMs on novel queries. To address this shortcoming, we combine parameter-efficient prompt tuning with a triplet-loss-based training strategy that uses auxiliary negative queries. Notably, our approach is the only parameter-efficient method that consistently surpasses the original VLM on novel classes. Our adapted VLMs can seamlessly be integrated into existing OVS pipelines, e.g., improving OVSeg by +6.0% mIoU on ADE20K for open-vocabulary 2D segmentation, and OpenMask3D by +4.1% AP on ScanNet++ Offices for open-vocabulary 3D instance segmentation without other changes. The project page is available at https://open-das.github.io/.
△ Less
Submitted 29 October, 2024; v1 submitted 30 May, 2024;
originally announced May 2024.
-
Quasi-Newton FDE in One-Bit Pseudo-Randomly Quantized Massive MIMO-OFDM Systems
Authors:
Gökhan Yılmaz,
Ali Özgür Yılmaz
Abstract:
This letter offers a new frequency domain equalization (FDE) scheme that can work with a pseudo-random quantization (PRQ) scheme utilizing non-zero threshold quantization in one-bit uplink multi-user massive multiple-input multiple-output (MIMO) systems to mitigate quantization distortion and support high-order modulation schemes. The equalizer is based on Newton's method (NM) and applicable for o…
▽ More
This letter offers a new frequency domain equalization (FDE) scheme that can work with a pseudo-random quantization (PRQ) scheme utilizing non-zero threshold quantization in one-bit uplink multi-user massive multiple-input multiple-output (MIMO) systems to mitigate quantization distortion and support high-order modulation schemes. The equalizer is based on Newton's method (NM) and applicable for orthogonal frequency division multiplexing (OFDM) transmission under frequency-selective fading by exploiting the properties of massive MIMO. We develop a low-complexity FDE scheme to obtain a quasi-Newton method. The proposed detector outperforms the benchmark detector with comparable complexity.
△ Less
Submitted 23 December, 2023; v1 submitted 7 June, 2023;
originally announced June 2023.
-
Pseudo-Random Quantization Based Two-Stage Detection in One-Bit Massive MIMO Systems
Authors:
Gökhan Yılmaz,
Ali Özgür Yılmaz
Abstract:
Utilizing low-resolution analog-to-digital converters (ADCs) in uplink massive multiple-input multiple-output (MIMO) systems is a practical solution to decrease power consumption. The performance gap between the low and high-resolution systems is small at low signal-to-noise ratio (SNR) regimes. However, at high SNR and with high modulation orders, the achievable rate saturates after a finite SNR…
▽ More
Utilizing low-resolution analog-to-digital converters (ADCs) in uplink massive multiple-input multiple-output (MIMO) systems is a practical solution to decrease power consumption. The performance gap between the low and high-resolution systems is small at low signal-to-noise ratio (SNR) regimes. However, at high SNR and with high modulation orders, the achievable rate saturates after a finite SNR value due to the stochastic resonance (SR) phenomenon. This paper proposes a novel pseudo-random quantization (PRQ) scheme by modifying the quantization thresholds that can help compensate for the effects of SR and makes communication with high-order modulation schemes such as $1024$-QAM in one-bit quantized uplink massive MIMO systems possible. Moreover, modified linear detectors for non-zero threshold quantization are derived, and a two-stage uplink detector for single-carrier (SC) multi-user systems is proposed. The first stage is an iterative method called Boxed Newton Detector (BND) that utilizes Newton's Method to maximize the log-likelihood with box constraints. The second stage, Nearest Codeword Detector (NCD), exploits the first stage solution and creates a small set of most likely candidates based on sign constraints to increase detection performance. The proposed two-stage method with PRQ outperforms the state-of-the-art detectors from the literature with comparable complexity while supporting high-order modulation schemes.
△ Less
Submitted 7 June, 2023;
originally announced June 2023.
-
SleepMore: Inferring Sleep Duration at Scale via Multi-Device WiFi Sensing
Authors:
Camellia Zakaria,
Gizem Yilmaz,
Priyanka Mammen,
Michael Chee,
Prashant Shenoy,
Rajesh Balan
Abstract:
The availability of commercial wearable trackers equipped with features to monitor sleep duration and quality has enabled more useful sleep health monitoring applications and analyses. However, much research has reported the challenge of long-term user retention in sleep monitoring through these modalities. Since modern Internet users own multiple mobile devices, our work explores the possibility…
▽ More
The availability of commercial wearable trackers equipped with features to monitor sleep duration and quality has enabled more useful sleep health monitoring applications and analyses. However, much research has reported the challenge of long-term user retention in sleep monitoring through these modalities. Since modern Internet users own multiple mobile devices, our work explores the possibility of employing ubiquitous mobile devices and passive WiFi sensing techniques to predict sleep duration as the fundamental measure for complementing long-term sleep monitoring initiatives. In this paper, we propose SleepMore, an accurate and easy-to-deploy sleep-tracking approach based on machine learning over the user's WiFi network activity. It first employs a semi-personalized random forest model with an infinitesimal jackknife variance estimation method to classify a user's network activity behavior into sleep and awake states per minute granularity. Through a moving average technique, the system uses these state sequences to estimate the user's nocturnal sleep period and its uncertainty rate. Uncertainty quantification enables SleepMore to overcome the impact of noisy WiFi data that can yield large prediction errors. We validate SleepMore using data from a month-long user study involving 46 college students and draw comparisons with the Oura Ring wearable. Beyond the college campus, we evaluate SleepMore on non-student users of different housing profiles. Our results demonstrate that SleepMore produces statistically indistinguishable sleep statistics from the Oura ring baseline for predictions made within a 5% uncertainty rate. These errors range between 15-28 minutes for determining sleep time and 7-29 minutes for determining wake time, proving statistically significant improvements over prior work. Our in-depth analysis explains the sources of errors.
△ Less
Submitted 16 November, 2022; v1 submitted 24 October, 2022;
originally announced October 2022.
-
Weighted Maximum Likelihood for Controller Tuning
Authors:
Angel Romero,
Shreedhar Govil,
Gonca Yilmaz,
Yunlong Song,
Davide Scaramuzza
Abstract:
Recently, Model Predictive Contouring Control (MPCC) has arisen as the state-of-the-art approach for model-based agile flight. MPCC benefits from great flexibility in trading-off between progress maximization and path following at runtime without relying on globally optimized trajectories. However, finding the optimal set of tuning parameters for MPCC is challenging because (i) the full quadrotor…
▽ More
Recently, Model Predictive Contouring Control (MPCC) has arisen as the state-of-the-art approach for model-based agile flight. MPCC benefits from great flexibility in trading-off between progress maximization and path following at runtime without relying on globally optimized trajectories. However, finding the optimal set of tuning parameters for MPCC is challenging because (i) the full quadrotor dynamics are non-linear, (ii) the cost function is highly non-convex, and (iii) of the high dimensionality of the hyperparameter space. This paper leverages a probabilistic Policy Search method - Weighted Maximum Likelihood (WML)- to automatically learn the optimal objective for MPCC. WML is sample-efficient due to its closed-form solution for updating the learning parameters. Additionally, the data efficiency provided by the use of a model-based approach allows us to directly train in a high-fidelity simulator, which in turn makes our approach able to transfer zero-shot to the real world. We validate our approach in the real world, where we show that our method outperforms both the previous manually tuned controller and the state-of-the-art auto-tuning baseline reaching speeds of 75 km/h.
△ Less
Submitted 2 March, 2023; v1 submitted 20 October, 2022;
originally announced October 2022.
