Showing 1–4 of 4 results for author: Karakas, M

Does Feedback Help in Bandits with Arm Erasures?

Authors: Merve Karakas, Osama Hanna, Lin F. Yang, Christina Fragouli

Abstract: We study a distributed multi-armed bandit (MAB) problem over arm erasure channels, motivated by the increasing adoption of MAB algorithms over communication-constrained networks. In this setup, the learner communicates the chosen arm to play to an agent over an erasure channel with probability $ε\in [0,1)$; if an erasure occurs, the agent continues pulling the last successfully received arm; the l… ▽ More We study a distributed multi-armed bandit (MAB) problem over arm erasure channels, motivated by the increasing adoption of MAB algorithms over communication-constrained networks. In this setup, the learner communicates the chosen arm to play to an agent over an erasure channel with probability $ε\in [0,1)$; if an erasure occurs, the agent continues pulling the last successfully received arm; the learner always observes the reward of the arm pulled. In past work, we considered the case where the agent cannot convey feedback to the learner, and thus the learner does not know whether the arm played is the requested or the last successfully received one. In this paper, we instead consider the case where the agent can send feedback to the learner on whether the arm request was received, and thus the learner exactly knows which arm was played. Surprisingly, we prove that erasure feedback does not improve the worst-case regret upper bound order over the previously studied no-feedback setting. In particular, we prove a regret lower bound of $Ω(\sqrt{KT} + K / (1 - ε))$, where $K$ is the number of arms and $T$ the time horizon, that matches no-feedback upper bounds up to logarithmic factors. We note however that the availability of feedback enables simpler algorithm designs that may achieve better constants (albeit not better order) regret bounds; we design one such algorithm and evaluate its performance numerically. △ Less

Submitted 29 April, 2025; originally announced April 2025.

Enhancing Binary Search via Overlapping Partitions

Authors: Kaan Buyukkalayci, Merve Karakas, Xinlin Li, Christina Fragouli

Abstract: This paper considers the task of performing binary search under noisy decisions, focusing on the application of target area localization. In the presence of noise, the classical partitioning approach of binary search is prone to error propagation due to the use of strictly disjoint splits. While existing works on noisy binary search propose techniques such as query repetition or probabilistic upda… ▽ More This paper considers the task of performing binary search under noisy decisions, focusing on the application of target area localization. In the presence of noise, the classical partitioning approach of binary search is prone to error propagation due to the use of strictly disjoint splits. While existing works on noisy binary search propose techniques such as query repetition or probabilistic updates to mitigate errors, they often lack explicit mechanisms to manage the trade-off between error probability and search complexity, with some providing only asymptotic guarantees. To address this gap, we propose a binary search framework with tunable overlapping partitions, which introduces controlled redundancy into the search process to enhance robustness against noise. We analyze the performance of the proposed algorithm in both discrete and continuous domains for the problem of area localization, quantifying how the overlap parameter impacts the trade-off between search tree depth and error probability. Unlike previous methods, this approach allows for direct control over the balance between reliability and efficiency. Our results emphasize the versatility and effectiveness of the proposed method, providing a principled extension to existing noisy search paradigms and enabling new insights into the interplay between partitioning strategies and measurement reliability. △ Less

Submitted 29 April, 2025; v1 submitted 29 April, 2025; originally announced April 2025.

Learning for Bandits under Action Erasures

Authors: Osama Hanna, Merve Karakas, Lin F. Yang, Christina Fragouli

Abstract: We consider a novel multi-arm bandit (MAB) setup, where a learner needs to communicate the actions to distributed agents over erasure channels, while the rewards for the actions are directly available to the learner through external sensors. In our model, while the distributed agents know if an action is erased, the central learner does not (there is no feedback), and thus does not know whether th… ▽ More We consider a novel multi-arm bandit (MAB) setup, where a learner needs to communicate the actions to distributed agents over erasure channels, while the rewards for the actions are directly available to the learner through external sensors. In our model, while the distributed agents know if an action is erased, the central learner does not (there is no feedback), and thus does not know whether the observed reward resulted from the desired action or not. We propose a scheme that can work on top of any (existing or future) MAB algorithm and make it robust to action erasures. Our scheme results in a worst-case regret over action-erasure channels that is at most a factor of $O(1/\sqrt{1-ε})$ away from the no-erasure worst-case regret of the underlying MAB algorithm, where $ε$ is the erasure probability. We also propose a modification of the successive arm elimination algorithm and prove that its worst-case regret is $\Tilde{O}(\sqrt{KT}+K/(1-ε))$, which we prove is optimal by providing a matching lower bound. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Multi-Agent Bandit Learning through Heterogeneous Action Erasure Channels

Authors: Osama A. Hanna, Merve Karakas, Lin F. Yang, Christina Fragouli

Abstract: Multi-Armed Bandit (MAB) systems are witnessing an upswing in applications within multi-agent distributed environments, leading to the advancement of collaborative MAB algorithms. In such settings, communication between agents executing actions and the primary learner making decisions can hinder the learning process. A prevalent challenge in distributed learning is action erasure, often induced by… ▽ More Multi-Armed Bandit (MAB) systems are witnessing an upswing in applications within multi-agent distributed environments, leading to the advancement of collaborative MAB algorithms. In such settings, communication between agents executing actions and the primary learner making decisions can hinder the learning process. A prevalent challenge in distributed learning is action erasure, often induced by communication delays and/or channel noise. This results in agents possibly not receiving the intended action from the learner, subsequently leading to misguided feedback. In this paper, we introduce novel algorithms that enable learners to interact concurrently with distributed agents across heterogeneous action erasure channels with different action erasure probabilities. We illustrate that, in contrast to existing bandit algorithms, which experience linear regret, our algorithms assure sub-linear regret guarantees. Our proposed solutions are founded on a meticulously crafted repetition protocol and scheduling of learning across heterogeneous channels. To our knowledge, these are the first algorithms capable of effectively learning through heterogeneous action erasure channels. We substantiate the superior performance of our algorithm through numerical experiments, emphasizing their practical significance in addressing issues related to communication constraints and delays in multi-agent environments. △ Less

Submitted 29 April, 2024; v1 submitted 21 December, 2023; originally announced December 2023.