Showing 1–2 of 2 results for author: Bambos, N
-
Finite-Time Bounds for Two-Time-Scale Stochastic Approximation with Arbitrary Norm Contractions and Markovian Noise
Authors:
Siddharth Chandak,
Shaan Ul Haque,
Nicholas Bambos
Abstract:
Two-time-scale Stochastic Approximation (SA) is an iterative algorithm with applications in reinforcement learning and optimization. Prior finite time analysis of such algorithms has focused on fixed point iterations with mappings contractive under Euclidean norm. Motivated by applications in reinforcement learning, we give the first mean square bound on non linear two-time-scale SA where the iter…
▽ More
Two-time-scale Stochastic Approximation (SA) is an iterative algorithm with applications in reinforcement learning and optimization. Prior finite time analysis of such algorithms has focused on fixed point iterations with mappings contractive under Euclidean norm. Motivated by applications in reinforcement learning, we give the first mean square bound on non linear two-time-scale SA where the iterations have arbitrary norm contractive mappings and Markovian noise. We show that the mean square error decays at a rate of $O(1/n^{2/3})$ in the general case, and at a rate of $O(1/n)$ in a special case where the slower timescale is noiseless. Our analysis uses the generalized Moreau envelope to handle the arbitrary norm contractions and solutions of Poisson equation to deal with the Markovian noise. By analyzing the SSP Q-Learning algorithm, we give the first $O(1/n)$ bound for an algorithm for asynchronous control of MDPs under the average reward criterion. We also obtain a rate of $O(1/n)$ for Q-Learning with Polyak-averaging and provide an algorithm for learning Generalized Nash Equilibrium (GNE) for strongly monotone games which converges at a rate of $O(1/n^{2/3})$.
△ Less
Submitted 24 March, 2025;
originally announced March 2025.
-
Predicting Pediatric Surgical Durations
Authors:
Neal Master,
David Scheinker,
Nicholas Bambos
Abstract:
Effective management of operating room resources relies on accurate predictions of surgical case durations. This prediction problem is known to be particularly difficult in pediatric hospitals due to the extreme variation in pediatric patient populations. We propose a novel metric for measuring accuracy of predictions which captures key issues relevant to hospital operations. With this metric in m…
▽ More
Effective management of operating room resources relies on accurate predictions of surgical case durations. This prediction problem is known to be particularly difficult in pediatric hospitals due to the extreme variation in pediatric patient populations. We propose a novel metric for measuring accuracy of predictions which captures key issues relevant to hospital operations. With this metric in mind we propose several tree-based prediction models. Some are automated (they do not require input from surgeons) while others are semi-automated (they do require input from surgeons). We see that many of our automated methods generally outperform currently used algorithms and even achieve the same performance as surgeons. Our semi-automated methods can outperform surgeons by a significant margin. We gain insights into the predictive value of different features and suggest avenues of future work.
△ Less
Submitted 31 December, 2016; v1 submitted 15 May, 2016;
originally announced May 2016.
