-
Machine Learning to Generate Adjustable Dose Distributions in Head-and-Neck Cancer Radiation Therapy
Authors:
Davood Hajinezhad,
Afshin Oroojlooy,
Mohammadreza Nazari,
Xin Hunt,
Jorge Silva,
Colette Shen,
Bhisham Chera,
Shiva K. Das
Abstract:
In this work, we propose a Machine Learning model that generates an adjustable 3D dose distribution for external beam radiation therapy for head-and-neck cancer treatments. In contrast to existing Machine Learning methods that provide a single model, we create pairs of models for each organ-at-risk, namely lower-extreme and upper-extreme models. These model pairs for an organ-at-risk propose doses…
▽ More
In this work, we propose a Machine Learning model that generates an adjustable 3D dose distribution for external beam radiation therapy for head-and-neck cancer treatments. In contrast to existing Machine Learning methods that provide a single model, we create pairs of models for each organ-at-risk, namely lower-extreme and upper-extreme models. These model pairs for an organ-at-risk propose doses that give lower and higher doses to that organ-at-risk, while also encapsulating the dose trade-off to other organs-at-risk. By weighting and combining the model pairs for all organs-at-risk, we are able to dynamically create adjustable dose distributions that can be used, in real-time, to move doses between organs-at-risk, thereby customizing the dose distribution to the needs of a particular patient. We leverage a key observation that the training data set inherently contains the clinical trade-offs. We show that the adjustable distributions are able to provide reasonable clinical dose latitude in the trade-off of doses between organs-at-risk.
△ Less
Submitted 14 July, 2022;
originally announced July 2022.
-
Towards optimized actions in critical situations of soccer games with deep reinforcement learning
Authors:
Pegah Rahimian,
Afshin Oroojlooy,
Laszlo Toka
Abstract:
Soccer is a sparse rewarding game: any smart or careless action in critical situations can change the result of the match. Therefore players, coaches, and scouts are all curious about the best action to be performed in critical situations, such as the times with a high probability of losing ball possession or scoring a goal. This work proposes a new state representation for the soccer game and a b…
▽ More
Soccer is a sparse rewarding game: any smart or careless action in critical situations can change the result of the match. Therefore players, coaches, and scouts are all curious about the best action to be performed in critical situations, such as the times with a high probability of losing ball possession or scoring a goal. This work proposes a new state representation for the soccer game and a batch reinforcement learning to train a smart policy network. This network gets the contextual information of the situation and proposes the optimal action to maximize the expected goal for the team. We performed extensive numerical experiments on the soccer logs made by InStat for 104 European soccer matches. The results show that in all 104 games, the optimized policy obtains higher rewards than its counterpart in the behavior policy. Besides, our framework learns policies that are close to the expected behavior in the real world. For instance, in the optimized policy, we observe that some actions such as foul, or ball out can be sometimes more rewarding than a shot in specific situations.
△ Less
Submitted 14 September, 2021;
originally announced September 2021.
-
AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control
Authors:
Afshin Oroojlooy,
Mohammadreza Nazari,
Davood Hajinezhad,
Jorge Silva
Abstract:
We propose AttendLight, an end-to-end Reinforcement Learning (RL) algorithm for the problem of traffic signal control. Previous approaches for this problem have the shortcoming that they require training for each new intersection with a different structure or traffic flow distribution. AttendLight solves this issue by training a single, universal model for intersections with any number of roads, l…
▽ More
We propose AttendLight, an end-to-end Reinforcement Learning (RL) algorithm for the problem of traffic signal control. Previous approaches for this problem have the shortcoming that they require training for each new intersection with a different structure or traffic flow distribution. AttendLight solves this issue by training a single, universal model for intersections with any number of roads, lanes, phases (possible signals), and traffic flow. To this end, we propose a deep RL model which incorporates two attention models. The first attention model is introduced to handle different numbers of roads-lanes; and the second attention model is intended for enabling decision-making with any number of phases in an intersection. As a result, our proposed model works for any intersection configuration, as long as a similar configuration is represented in the training set. Experiments were conducted with both synthetic and real-world standard benchmark data-sets. The results we show cover intersections with three or four approaching roads; one-directional/bi-directional roads with one, two, and three lanes; different number of phases; and different traffic flows. We consider two regimes: (i) single-environment training, single-deployment, and (ii) multi-environment training, multi-deployment. AttendLight outperforms both classical and other RL-based approaches on all cases in both regimes.
△ Less
Submitted 12 October, 2020;
originally announced October 2020.
-
Reinforcement Learning for Solving the Vehicle Routing Problem
Authors:
Mohammadreza Nazari,
Afshin Oroojlooy,
Lawrence V. Snyder,
Martin Takáč
Abstract:
We present an end-to-end framework for solving the Vehicle Routing Problem (VRP) using reinforcement learning. In this approach, we train a single model that finds near-optimal solutions for problem instances sampled from a given distribution, only by observing the reward signals and following feasibility rules. Our model represents a parameterized stochastic policy, and by applying a policy gradi…
▽ More
We present an end-to-end framework for solving the Vehicle Routing Problem (VRP) using reinforcement learning. In this approach, we train a single model that finds near-optimal solutions for problem instances sampled from a given distribution, only by observing the reward signals and following feasibility rules. Our model represents a parameterized stochastic policy, and by applying a policy gradient algorithm to optimize its parameters, the trained model produces the solution as a sequence of consecutive actions in real time, without the need to re-train for every new problem instance. On capacitated VRP, our approach outperforms classical heuristics and Google's OR-Tools on medium-sized instances in solution quality with comparable computation time (after training). We demonstrate how our approach can handle problems with split delivery and explore the effect of such deliveries on the solution quality. Our proposed framework can be applied to other variants of the VRP such as the stochastic VRP, and has the potential to be applied more generally to combinatorial optimization problems.
△ Less
Submitted 21 May, 2018; v1 submitted 12 February, 2018;
originally announced February 2018.
