-
Reinforcement Learning for Robust Athletic Intelligence: Lessons from the 2nd 'AI Olympics with RealAIGym' Competition
Authors:
Felix Wiebe,
Niccolò Turcato,
Alberto Dalla Libera,
Jean Seong Bjorn Choe,
Bumkyu Choi,
Tim Lukas Faust,
Habib Maraqten,
Erfan Aghadavoodi,
Marco Cali,
Alberto Sinigaglia,
Giulio Giacomuzzo,
Diego Romeres,
Jong-kook Kim,
Gian Antonio Susto,
Shubham Vyas,
Dennis Mronga,
Boris Belousov,
Jan Peters,
Frank Kirchner,
Shivesh Kumar
Abstract:
In the field of robotics many different approaches ranging from classical planning over optimal control to reinforcement learning (RL) are developed and borrowed from other fields to achieve reliable control in diverse tasks. In order to get a clear understanding of their individual strengths and weaknesses and their applicability in real world robotic scenarios is it important to benchmark and co…
▽ More
In the field of robotics many different approaches ranging from classical planning over optimal control to reinforcement learning (RL) are developed and borrowed from other fields to achieve reliable control in diverse tasks. In order to get a clear understanding of their individual strengths and weaknesses and their applicability in real world robotic scenarios is it important to benchmark and compare their performances not only in a simulation but also on real hardware. The '2nd AI Olympics with RealAIGym' competition was held at the IROS 2024 conference to contribute to this cause and evaluate different controllers according to their ability to solve a dynamic control problem on an underactuated double pendulum system with chaotic dynamics. This paper describes the four different RL methods submitted by the participating teams, presents their performance in the swing-up task on a real double pendulum, measured against various criteria, and discusses their transferability from simulation to real hardware and their robustness to external disturbances.
△ Less
Submitted 19 March, 2025;
originally announced March 2025.
-
Velocity-History-Based Soft Actor-Critic Tackling IROS'24 Competition "AI Olympics with RealAIGym"
Authors:
Tim Lukas Faust,
Habib Maraqten,
Erfan Aghadavoodi,
Boris Belousov,
Jan Peters
Abstract:
The ``AI Olympics with RealAIGym'' competition challenges participants to stabilize chaotic underactuated dynamical systems with advanced control algorithms. In this paper, we present a novel solution submitted to IROS'24 competition, which builds upon Soft Actor-Critic (SAC), a popular model-free entropy-regularized Reinforcement Learning (RL) algorithm. We add a `context' vector to the state, wh…
▽ More
The ``AI Olympics with RealAIGym'' competition challenges participants to stabilize chaotic underactuated dynamical systems with advanced control algorithms. In this paper, we present a novel solution submitted to IROS'24 competition, which builds upon Soft Actor-Critic (SAC), a popular model-free entropy-regularized Reinforcement Learning (RL) algorithm. We add a `context' vector to the state, which encodes the immediate history via a Convolutional Neural Network (CNN) to counteract the unmodeled effects on the real system. Our method achieves high performance scores and competitive robustness scores on both tracks of the competition: Pendubot and Acrobot.
△ Less
Submitted 26 October, 2024;
originally announced October 2024.
-
Social Network Structure is Predictive of Health and Wellness
Authors:
Suwen Lin,
Louis Faust,
Pablo Robles-Granda,
Nitesh V. Chawla
Abstract:
Social networks influence health-related behaviors, such as obesity and smoking. While researchers have studied social networks as a driver for diffusion of influences and behaviors, it is less understood how the structure or topology of the network, in itself, impacts an individual's health behaviors and wellness state. In this paper, we investigate whether the structure or topology of a social n…
▽ More
Social networks influence health-related behaviors, such as obesity and smoking. While researchers have studied social networks as a driver for diffusion of influences and behaviors, it is less understood how the structure or topology of the network, in itself, impacts an individual's health behaviors and wellness state. In this paper, we investigate whether the structure or topology of a social network offers additional insight and predictability on an individual's health and wellness. We develop a model called the Network-Driven health predictor (NetCARE) that leverages features representative of social network structure. Using a large longitudinal data set of students enrolled in the NetHealth study at the University of Notre Dame, we show that the NetCARE model improves the overall prediction performance over the baseline models -- that use demographics and physical attributes -- by 38%, 65%, 55%, and 54% for the wellness states -- stress, happiness, positive attitude, and self-assessed health -- considered in this paper.
△ Less
Submitted 7 September, 2018; v1 submitted 31 August, 2018;
originally announced September 2018.
-
Long-term Compliance Habits: What Early Data Tells Us
Authors:
Louis Faust,
Priscilla Jiménez,
David Hachen,
Omar Lizardo,
Aaron Striegel,
Nitesh V. Chawla
Abstract:
The rise in popularity of physical activity trackers provides extensive opportunities for research on personal health, however, barriers such as compliance attrition can lead to substantial losses in data. As such, insights into student's compliance habits could support researcher's decisions when designing long-term studies. In this paper, we examined 392 students on a college campus currently tw…
▽ More
The rise in popularity of physical activity trackers provides extensive opportunities for research on personal health, however, barriers such as compliance attrition can lead to substantial losses in data. As such, insights into student's compliance habits could support researcher's decisions when designing long-term studies. In this paper, we examined 392 students on a college campus currently two and a half years into an ongoing study. We find that compliance data from as early as one month correlated with student's likelihood of dropping out of the study (p < .001) and compliance long-term (p < .001). The findings in this paper identify long-term compliance habits and the viability of their early detection.
△ Less
Submitted 11 April, 2018;
originally announced April 2018.
-
Beyond Volume: The Impact of Complex Healthcare Data on the Machine Learning Pipeline
Authors:
Keith Feldman,
Louis Faust,
Xian Wu,
Chao Huang,
Nitesh V. Chawla
Abstract:
From medical charts to national census, healthcare has traditionally operated under a paper-based paradigm. However, the past decade has marked a long and arduous transformation bringing healthcare into the digital age. Ranging from electronic health records, to digitized imaging and laboratory reports, to public health datasets, today, healthcare now generates an incredible amount of digital info…
▽ More
From medical charts to national census, healthcare has traditionally operated under a paper-based paradigm. However, the past decade has marked a long and arduous transformation bringing healthcare into the digital age. Ranging from electronic health records, to digitized imaging and laboratory reports, to public health datasets, today, healthcare now generates an incredible amount of digital information. Such a wealth of data presents an exciting opportunity for integrated machine learning solutions to address problems across multiple facets of healthcare practice and administration. Unfortunately, the ability to derive accurate and informative insights requires more than the ability to execute machine learning models. Rather, a deeper understanding of the data on which the models are run is imperative for their success. While a significant effort has been undertaken to develop models able to process the volume of data obtained during the analysis of millions of digitalized patient records, it is important to remember that volume represents only one aspect of the data. In fact, drawing on data from an increasingly diverse set of sources, healthcare data presents an incredibly complex set of attributes that must be accounted for throughout the machine learning pipeline. This chapter focuses on highlighting such challenges, and is broken down into three distinct components, each representing a phase of the pipeline. We begin with attributes of the data accounted for during preprocessing, then move to considerations during model building, and end with challenges to the interpretation of model output. For each component, we present a discussion around data as it relates to the healthcare domain and offer insight into the challenges each may impose on the efficiency of machine learning techniques.
△ Less
Submitted 26 January, 2018; v1 submitted 1 June, 2017;
originally announced June 2017.
