-
SAINT: Attention-Based Modeling of Sub-Action Dependencies in Multi-Action Policies
Authors:
Matthew Landers,
Taylor W. Killian,
Thomas Hartvigsen,
Afsaneh Doryab
Abstract:
The combinatorial structure of many real-world action spaces leads to exponential growth in the number of possible actions, limiting the effectiveness of conventional reinforcement learning algorithms. Recent approaches for combinatorial action spaces impose factorized or sequential structures over sub-actions, failing to capture complex joint behavior. We introduce the Sub-Action Interaction Netw…
▽ More
The combinatorial structure of many real-world action spaces leads to exponential growth in the number of possible actions, limiting the effectiveness of conventional reinforcement learning algorithms. Recent approaches for combinatorial action spaces impose factorized or sequential structures over sub-actions, failing to capture complex joint behavior. We introduce the Sub-Action Interaction Network using Transformers (SAINT), a novel policy architecture that represents multi-component actions as unordered sets and models their dependencies via self-attention conditioned on the global state. SAINT is permutation-invariant, sample-efficient, and compatible with standard policy optimization algorithms. In 15 distinct combinatorial environments across three task domains, including environments with nearly 17 million joint actions, SAINT consistently outperforms strong baselines.
△ Less
Submitted 17 May, 2025;
originally announced May 2025.
-
Factorized Deep Q-Network for Cooperative Multi-Agent Reinforcement Learning in Victim Tagging
Authors:
Maria Ana Cardei,
Afsaneh Doryab
Abstract:
Mass casualty incidents (MCIs) are a growing concern, characterized by complexity and uncertainty that demand adaptive decision-making strategies. The victim tagging step in the emergency medical response must be completed quickly and is crucial for providing information to guide subsequent time-constrained response actions. In this paper, we present a mathematical formulation of multi-agent victi…
▽ More
Mass casualty incidents (MCIs) are a growing concern, characterized by complexity and uncertainty that demand adaptive decision-making strategies. The victim tagging step in the emergency medical response must be completed quickly and is crucial for providing information to guide subsequent time-constrained response actions. In this paper, we present a mathematical formulation of multi-agent victim tagging to minimize the time it takes for responders to tag all victims. Five distributed heuristics are formulated and evaluated with simulation experiments. The heuristics considered are on-the go, practical solutions that represent varying levels of situational uncertainty in the form of global or local communication capabilities, showcasing practical constraints. We further investigate the performance of a multi-agent reinforcement learning (MARL) strategy, factorized deep Q-network (FDQN), to minimize victim tagging time as compared to baseline heuristics. Extensive simulations demonstrate that between the heuristics, methods with local communication are more efficient for adaptive victim tagging, specifically choosing the nearest victim with the option to replan. Analyzing all experiments, we find that our FDQN approach outperforms heuristics in smaller-scale scenarios, while heuristics excel in more complex scenarios. Our experiments contain diverse complexities that explore the upper limits of MARL capabilities for real-world applications and reveal key insights.
△ Less
Submitted 1 March, 2025;
originally announced March 2025.
-
Pairwise Spatiotemporal Partial Trajectory Matching for Co-movement Analysis
Authors:
Maria Cardei,
Sabit Ahmed,
Gretchen Chapman,
Afsaneh Doryab
Abstract:
Spatiotemporal pairwise movement analysis involves identifying shared geographic-based behaviors between individuals within specific time frames. Traditionally, this task relies on sequence modeling and behavior analysis techniques applied to tabular or video-based data, but these methods often lack interpretability and struggle to capture partial matching. In this paper, we propose a novel method…
▽ More
Spatiotemporal pairwise movement analysis involves identifying shared geographic-based behaviors between individuals within specific time frames. Traditionally, this task relies on sequence modeling and behavior analysis techniques applied to tabular or video-based data, but these methods often lack interpretability and struggle to capture partial matching. In this paper, we propose a novel method for pairwise spatiotemporal partial trajectory matching that transforms tabular spatiotemporal data into interpretable trajectory images based on specified time windows, allowing for partial trajectory analysis. This approach includes localization of trajectories, checking for spatial overlap, and pairwise matching using a Siamese Neural Network. We evaluate our method on a co-walking classification task, demonstrating its effectiveness in a novel co-behavior identification application. Our model surpasses established methods, achieving an F1-score up to 0.73. Additionally, we explore the method's utility for pair routine pattern analysis in real-world scenarios, providing insights into the frequency, timing, and duration of shared behaviors. This approach offers a powerful, interpretable framework for spatiotemporal behavior analysis, with potential applications in social behavior research, urban planning, and healthcare.
△ Less
Submitted 3 December, 2024;
originally announced December 2024.
-
BraVE: Offline Reinforcement Learning for Discrete Combinatorial Action Spaces
Authors:
Matthew Landers,
Taylor W. Killian,
Hugo Barnes,
Thomas Hartvigsen,
Afsaneh Doryab
Abstract:
Offline reinforcement learning in high-dimensional, discrete action spaces is challenging due to the exponential scaling of the joint action space with the number of sub-actions and the complexity of modeling sub-action dependencies. Existing methods either exhaustively evaluate the action space, making them computationally infeasible, or factorize Q-values, failing to represent joint sub-action e…
▽ More
Offline reinforcement learning in high-dimensional, discrete action spaces is challenging due to the exponential scaling of the joint action space with the number of sub-actions and the complexity of modeling sub-action dependencies. Existing methods either exhaustively evaluate the action space, making them computationally infeasible, or factorize Q-values, failing to represent joint sub-action effects. We propose Branch Value Estimation (BraVE), a value-based method that uses tree-structured action traversal to evaluate a linear number of joint actions while preserving dependency structure. BraVE outperforms prior offline RL methods by up to $20\times$ in environments with over four million actions.
△ Less
Submitted 17 May, 2025; v1 submitted 28 October, 2024;
originally announced October 2024.
-
Exploring Smartphone-based Spectrophotometry for Nutrient Identification and Quantification
Authors:
Andrew Balch,
Maria A. Cardei,
Afsaneh Doryab
Abstract:
Imbalanced nutrition is a global health issue with significant downstream effects. Current methods of assessing nutrient levels face several limitations, with accessibility being a major concern. In this paper, we take a step towards accessibly measuring nutrient status within the body. We explore the potential of smartphone-based spectrophotometry for identifying and quantifying nutrients in a so…
▽ More
Imbalanced nutrition is a global health issue with significant downstream effects. Current methods of assessing nutrient levels face several limitations, with accessibility being a major concern. In this paper, we take a step towards accessibly measuring nutrient status within the body. We explore the potential of smartphone-based spectrophotometry for identifying and quantifying nutrients in a solution by building and testing two prototype devices. We compared the prototypes and found that the limitations posed by the initial, simpler prototype were well addressed in the more portable and reliable second-generation device. With the second-generation prototype, we created and implemented a semi-automatic signal processing and analysis pipeline for analyzing absorption spectra. We thoroughly evaluated the prototypes by analyzing the effect of four different light sources and three reference spectra strategies. Results demonstrate that an LED bulb light source performed best, and all reference spectra strategies performed similarly. We then compared the second-generation prototype to a benchtop laboratory spectrophotometer to further validate the device. We applied the Beer-Lambert Law to demonstrate that our prototype is able to quantify the amount of vitamin B12 in a solution with an accuracy of up to 91.3%. Our in-depth analyses, discussions, and results demonstrate the potential use of smartphone-based spectrophotometry as an accessible method to identify and quantify nutrients and pave the way for future developments that can apply this approach to the human body.
△ Less
Submitted 15 October, 2024; v1 submitted 14 October, 2024;
originally announced October 2024.
-
Towards an Accessible, Noninvasive Micronutrient Status Assessment Method: A Comprehensive Review of Existing Techniques
Authors:
Andrew Balch,
Maria A. Cardei,
Sibylle Kranz,
Afsaneh Doryab
Abstract:
Nutrients are critical to the functioning of the human body and their imbalance can result in detrimental health concerns. The majority of nutritional literature focuses on macronutrients, often ignoring the more critical nuances of micronutrient balance, which require more precise regulation. Currently, micronutrient status is routinely assessed via complex methods that are arduous for both the p…
▽ More
Nutrients are critical to the functioning of the human body and their imbalance can result in detrimental health concerns. The majority of nutritional literature focuses on macronutrients, often ignoring the more critical nuances of micronutrient balance, which require more precise regulation. Currently, micronutrient status is routinely assessed via complex methods that are arduous for both the patient and the clinician. To address the global burden of micronutrient malnutrition, innovations in assessment must be accessible and noninvasive. In support of this task, this article synthesizes useful background information on micronutrients themselves, reviews the state of biofluid and physiological analyses for their assessment, and presents actionable opportunities to push the field forward. By taking a unique, clinical perspective that is absent from technological research on the topic, we find that the state of the art suffers from limited clinical relevance, a lack of overlap between biofluid and physiological approaches, and highly invasive and inaccessible solutions. Future work has the opportunity to maximize the impact of a novel assessment method by incorporating clinical relevance, the holistic nature of micronutrition, and prioritizing accessible and noninvasive systems.
△ Less
Submitted 7 June, 2025; v1 submitted 20 August, 2024;
originally announced August 2024.
-
Towards a Computational Framework for Automated Discovery and Modeling of Biological Rhythms from Wearable Data Streams
Authors:
Runze Yan,
Afsaneh Doryab
Abstract:
Modeling biological rhythms helps understand the complex principles behind the physical and psychological abnormalities of human bodies, to plan life schedules, and avoid persisting fatigue and mood and sleep alterations due to the desynchronization of those rhythms. The first step in modeling biological rhythms is to identify their characteristics, such as cyclic periods, phase, and amplitude. Ho…
▽ More
Modeling biological rhythms helps understand the complex principles behind the physical and psychological abnormalities of human bodies, to plan life schedules, and avoid persisting fatigue and mood and sleep alterations due to the desynchronization of those rhythms. The first step in modeling biological rhythms is to identify their characteristics, such as cyclic periods, phase, and amplitude. However, human rhythms are susceptible to external events, which cause irregular fluctuations in waveforms and affect the characterization of each rhythm. In this paper, we present our exploratory work towards developing a computational framework for automated discovery and modeling of human rhythms. We first identify cyclic periods in time series data using three different methods and test their performance on both synthetic data and real fine-grained biological data. We observe consistent periods are detected by all three methods. We then model inner cycles within each period through identifying change points to observe fluctuations in biological data that may inform the impact of external events on human rhythms. The results provide initial insights into the design of a computational framework for discovering and modeling human rhythms.
△ Less
Submitted 13 September, 2021;
originally announced September 2021.
-
Understanding health and behavioral trends of successful students through machine learning models
Authors:
Abigale Kim,
Fateme Nikseresht,
Janine M. Dutcher,
Michael Tumminia,
Daniella Villalba,
Sheldon Cohen,
Kasey Creswel,
David Creswell,
Anind K. Dey,
Jennifer Mankoff,
Afsaneh Doryab
Abstract:
This study analyzes patterns of physical, mental, lifestyle, and personality factors in college students in different periods over the course of a semester and models their relationships with students' academic performance. The data analyzed was collected through smartphones and Fitbit. The use of machine learning models derived from the gathered data was employed to observe the extent of students…
▽ More
This study analyzes patterns of physical, mental, lifestyle, and personality factors in college students in different periods over the course of a semester and models their relationships with students' academic performance. The data analyzed was collected through smartphones and Fitbit. The use of machine learning models derived from the gathered data was employed to observe the extent of students' behavior associated with their GPA, lifestyle, physical health, mental health, and personality attributes. A mutual agreement method was used in which rather than looking at the accuracy of results, the model parameters and weights of features were used to find common behavioral trends. From the results of the model creation, it was determined that the most significant indicator of academic success defined as a higher GPA, was the places a student spent their time. Lifestyle and personality factors were deemed more significant than mental and physical factors. This study will provide insight into the impact of different factors and the timing of those factors on students' academic performance.
△ Less
Submitted 23 January, 2021;
originally announced February 2021.
-
Detection of Racial Bias from Physiological Responses
Authors:
Fateme Nikseresht,
Runze Yan,
Rachel Lew,
Yingzheng Liu,
Rose M. Sebastian,
Afsaneh Doryab
Abstract:
Despite the evolution of norms and regulations to mitigate the harm from biases, harmful discrimination linked to an individual's unconscious biases persists. Our goal is to better understand and detect the physiological and behavioral indicators of implicit biases. This paper investigates whether we can reliably detect racial bias from physiological responses, including heart rate, conductive ski…
▽ More
Despite the evolution of norms and regulations to mitigate the harm from biases, harmful discrimination linked to an individual's unconscious biases persists. Our goal is to better understand and detect the physiological and behavioral indicators of implicit biases. This paper investigates whether we can reliably detect racial bias from physiological responses, including heart rate, conductive skin response, skin temperature, and micro-body movements. We analyzed data from 46 subjects whose physiological data was collected with Empatica E4 wristband while taking an Implicit Association Test (IAT). Our machine learning and statistical analysis show that implicit bias can be predicted from physiological signals with 76.1% accuracy. Our results also show that the EDA signal associated with skin response has the strongest correlation with racial bias and that there are significant differences between the values of EDA features for biased and unbiased participants.
△ Less
Submitted 1 February, 2021;
originally announced February 2021.
-
HHAR-net: Hierarchical Human Activity Recognition using Neural Networks
Authors:
Mehrdad Fazli,
Kamran Kowsari,
Erfaneh Gharavi,
Laura Barnes,
Afsaneh Doryab
Abstract:
Activity recognition using built-in sensors in smart and wearable devices provides great opportunities to understand and detect human behavior in the wild and gives a more holistic view of individuals' health and well being. Numerous computational methods have been applied to sensor streams to recognize different daily activities. However, most methods are unable to capture different layers of act…
▽ More
Activity recognition using built-in sensors in smart and wearable devices provides great opportunities to understand and detect human behavior in the wild and gives a more holistic view of individuals' health and well being. Numerous computational methods have been applied to sensor streams to recognize different daily activities. However, most methods are unable to capture different layers of activities concealed in human behavior. Also, the performance of the models starts to decrease with increasing the number of activities. This research aims at building a hierarchical classification with Neural Networks to recognize human activities based on different levels of abstraction. We evaluate our model on the Extrasensory dataset; a dataset collected in the wild and containing data from smartphones and smartwatches. We use a two-level hierarchy with a total of six mutually exclusive labels namely, "lying down", "sitting", "standing in place", "walking", "running", and "bicycling" divided into "stationary" and "non-stationary". The results show that our model can recognize low-level activities (stationary/non-stationary) with 95.8% accuracy and overall accuracy of 92.8% over six labels. This is 3% above our best performing baseline.
△ Less
Submitted 10 November, 2020; v1 submitted 28 October, 2020;
originally announced October 2020.
-
Can Smartphone Co-locations Detect Friendship? It Depends How You Model It
Authors:
Momin M. Malik,
Afsaneh Doryab,
Michael Merrill,
Jürgen Pfeffer,
Anind K. Dey
Abstract:
We present a study to detect friendship, its strength, and its change from smartphone location data collectedamong members of a fraternity. We extract a rich set of co-location features and build classifiers that detectfriendships and close friendship at 30% above a random baseline. We design cross-validation schema to testour model performance in specific application settings, finding it robust t…
▽ More
We present a study to detect friendship, its strength, and its change from smartphone location data collectedamong members of a fraternity. We extract a rich set of co-location features and build classifiers that detectfriendships and close friendship at 30% above a random baseline. We design cross-validation schema to testour model performance in specific application settings, finding it robust to seeing new dyads and to temporalvariance.
△ Less
Submitted 30 August, 2020; v1 submitted 6 August, 2020;
originally announced August 2020.
-
A Robot's Expressive Language Affects Human Strategy and Perceptions in a Competitive Game
Authors:
Aaron M. Roth,
Samantha Reig,
Umang Bhatt,
Jonathan Shulgach,
Tamara Amin,
Afsaneh Doryab,
Fei Fang,
Manuela Veloso
Abstract:
As robots are increasingly endowed with social and communicative capabilities, they will interact with humans in more settings, both collaborative and competitive. We explore human-robot relationships in the context of a competitive Stackelberg Security Game. We vary humanoid robot expressive language (in the form of "encouraging" or "discouraging" verbal commentary) and measure the impact on part…
▽ More
As robots are increasingly endowed with social and communicative capabilities, they will interact with humans in more settings, both collaborative and competitive. We explore human-robot relationships in the context of a competitive Stackelberg Security Game. We vary humanoid robot expressive language (in the form of "encouraging" or "discouraging" verbal commentary) and measure the impact on participants' rationality, strategy prioritization, mood, and perceptions of the robot. We learn that a robot opponent that makes discouraging comments causes a human to play a game less rationally and to perceive the robot more negatively. We also contribute a simple open source Natural Language Processing framework for generating expressive sentences, which was used to generate the speech of our autonomous social robot.
△ Less
Submitted 24 October, 2019;
originally announced October 2019.
-
Extraction of Behavioral Features from Smartphone and Wearable Data
Authors:
Afsaneh Doryab,
Prerna Chikarsel,
Xinwen Liu,
Anind K. Dey
Abstract:
The rich set of sensors in smartphones and wearable devices provides the possibility to passively collect streams of data in the wild. The raw data streams, however, can rarely be directly used in the modeling pipeline. We provide a generic framework that can process raw data streams and extract useful features related to non-verbal human behavior. This framework can be used by researchers in the…
▽ More
The rich set of sensors in smartphones and wearable devices provides the possibility to passively collect streams of data in the wild. The raw data streams, however, can rarely be directly used in the modeling pipeline. We provide a generic framework that can process raw data streams and extract useful features related to non-verbal human behavior. This framework can be used by researchers in the field who are interested in processing data from smartphones and Wearable devices.
△ Less
Submitted 8 January, 2019; v1 submitted 18 December, 2018;
originally announced December 2018.
-
The Impact of Humanoid Affect Expression on Human Behavior in a Game-Theoretic Setting
Authors:
Aaron M. Roth,
Umang Bhatt,
Tamara Amin,
Afsaneh Doryab,
Fei Fang,
Manuela Veloso
Abstract:
With the rapid development of robot and other intelligent and autonomous agents, how a human could be influenced by a robot's expressed mood when making decisions becomes a crucial question in human-robot interaction. In this pilot study, we investigate (1) in what way a robot can express a certain mood to influence a human's decision making behavioral model; (2) how and to what extent the human w…
▽ More
With the rapid development of robot and other intelligent and autonomous agents, how a human could be influenced by a robot's expressed mood when making decisions becomes a crucial question in human-robot interaction. In this pilot study, we investigate (1) in what way a robot can express a certain mood to influence a human's decision making behavioral model; (2) how and to what extent the human will be influenced in a game theoretic setting. More specifically, we create an NLP model to generate sentences that adhere to a specific affective expression profile. We use these sentences for a humanoid robot as it plays a Stackelberg security game against a human. We investigate the behavioral model of the human player.
△ Less
Submitted 10 June, 2018;
originally announced June 2018.
