-
Proceedings of 1st Workshop on Advancing Artificial Intelligence through Theory of Mind
Authors:
Mouad Abrini,
Omri Abend,
Dina Acklin,
Henny Admoni,
Gregor Aichinger,
Nitay Alon,
Zahra Ashktorab,
Ashish Atreja,
Moises Auron,
Alexander Aufreiter,
Raghav Awasthi,
Soumya Banerjee,
Joe M. Barnby,
Rhea Basappa,
Severin Bergsmann,
Djallel Bouneffouf,
Patrick Callaghan,
Marc Cavazza,
Thierry Chaminade,
Sonia Chernova,
Mohamed Chetouan,
Moumita Choudhury,
Axel Cleeremans,
Jacek B. Cywinski,
Fabio Cuzzolin
, et al. (83 additional authors not shown)
Abstract:
This volume includes a selection of papers presented at the Workshop on Advancing Artificial Intelligence through Theory of Mind held at AAAI 2025 in Philadelphia US on 3rd March 2025. The purpose of this volume is to provide an open access and curated anthology for the ToM and AI research community.
This volume includes a selection of papers presented at the Workshop on Advancing Artificial Intelligence through Theory of Mind held at AAAI 2025 in Philadelphia US on 3rd March 2025. The purpose of this volume is to provide an open access and curated anthology for the ToM and AI research community.
△ Less
Submitted 28 April, 2025;
originally announced May 2025.
-
Bi-Directional Mental Model Reconciliation for Human-Robot Interaction with Large Language Models
Authors:
Nina Moorman,
Michelle Zhao,
Matthew B. Luebbers,
Sanne Van Waveren,
Reid Simmons,
Henny Admoni,
Sonia Chernova,
Matthew Gombolay
Abstract:
In human-robot interactions, human and robot agents maintain internal mental models of their environment, their shared task, and each other. The accuracy of these representations depends on each agent's ability to perform theory of mind, i.e. to understand the knowledge, preferences, and intentions of their teammate. When mental models diverge to the extent that it affects task execution, reconcil…
▽ More
In human-robot interactions, human and robot agents maintain internal mental models of their environment, their shared task, and each other. The accuracy of these representations depends on each agent's ability to perform theory of mind, i.e. to understand the knowledge, preferences, and intentions of their teammate. When mental models diverge to the extent that it affects task execution, reconciliation becomes necessary to prevent the degradation of interaction. We propose a framework for bi-directional mental model reconciliation, leveraging large language models to facilitate alignment through semi-structured natural language dialogue. Our framework relaxes the assumption of prior model reconciliation work that either the human or robot agent begins with a correct model for the other agent to align to. Through our framework, both humans and robots are able to identify and communicate missing task-relevant context during interaction, iteratively progressing toward a shared mental model.
△ Less
Submitted 10 March, 2025;
originally announced March 2025.
-
Clean for Haskell Programmers
Authors:
Mart Lubbers,
Peter Achten
Abstract:
This note is meant to give people who are familiar with the functional programming language Haskell a concise overview of Clean language elements and how they differ from Haskell.
This note is meant to give people who are familiar with the functional programming language Haskell a concise overview of Clean language elements and how they differ from Haskell.
△ Less
Submitted 29 October, 2024;
originally announced November 2024.
-
Workspace Optimization Techniques to Improve Prediction of Human Motion During Human-Robot Collaboration
Authors:
Yi-Shiuan Tung,
Matthew B. Luebbers,
Alessandro Roncone,
Bradley Hayes
Abstract:
Understanding human intentions is critical for safe and effective human-robot collaboration. While state of the art methods for human goal prediction utilize learned models to account for the uncertainty of human motion data, that data is inherently stochastic and high variance, hindering those models' utility for interactions requiring coordination, including safety-critical or close-proximity ta…
▽ More
Understanding human intentions is critical for safe and effective human-robot collaboration. While state of the art methods for human goal prediction utilize learned models to account for the uncertainty of human motion data, that data is inherently stochastic and high variance, hindering those models' utility for interactions requiring coordination, including safety-critical or close-proximity tasks. Our key insight is that robot teammates can deliberately configure shared workspaces prior to interaction in order to reduce the variance in human motion, realizing classifier-agnostic improvements in goal prediction. In this work, we present an algorithmic approach for a robot to arrange physical objects and project "virtual obstacles" using augmented reality in shared human-robot workspaces, optimizing for human legibility over a given set of tasks. We compare our approach against other workspace arrangement strategies using two human-subjects studies, one in a virtual 2D navigation domain and the other in a live tabletop manipulation domain involving a robotic manipulator arm. We evaluate the accuracy of human motion prediction models learned from each condition, demonstrating that our workspace optimization technique with virtual obstacles leads to higher robot prediction accuracy using less training data.
△ Less
Submitted 23 January, 2024;
originally announced January 2024.
-
Improving Human Legibility in Collaborative Robot Tasks through Augmented Reality and Workspace Preparation
Authors:
Yi-Shiuan Tung,
Matthew B. Luebbers,
Alessandro Roncone,
Bradley Hayes
Abstract:
Understanding the intentions of human teammates is critical for safe and effective human-robot interaction. The canonical approach for human-aware robot motion planning is to first predict the human's goal or path, and then construct a robot plan that avoids collision with the human. This method can generate unsafe interactions if the human model and subsequent predictions are inaccurate. In this…
▽ More
Understanding the intentions of human teammates is critical for safe and effective human-robot interaction. The canonical approach for human-aware robot motion planning is to first predict the human's goal or path, and then construct a robot plan that avoids collision with the human. This method can generate unsafe interactions if the human model and subsequent predictions are inaccurate. In this work, we present an algorithmic approach for both arranging the configuration of objects in a shared human-robot workspace, and projecting ``virtual obstacles'' in augmented reality, optimizing for legibility in a given task. These changes to the workspace result in more legible human behavior, improving robot predictions of human goals, thereby improving task fluency and safety. To evaluate our approach, we propose two user studies involving a collaborative tabletop task with a manipulator robot, and a warehouse navigation task with a mobile robot.
△ Less
Submitted 9 November, 2023;
originally announced November 2023.
-
Writing Internet of Things Applicatations with Task-Oriented Programming
Authors:
Mart Lubbers,
Pieter Koopman,
Rinus Plasmeijer
Abstract:
The Internet of Things (IoT) is growing fast. In 2018, there was approximately one connected device per person on earth and the number has been growing ever since. The devices interact with the environment via different modalities at the same time using sensors and actuators making the programs parallel. Yet, writing this type of programs is difficult because the devices have little computation po…
▽ More
The Internet of Things (IoT) is growing fast. In 2018, there was approximately one connected device per person on earth and the number has been growing ever since. The devices interact with the environment via different modalities at the same time using sensors and actuators making the programs parallel. Yet, writing this type of programs is difficult because the devices have little computation power and memory, the platforms are heterogeneous and the languages are low level. Task Oriented Programming (TOP) is a declarative programming language paradigm that is used to express coordination of work, collaboration of users and systems, the distribution of shared data and the human-computer interaction. The mTask language is a specialized, yet full-fledged, multi-backend TOP language for IoT devices. With the bytecode interpretation backend and the integration with iTask, tasks can be executed on the device dynamically. This means that -- according to the current state of affairs -- tasks can be tailor-made at run time, compiled to device-agnostic bytecode and shipped to the device for interpretation. Tasks sent to the device are fully integrated in iTask to allow every form of interaction with the tasks such as observation of the task value and interaction with Shared Data Sources (SDSs). The entire IoT application -- both server and devices -- are programmed in a single language, albeit using two embedded Domain Specific Languages (EDSLs).
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
SusTrainable: Promoting Sustainability as a Fundamental Driver in Software Development Training and Education. Teacher Training, November 1-5, Nijmegen, The Netherlands. Revised lecture notes
Authors:
Pieter Koopman,
Mart Lubbers,
João Paulo Fernandes
Abstract:
These are the proceedings of the first teacher training of the Erasmus+ project No. 2020-1-PT01-KA203-078646 -- Sustrainable. The full title of this project is Promoting Sustainability as a Fundamental Driver in Software Development Training and Education and the interested reader may know more about it at:
https://sustrainable.github.io
The flagship contribution is the organization of two sum…
▽ More
These are the proceedings of the first teacher training of the Erasmus+ project No. 2020-1-PT01-KA203-078646 -- Sustrainable. The full title of this project is Promoting Sustainability as a Fundamental Driver in Software Development Training and Education and the interested reader may know more about it at:
https://sustrainable.github.io
The flagship contribution is the organization of two summer schools on sustainable software production. The first summer school is moved due to the Corona pandemic. It is now scheduled for July 2022 in Rijeka, Croatia. This summer school will consist of lectures and practical work for master and PhD students in the area of computing science and closely related fields. There will be contributions from Plovdiv University, University of Košice, University of Coimbra, University of Minho, Eötvös Loránd University, University of Rijeka, Radboud University Nijmegen, University of Pula, University of Amsterdam and Babeş-Bolyai University.
To prepare and streamline this summer school, the consortium organized a teacher training in Nijmegen, the Netherlands. This was an event of five full days on November 1-5, 2022 organized by Ingrid Berenbroek, Mart Lubbers and Pieter Koopman on the campus of the Radboud University Nijmegen, The Netherlands. Sustainability is one of the themes within the strategy of the Radboud University called `A Significant Impact', see https://www.ru.nl/sustainability/organisation/radboud-sustainable/. Sustainability plays an important role in education, research, and management.
The contributions were reviewed and give a good overview of what can be expected from the summer school. Based on these papers and the very positive and constructive atmosphere, we expect a high quality and pleasant summer school. We are looking forward to contributing to a sustainable future.
△ Less
Submitted 4 April, 2022;
originally announced April 2022.
-
Measuring transnational social fields through binational link-tracing sampling
Authors:
Marian-Gabriel Hâncean,
Miranda J. Lubbers,
José Luis Molina
Abstract:
We advance binational link-tracing sampling design, an innovative data collection methodology for sampling from transnational social fields, i.e., transnational networks embedding migrants and non-migrants. This paper shows the practical challenges of such a design, the representativeness of the samples and the qualities of the resulted networks. We performed 303 face-to-face structured interviews…
▽ More
We advance binational link-tracing sampling design, an innovative data collection methodology for sampling from transnational social fields, i.e., transnational networks embedding migrants and non-migrants. This paper shows the practical challenges of such a design, the representativeness of the samples and the qualities of the resulted networks. We performed 303 face-to-face structured interviews on sociodemographic variables, migration trajectories and personal networks of people living in a Romanian migration sending community (Dâmboviţa) and in a migration receiving Spanish town (Castellón), simultaneously in both sites. Inter-connecting the personal networks, we built a multi-layered complex network structure embedding 4,855 nominated people, 5,477 directed ties (nominations) and 2,540 edges. Results indicate that the participants' unique identification is a particularly difficult challenge, the representativeness of the data is not optimal (homophily on observed attributes was detected in the nomination patterns), and the relational and attribute data allow to explore the social organization of the Romanian migrant enclave in Castellón, as well as its connectivity to other places. Furthermore, we provide methodological suggestions for improving link-tracing sampling from transnational networks of migration. Our research contributes to the emerging efforts of applying social network analysis to the study of international migration.
△ Less
Submitted 19 June, 2020;
originally announced June 2020.
-
Automated Failure-Mode Clustering and Labeling for Informed Car-To-Driver Handover in Autonomous Vehicles
Authors:
Aaquib Tabrez,
Matthew B. Luebbers,
Bradley Hayes
Abstract:
The car-to-driver handover is a critically important component of safe autonomous vehicle operation when the vehicle is unable to safely proceed on its own. Current implementations of this handover in automobiles take the form of a generic alarm indicating an imminent transfer of control back to the human driver. However, certain levels of vehicle autonomy may allow the driver to engage in other,…
▽ More
The car-to-driver handover is a critically important component of safe autonomous vehicle operation when the vehicle is unable to safely proceed on its own. Current implementations of this handover in automobiles take the form of a generic alarm indicating an imminent transfer of control back to the human driver. However, certain levels of vehicle autonomy may allow the driver to engage in other, non-driving related tasks prior to a handover, leading to substantial difficulty in quickly regaining situational awareness. This delay in re-orientation could potentially lead to life-threatening failures unless mitigating steps are taken. Explainable AI has been shown to improve fluency and teamwork in human-robot collaboration scenarios. Therefore, we hypothesize that by utilizing autonomous explanation, these car-to-driver handovers can be performed more safely and reliably. The rationale is, by providing the driver with additional situational knowledge, they will more rapidly focus on the relevant parts of the driving environment. Towards this end, we propose an algorithmic failure-mode identification and explanation approach to enable informed handovers from vehicle to driver. Furthermore, we propose a set of human-subjects driving-simulator studies to determine the appropriate form of explanation during handovers, as well as validate our framework.
△ Less
Submitted 9 May, 2020;
originally announced May 2020.
