-
On the Critical Role of Conventions in Adaptive Human-AI Collaboration
Authors:
Andy Shih,
Arjun Sawhney,
Jovana Kondic,
Stefano Ermon,
Dorsa Sadigh
Abstract:
Humans can quickly adapt to new partners in collaborative tasks (e.g. playing basketball), because they understand which fundamental skills of the task (e.g. how to dribble, how to shoot) carry over across new partners. Humans can also quickly adapt to similar tasks with the same partners by carrying over conventions that they have developed (e.g. raising hand signals pass the ball), without learn…
▽ More
Humans can quickly adapt to new partners in collaborative tasks (e.g. playing basketball), because they understand which fundamental skills of the task (e.g. how to dribble, how to shoot) carry over across new partners. Humans can also quickly adapt to similar tasks with the same partners by carrying over conventions that they have developed (e.g. raising hand signals pass the ball), without learning to coordinate from scratch. To collaborate seamlessly with humans, AI agents should adapt quickly to new partners and new tasks as well. However, current approaches have not attempted to distinguish between the complexities intrinsic to a task and the conventions used by a partner, and more generally there has been little focus on leveraging conventions for adapting to new settings. In this work, we propose a learning framework that teases apart rule-dependent representation from convention-dependent representation in a principled way. We show that, under some assumptions, our rule-dependent representation is a sufficient statistic of the distribution over best-response strategies across partners. Using this separation of representations, our agents are able to adapt quickly to new partners, and to coordinate with old partners on new tasks in a zero-shot manner. We experimentally validate our approach on three collaborative tasks varying in complexity: a contextual multi-armed bandit, a block placing task, and the card game Hanabi.
△ Less
Submitted 6 April, 2021;
originally announced April 2021.
-
Causal Reasoning in Simulation for Structure and Transfer Learning of Robot Manipulation Policies
Authors:
Tabitha Edith Lee,
Jialiang Zhao,
Amrita S. Sawhney,
Siddharth Girdhar,
Oliver Kroemer
Abstract:
We present CREST, an approach for causal reasoning in simulation to learn the relevant state space for a robot manipulation policy. Our approach conducts interventions using internal models, which are simulations with approximate dynamics and simplified assumptions. These interventions elicit the structure between the state and action spaces, enabling construction of neural network policies with o…
▽ More
We present CREST, an approach for causal reasoning in simulation to learn the relevant state space for a robot manipulation policy. Our approach conducts interventions using internal models, which are simulations with approximate dynamics and simplified assumptions. These interventions elicit the structure between the state and action spaces, enabling construction of neural network policies with only relevant states as input. These policies are pretrained using the internal model with domain randomization over the relevant states. The policy network weights are then transferred to the target domain (e.g., the real world) for fine tuning. We perform extensive policy transfer experiments in simulation for two representative manipulation tasks: block stacking and crate opening. Our policies are shown to be more robust to domain shifts, more sample efficient to learn, and scale to more complex settings with larger state spaces. We also show improved zero-shot sim-to-real transfer of our policies for the block stacking task.
△ Less
Submitted 13 March, 2022; v1 submitted 30 March, 2021;
originally announced March 2021.
-
Playing with Food: Learning Food Item Representations through Interactive Exploration
Authors:
Amrita Sawhney,
Steven Lee,
Kevin Zhang,
Manuela Veloso,
Oliver Kroemer
Abstract:
A key challenge in robotic food manipulation is modeling the material properties of diverse and deformable food items. We propose using a multimodal sensory approach to interact and play with food that facilitates the ability to distinguish these properties across food items. First, we use a robotic arm and an array of sensors, which are synchronized using ROS, to collect a diverse dataset consist…
▽ More
A key challenge in robotic food manipulation is modeling the material properties of diverse and deformable food items. We propose using a multimodal sensory approach to interact and play with food that facilitates the ability to distinguish these properties across food items. First, we use a robotic arm and an array of sensors, which are synchronized using ROS, to collect a diverse dataset consisting of 21 unique food items with varying slices and properties. Afterwards, we learn visual embedding networks that utilize a combination of proprioceptive, audio, and visual data to encode similarities among food items using a triplet loss formulation. Our evaluations show that embeddings learned through interactions can successfully increase performance in a wide range of material and shape classification tasks. We envision that these learned embeddings can be utilized as a basis for planning and selecting optimal parameters for more material-aware robotic food manipulation skills. Furthermore, we hope to stimulate further innovations in the field of food robotics by sharing this food playing dataset with the research community.
△ Less
Submitted 6 January, 2021;
originally announced January 2021.
-
Analyzing Offline Social Engagements: An Empirical Study of Meetup Events Related to Software Development
Authors:
Abhishek Sharma,
Gede Artha Azriadi Prana,
Anamika Sawhney,
Nachiappan Nagappan,
David Lo
Abstract:
Software developers use a variety of social media channels and tools in order to keep themselves up to date, collaborate with other developers, and find projects to contribute to. Meetup is one of such social media used by software developers to organize community gatherings. Liu et al. characterized Meetup as an event-based social network (EBSN) which contains valuable offline social interactions…
▽ More
Software developers use a variety of social media channels and tools in order to keep themselves up to date, collaborate with other developers, and find projects to contribute to. Meetup is one of such social media used by software developers to organize community gatherings. Liu et al. characterized Meetup as an event-based social network (EBSN) which contains valuable offline social interactions in addition to online interactions. Recently, Storey et al. found out that Meetup was one of the social channels used by developers. We in this work investigate in detail the dynamics of Meetup groups and events related to software development, which has not been done in any of the previous works.
First, we identified 6,317 Meetup groups related to software development and extracted 185,758 events organized by them. Then we took a statistically significant sample of 452 events on which we performed open coding, based on which we were able to develop 9 categories of events (8 main categories + Others). Next, we did a popularity analysis of the categories of events and found that Talks by Domain Experts, Hands-on Sessions, and Open Discussions are the most popular categories of events organized by Meetup groups related to software development. Our findings show that more popular categories are those where developers can learn and gain knowledge. On doing a diversity analysis of Meetup groups we found 19.82% of the members on an average are female, which is a larger proportion as compared to numbers reported in previous studies on other social media. From a broader software development community point of view information from this new forum can be valuable to identify and understand emerging topics and associations among them which can be helpful to identify future trends as well as current best practices.
△ Less
Submitted 16 December, 2019;
originally announced December 2019.
