-
Ergodic Generative Flows
Authors:
Leo Maxime Brunswic,
Mateo Clemente,
Rui Heng Yang,
Adam Sigal,
Amir Rasouli,
Yinchuan Li
Abstract:
Generative Flow Networks (GFNs) were initially introduced on directed acyclic graphs to sample from an unnormalized distribution density. Recent works have extended the theoretical framework for generative methods allowing more flexibility and enhancing application range. However, many challenges remain in training GFNs in continuous settings and for imitation learning (IL), including intractabili…
▽ More
Generative Flow Networks (GFNs) were initially introduced on directed acyclic graphs to sample from an unnormalized distribution density. Recent works have extended the theoretical framework for generative methods allowing more flexibility and enhancing application range. However, many challenges remain in training GFNs in continuous settings and for imitation learning (IL), including intractability of flow-matching loss, limited tests of non-acyclic training, and the need for a separate reward model in imitation learning. The present work proposes a family of generative flows called Ergodic Generative Flows (EGFs) which are used to address the aforementioned issues. First, we leverage ergodicity to build simple generative flows with finitely many globally defined transformations (diffeomorphisms) with universality guarantees and tractable flow-matching loss (FM loss). Second, we introduce a new loss involving cross-entropy coupled to weak flow-matching control, coined KL-weakFM loss. It is designed for IL training without a separate reward model. We evaluate IL-EGFs on toy 2D tasks and real-world datasets from NASA on the sphere, using the KL-weakFM loss. Additionally, we conduct toy 2D reinforcement learning experiments with a target reward, using the FM loss.
△ Less
Submitted 6 May, 2025;
originally announced May 2025.
-
Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensor
Authors:
Trevor Ablett,
Oliver Limoyo,
Adam Sigal,
Affan Jilani,
Jonathan Kelly,
Kaleem Siddiqi,
Francois Hogan,
Gregory Dudek
Abstract:
Contact-rich tasks continue to present many challenges for robotic manipulation. In this work, we leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact-rich tasks that involve relative motion (e.g., slipping and sliding) between the end-effector and the manipulated object. We introduce two algorithmic contributions, tactile force matching and…
▽ More
Contact-rich tasks continue to present many challenges for robotic manipulation. In this work, we leverage a multimodal visuotactile sensor within the framework of imitation learning (IL) to perform contact-rich tasks that involve relative motion (e.g., slipping and sliding) between the end-effector and the manipulated object. We introduce two algorithmic contributions, tactile force matching and learned mode switching, as complimentary methods for improving IL. Tactile force matching enhances kinesthetic teaching by reading approximate forces during the demonstration and generating an adapted robot trajectory that recreates the recorded forces. Learned mode switching uses IL to couple visual and tactile sensor modes with the learned motion policy, simplifying the transition from reaching to contacting. We perform robotic manipulation experiments on four door-opening tasks with a variety of observation and algorithm configurations to study the utility of multimodal visuotactile sensing and our proposed improvements. Our results show that the inclusion of force matching raises average policy success rates by 62.5%, visuotactile mode switching by 30.3%, and visuotactile data as a policy input by 42.5%, emphasizing the value of see-through tactile sensing for IL, both for data collection to allow force matching, and for policy execution to enable accurate task feedback. Project site: https://papers.starslab.ca/sts-il/
△ Less
Submitted 26 January, 2025; v1 submitted 2 November, 2023;
originally announced November 2023.
-
SAGE: Smart home Agent with Grounded Execution
Authors:
Dmitriy Rivkin,
Francois Hogan,
Amal Feriani,
Abhisek Konar,
Adam Sigal,
Steve Liu,
Greg Dudek
Abstract:
The common sense reasoning abilities and vast general knowledge of Large Language Models (LLMs) make them a natural fit for interpreting user requests in a Smart Home assistant context. LLMs, however, lack specific knowledge about the user and their home limit their potential impact. SAGE (Smart Home Agent with Grounded Execution), overcomes these and other limitations by using a scheme in which a…
▽ More
The common sense reasoning abilities and vast general knowledge of Large Language Models (LLMs) make them a natural fit for interpreting user requests in a Smart Home assistant context. LLMs, however, lack specific knowledge about the user and their home limit their potential impact. SAGE (Smart Home Agent with Grounded Execution), overcomes these and other limitations by using a scheme in which a user request triggers an LLM-controlled sequence of discrete actions. These actions can be used to retrieve information, interact with the user, or manipulate device states. SAGE controls this process through a dynamically constructed tree of LLM prompts, which help it decide which action to take next, whether an action was successful, and when to terminate the process. The SAGE action set augments an LLM's capabilities to support some of the most critical requirements for a Smart Home assistant. These include: flexible and scalable user preference management ("is my team playing tonight?"), access to any smart device's full functionality without device-specific code via API reading "turn down the screen brightness on my dryer", persistent device state monitoring ("remind me to throw out the milk when I open the fridge"), natural device references using only a photo of the room ("turn on the light on the dresser"), and more. We introduce a benchmark of 50 new and challenging smart home tasks where SAGE achieves a 75% success rate, significantly outperforming existing LLM-enabled baselines (30% success rate).
△ Less
Submitted 19 January, 2024; v1 submitted 1 November, 2023;
originally announced November 2023.
-
Improving Generalization in Reinforcement Learning Training Regimes for Social Robot Navigation
Authors:
Adam Sigal,
Hsiu-Chin Lin,
AJung Moon
Abstract:
In order for autonomous mobile robots to navigate in human spaces, they must abide by our social norms. Reinforcement learning (RL) has emerged as an effective method to train sequential decision-making policies that are able to respect these norms. However, a large portion of existing work in the field conducts both RL training and testing in simplistic environments. This limits the generalizatio…
▽ More
In order for autonomous mobile robots to navigate in human spaces, they must abide by our social norms. Reinforcement learning (RL) has emerged as an effective method to train sequential decision-making policies that are able to respect these norms. However, a large portion of existing work in the field conducts both RL training and testing in simplistic environments. This limits the generalization potential of these models to unseen environments, and the meaningfulness of their reported results. We propose a method to improve the generalization performance of RL social navigation methods using curriculum learning. By employing multiple environment types and by modeling pedestrians using multiple dynamics models, we are able to progressively diversify and escalate difficulty in training. Our results show that the use of curriculum learning in training can be used to achieve better generalization performance than previous training methods. We also show that results presented in many existing state-of-the-art RL social navigation works do not evaluate their methods outside of their training environments, and thus do not reflect their policies' failure to adequately generalize to out-of-distribution scenarios. In response, we validate our training approach on larger and more crowded testing environments than those used in training, allowing for more meaningful measurements of model performance.
△ Less
Submitted 28 February, 2024; v1 submitted 28 August, 2023;
originally announced August 2023.
-
Collective and single cell behavior in epithelial contact inhibition
Authors:
Alberto Puliafito,
Lars Hufnagel,
Pierre Neveu,
Sebastian Streichan,
Alex Sigal,
Deborah K. Fygenson,
Boris I. Shraiman
Abstract:
Control of cell proliferation is a fundamental aspect of tissue physiology central to morphogenesis, wound healing and cancer. Although many of the molecular genetic factors are now known, the system level regulation of growth is still poorly understood. A simple form of inhibition of cell proliferation is encountered in vitro in normally differentiating epithelial cell cultures and is known as "c…
▽ More
Control of cell proliferation is a fundamental aspect of tissue physiology central to morphogenesis, wound healing and cancer. Although many of the molecular genetic factors are now known, the system level regulation of growth is still poorly understood. A simple form of inhibition of cell proliferation is encountered in vitro in normally differentiating epithelial cell cultures and is known as "contact inhibition". The study presented here provides a quantitative characterization of contact inhibition dynamics on tissue-wide and single cell levels. Using long-term tracking of cultured MDCK cells we demonstrate that inhibition of cell division in a confluent monolayer follows inhibition of cell motility and sets in when mechanical constraint on local expansion causes divisions to reduce cell area. We quantify cell motility and cell cycle statistics in the low density confluent regime and their change across the transition to epithelial morphology which occurs with increasing cell density. We then study the dynamics of cell area distribution arising through reductive division, determine the average mitotic rate as a function of cell size and demonstrate that complete arrest of mitosis occurs when cell area falls below a critical value. We also present a simple computational model of growth mechanics which captures all aspects of the observed behavior. Our measurements and analysis show that contact inhibition is a consequence of mechanical interaction and constraint rather than interfacial contact alone, and define quantitative phenotypes that can guide future studies of molecular mechanisms underlying contact inhibition.
△ Less
Submitted 2 December, 2011;
originally announced December 2011.
