Showing 1–50 of 59 results for author: Nori, F

A Survey on Imitation Learning for Contact-Rich Tasks in Robotics

Authors: Toshiaki Tsuji, Yasuhiro Kato, Gokhan Solak, Heng Zhang, Tadej Petrič, Francesco Nori, Arash Ajoudani

Abstract: This paper comprehensively surveys research trends in imitation learning for contact-rich robotic tasks. Contact-rich tasks, which require complex physical interactions with the environment, represent a central challenge in robotics due to their nonlinear dynamics and sensitivity to small positional deviations. The paper examines demonstration collection methodologies, including teaching methods a… ▽ More This paper comprehensively surveys research trends in imitation learning for contact-rich robotic tasks. Contact-rich tasks, which require complex physical interactions with the environment, represent a central challenge in robotics due to their nonlinear dynamics and sensitivity to small positional deviations. The paper examines demonstration collection methodologies, including teaching methods and sensory modalities crucial for capturing subtle interaction dynamics. We then analyze imitation learning approaches, highlighting their applications to contact-rich manipulation. Recent advances in multimodal learning and foundation models have significantly enhanced performance in complex contact tasks across industrial, household, and healthcare domains. Through systematic organization of current research and identification of challenges, this survey provides a foundation for future advancements in contact-rich robotic manipulation. △ Less

Submitted 16 June, 2025; originally announced June 2025.

Comments: 47pages, 1 figures

ExoStart: Efficient learning for dexterous manipulation with sensorized exoskeleton demonstrations

Authors: Zilin Si, Jose Enrique Chen, M. Emre Karagozler, Antonia Bronars, Jonathan Hutchinson, Thomas Lampe, Nimrod Gileadi, Taylor Howell, Stefano Saliceti, Lukasz Barczyk, Ilan Olivarez Correa, Tom Erez, Mohit Shridhar, Murilo Fernandes Martins, Konstantinos Bousmalis, Nicolas Heess, Francesco Nori, Maria Bauza Villalonga

Abstract: Recent advancements in teleoperation systems have enabled high-quality data collection for robotic manipulators, showing impressive results in learning manipulation at scale. This progress suggests that extending these capabilities to robotic hands could unlock an even broader range of manipulation skills, especially if we could achieve the same level of dexterity that human hands exhibit. However… ▽ More Recent advancements in teleoperation systems have enabled high-quality data collection for robotic manipulators, showing impressive results in learning manipulation at scale. This progress suggests that extending these capabilities to robotic hands could unlock an even broader range of manipulation skills, especially if we could achieve the same level of dexterity that human hands exhibit. However, teleoperating robotic hands is far from a solved problem, as it presents a significant challenge due to the high degrees of freedom of robotic hands and the complex dynamics occurring during contact-rich settings. In this work, we present ExoStart, a general and scalable learning framework that leverages human dexterity to improve robotic hand control. In particular, we obtain high-quality data by collecting direct demonstrations without a robot in the loop using a sensorized low-cost wearable exoskeleton, capturing the rich behaviors that humans can demonstrate with their own hands. We also propose a simulation-based dynamics filter that generates dynamically feasible trajectories from the collected demonstrations and use the generated trajectories to bootstrap an auto-curriculum reinforcement learning method that relies only on simple sparse rewards. The ExoStart pipeline is generalizable and yields robust policies that transfer zero-shot to the real robot. Our results demonstrate that ExoStart can generate dexterous real-world hand skills, achieving a success rate above 50% on a wide range of complex tasks such as opening an AirPods case or inserting and turning a key in a lock. More details and videos can be found in https://sites.google.com/view/exostart. △ Less

Submitted 13 June, 2025; originally announced June 2025.

Gemini Robotics: Bringing AI into the Physical World

Authors: Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, Steven Bohez, Konstantinos Bousmalis, Anthony Brohan, Thomas Buschmann, Arunkumar Byravan, Serkan Cabi, Ken Caluwaerts, Federico Casarini, Oscar Chang, Jose Enrique Chen, Xi Chen, Hao-Tien Lewis Chiang, Krzysztof Choromanski, David D'Ambrosio, Sudeep Dasari , et al. (93 additional authors not shown)

Abstract: Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as robots remains a significant challenge. This report introduces a new family of AI models purposefully designed for robotics and built upon the foundation of Gemini 2.0. We present Gemini Robotics, an advanced Vision-Lang… ▽ More Recent advancements in large multimodal models have led to the emergence of remarkable generalist capabilities in digital domains, yet their translation to physical agents such as robots remains a significant challenge. This report introduces a new family of AI models purposefully designed for robotics and built upon the foundation of Gemini 2.0. We present Gemini Robotics, an advanced Vision-Language-Action (VLA) generalist model capable of directly controlling robots. Gemini Robotics executes smooth and reactive movements to tackle a wide range of complex manipulation tasks while also being robust to variations in object types and positions, handling unseen environments as well as following diverse, open vocabulary instructions. We show that with additional fine-tuning, Gemini Robotics can be specialized to new capabilities including solving long-horizon, highly dexterous tasks, learning new short-horizon tasks from as few as 100 demonstrations and adapting to completely novel robot embodiments. This is made possible because Gemini Robotics builds on top of the Gemini Robotics-ER model, the second model we introduce in this work. Gemini Robotics-ER (Embodied Reasoning) extends Gemini's multimodal reasoning capabilities into the physical world, with enhanced spatial and temporal understanding. This enables capabilities relevant to robotics including object detection, pointing, trajectory and grasp prediction, as well as multi-view correspondence and 3D bounding box predictions. We show how this novel combination can support a variety of robotics applications. We also discuss and address important safety considerations related to this new class of robotics foundation models. The Gemini Robotics family marks a substantial step towards developing general-purpose robots that realizes AI's potential in the physical world. △ Less

Submitted 25 March, 2025; originally announced March 2025.

Proc4Gem: Foundation models for physical agency through procedural generation

Authors: Yixin Lin, Jan Humplik, Sandy H. Huang, Leonard Hasenclever, Francesco Romano, Stefano Saliceti, Daniel Zheng, Jose Enrique Chen, Catarina Barros, Adrian Collister, Matt Young, Adil Dostmohamed, Ben Moran, Ken Caluwaerts, Marissa Giustina, Joss Moore, Kieran Connell, Francesco Nori, Nicolas Heess, Steven Bohez, Arunkumar Byravan

Abstract: In robot learning, it is common to either ignore the environment semantics, focusing on tasks like whole-body control which only require reasoning about robot-environment contacts, or conversely to ignore contact dynamics, focusing on grounding high-level movement in vision and language. In this work, we show that advances in generative modeling, photorealistic rendering, and procedural generation… ▽ More In robot learning, it is common to either ignore the environment semantics, focusing on tasks like whole-body control which only require reasoning about robot-environment contacts, or conversely to ignore contact dynamics, focusing on grounding high-level movement in vision and language. In this work, we show that advances in generative modeling, photorealistic rendering, and procedural generation allow us to tackle tasks requiring both. By generating contact-rich trajectories with accurate physics in semantically-diverse simulations, we can distill behaviors into large multimodal models that directly transfer to the real world: a system we call Proc4Gem. Specifically, we show that a foundation model, Gemini, fine-tuned on only simulation data, can be instructed in language to control a quadruped robot to push an object with its body to unseen targets in unseen real-world environments. Our real-world results demonstrate the promise of using simulation to imbue foundation models with physical agency. Videos can be found at our website: https://sites.google.com/view/proc4gem △ Less

Submitted 11 March, 2025; originally announced March 2025.

Data-driven model reconstruction for nonlinear wave dynamics

Authors: Ekaterina Smolina, Lev Smirnov, Daniel Leykam, Franco Nori, Daria Smirnova

Abstract: The use of machine learning to predict wave dynamics is a topic of growing interest, but commonly-used deep learning approaches suffer from a lack of interpretability of the trained models. Here we present an interpretable machine learning framework for analyzing the nonlinear evolution dynamics of optical wavepackets in complex wave media. We use sparse regression to reduce microscopic discrete l… ▽ More The use of machine learning to predict wave dynamics is a topic of growing interest, but commonly-used deep learning approaches suffer from a lack of interpretability of the trained models. Here we present an interpretable machine learning framework for analyzing the nonlinear evolution dynamics of optical wavepackets in complex wave media. We use sparse regression to reduce microscopic discrete lattice models to simpler effective continuum models which can accurately describe the dynamics of the wavepacket envelope. We apply our approach to valley-Hall domain walls in honeycomb photonic lattices of laser-written waveguides with Kerr-type nonlinearity and different boundary shapes. The reconstructed equations accurately reproduce the linear dispersion and nonlinear effects including self-steepening and self-focusing. This scheme is proven free of the a priori limitations imposed by the underlying hierarchy of scales traditionally employed in asymptotic analytical methods. It represents a powerful interpretable machine learning technique of interest for advancing design capabilities in photonics and framing the complex interaction-driven dynamics in various topological materials. △ Less

Submitted 18 November, 2024; originally announced November 2024.

Comments: 6 pages, 5 figures

DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots

Authors: Maria Bauza, Jose Enrique Chen, Valentin Dalibard, Nimrod Gileadi, Roland Hafner, Murilo F. Martins, Joss Moore, Rugile Pevceviciute, Antoine Laurens, Dushyant Rao, Martina Zambelli, Martin Riedmiller, Jon Scholz, Konstantinos Bousmalis, Francesco Nori, Nicolas Heess

Abstract: We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to a… ▽ More We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to achieve successful zero-shot sim-to-real transfer. Transferred policies are learned directly from raw pixels from multiple cameras and robot proprioception. Our approach outperforms policies learned from demonstrations on the real robot and requires 100 times fewer demonstrations, collected in simulation. More details and videos in https://sites.google.com/view/demostart. △ Less

Submitted 12 September, 2024; v1 submitted 10 September, 2024; originally announced September 2024.

Comments: 15 pages total with 7 pages of appendix. 9 Figures, 4 in the main text and 5 in the appendix

Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learning

Authors: Dhruva Tirumala, Markus Wulfmeier, Ben Moran, Sandy Huang, Jan Humplik, Guy Lever, Tuomas Haarnoja, Leonard Hasenclever, Arunkumar Byravan, Nathan Batchelor, Neil Sreendra, Kushal Patel, Marlon Gwira, Francesco Nori, Martin Riedmiller, Nicolas Heess

Abstract: We apply multi-agent deep reinforcement learning (RL) to train end-to-end robot soccer policies with fully onboard computation and sensing via egocentric RGB vision. This setting reflects many challenges of real-world robotics, including active perception, agile full-body control, and long-horizon planning in a dynamic, partially-observable, multi-agent domain. We rely on large-scale, simulation-b… ▽ More We apply multi-agent deep reinforcement learning (RL) to train end-to-end robot soccer policies with fully onboard computation and sensing via egocentric RGB vision. This setting reflects many challenges of real-world robotics, including active perception, agile full-body control, and long-horizon planning in a dynamic, partially-observable, multi-agent domain. We rely on large-scale, simulation-based data generation to obtain complex behaviors from egocentric vision which can be successfully transferred to physical robots using low-cost sensors. To achieve adequate visual realism, our simulation combines rigid-body physics with learned, realistic rendering via multiple Neural Radiance Fields (NeRFs). We combine teacher-based multi-agent RL and cross-experiment data reuse to enable the discovery of sophisticated soccer strategies. We analyze active-perception behaviors including object tracking and ball seeking that emerge when simply optimizing perception-agnostic soccer play. The agents display equivalent levels of performance and agility as policies with access to privileged, ground-truth state. To our knowledge, this paper constitutes a first demonstration of end-to-end training for multi-agent robot soccer, mapping raw pixel observations to joint-level actions, that can be deployed in the real world. Videos of the game-play and analyses can be seen on our website https://sites.google.com/view/vision-soccer . △ Less

Submitted 3 May, 2024; originally announced May 2024.

Mastering Stacking of Diverse Shapes with Large-Scale Iterative Reinforcement Learning on Real Robots

Authors: Thomas Lampe, Abbas Abdolmaleki, Sarah Bechtle, Sandy H. Huang, Jost Tobias Springenberg, Michael Bloesch, Oliver Groth, Roland Hafner, Tim Hertweck, Michael Neunert, Markus Wulfmeier, Jingwei Zhang, Francesco Nori, Nicolas Heess, Martin Riedmiller

Abstract: Reinforcement learning solely from an agent's self-generated data is often believed to be infeasible for learning on real robots, due to the amount of data needed. However, if done right, agents learning from real data can be surprisingly efficient through re-using previously collected sub-optimal data. In this paper we demonstrate how the increased understanding of off-policy learning methods and… ▽ More Reinforcement learning solely from an agent's self-generated data is often believed to be infeasible for learning on real robots, due to the amount of data needed. However, if done right, agents learning from real data can be surprisingly efficient through re-using previously collected sub-optimal data. In this paper we demonstrate how the increased understanding of off-policy learning methods and their embedding in an iterative online/offline scheme (``collect and infer'') can drastically improve data-efficiency by using all the collected experience, which empowers learning from real robot experience only. Moreover, the resulting policy improves significantly over the state of the art on a recently proposed real robot manipulation benchmark. Our approach learns end-to-end, directly from pixels, and does not rely on additional human domain knowledge such as a simulator or demonstrations. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Identifying topology of leaky photonic lattices with machine learning

Authors: Ekaterina O. Smolina, Lev A. Smirnov, Daniel Leykam, Franco Nori, Daria A. Smirnova

Abstract: We show how machine learning techniques can be applied for the classification of topological phases in leaky photonic lattices using limited measurement data. We propose an approach based solely on bulk intensity measurements, thus exempt from the need for complicated phase retrieval procedures. In particular, we design a fully connected neural network that accurately determines topological proper… ▽ More We show how machine learning techniques can be applied for the classification of topological phases in leaky photonic lattices using limited measurement data. We propose an approach based solely on bulk intensity measurements, thus exempt from the need for complicated phase retrieval procedures. In particular, we design a fully connected neural network that accurately determines topological properties from the output intensity distribution in dimerized waveguide arrays with leaky channels, after propagation of a spatially localized initial excitation at a finite distance, in a setting that closely emulates realistic experimental conditions. △ Less

Submitted 28 August, 2023; originally announced August 2023.

Comments: 9 pages, 8 figures

RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation

Authors: Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Coline Devin, Alex X. Lee, Maria Bauza, Todor Davchev, Yuxiang Zhou, Agrim Gupta, Akhil Raju, Antoine Laurens, Claudio Fantacci, Valentin Dalibard, Martina Zambelli, Murilo Martins, Rugile Pevceviciute, Michiel Blokzijl, Misha Denil, Nathan Batchelor, Thomas Lampe, Emilio Parisotto, Konrad Żołna, Scott Reed, Sergio Gómez Colmenarejo, Jon Scholz , et al. (14 additional authors not shown)

Abstract: The ability to leverage heterogeneous robotic experience from different robots and tasks to quickly master novel skills and embodiments has the potential to transform robot learning. Inspired by recent advances in foundation models for vision and language, we propose a multi-embodiment, multi-task generalist agent for robotic manipulation. This agent, named RoboCat, is a visual goal-conditioned de… ▽ More The ability to leverage heterogeneous robotic experience from different robots and tasks to quickly master novel skills and embodiments has the potential to transform robot learning. Inspired by recent advances in foundation models for vision and language, we propose a multi-embodiment, multi-task generalist agent for robotic manipulation. This agent, named RoboCat, is a visual goal-conditioned decision transformer capable of consuming action-labelled visual experience. This data spans a large repertoire of motor control skills from simulated and real robotic arms with varying sets of observations and actions. With RoboCat, we demonstrate the ability to generalise to new tasks and robots, both zero-shot as well as through adaptation using only 100-1000 examples for the target task. We also show how a trained model itself can be used to generate data for subsequent training iterations, thus providing a basic building block for an autonomous improvement loop. We investigate the agent's capabilities, with large-scale evaluations both in simulation and on three different real robot embodiments. We find that as we grow and diversify its training data, RoboCat not only shows signs of cross-task transfer, but also becomes more efficient at adapting to new tasks. △ Less

Submitted 22 December, 2023; v1 submitted 20 June, 2023; originally announced June 2023.

Comments: Transactions on Machine Learning Research (12/2023)

Barkour: Benchmarking Animal-level Agility with Quadruped Robots

Authors: Ken Caluwaerts, Atil Iscen, J. Chase Kew, Wenhao Yu, Tingnan Zhang, Daniel Freeman, Kuang-Huei Lee, Lisa Lee, Stefano Saliceti, Vincent Zhuang, Nathan Batchelor, Steven Bohez, Federico Casarini, Jose Enrique Chen, Omar Cortes, Erwin Coumans, Adil Dostmohamed, Gabriel Dulac-Arnold, Alejandro Escontrela, Erik Frey, Roland Hafner, Deepali Jain, Bauyrjan Jyenis, Yuheng Kuang, Edward Lee , et al. (19 additional authors not shown)

Abstract: Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biological counterparts and show various agile skills to navigate complex environments quickly. Despite the interest, the field lacks systematic benchmarks to measure the performance of control policies and hardware in agili… ▽ More Animals have evolved various agile locomotion strategies, such as sprinting, leaping, and jumping. There is a growing interest in developing legged robots that move like their biological counterparts and show various agile skills to navigate complex environments quickly. Despite the interest, the field lacks systematic benchmarks to measure the performance of control policies and hardware in agility. We introduce the Barkour benchmark, an obstacle course to quantify agility for legged robots. Inspired by dog agility competitions, it consists of diverse obstacles and a time based scoring mechanism. This encourages researchers to develop controllers that not only move fast, but do so in a controllable and versatile way. To set strong baselines, we present two methods for tackling the benchmark. In the first approach, we train specialist locomotion skills using on-policy reinforcement learning methods and combine them with a high-level navigation controller. In the second approach, we distill the specialist skills into a Transformer-based generalist locomotion policy, named Locomotion-Transformer, that can handle various terrains and adjust the robot's gait based on the perceived environment and robot states. Using a custom-built quadruped robot, we demonstrate that our method can complete the course at half the speed of a dog. We hope that our work represents a step towards creating controllers that enable robots to reach animal-level agility. △ Less

Submitted 23 May, 2023; originally announced May 2023.

Comments: 17 pages, 19 figures

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

Authors: Tuomas Haarnoja, Ben Moran, Guy Lever, Sandy H. Huang, Dhruva Tirumala, Jan Humplik, Markus Wulfmeier, Saran Tunyasuvunakool, Noah Y. Siegel, Roland Hafner, Michael Bloesch, Kristian Hartikainen, Arunkumar Byravan, Leonard Hasenclever, Yuval Tassa, Fereshteh Sadeghi, Nathan Batchelor, Federico Casarini, Stefano Saliceti, Charles Game, Neil Sreendra, Kushal Patel, Marlon Gwira, Andrea Huber, Nicole Hurley , et al. (3 additional authors not shown)

Abstract: We investigate whether Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies in dynamic environments. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. The resulting agent exhibits robust… ▽ More We investigate whether Deep Reinforcement Learning (Deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies in dynamic environments. We used Deep RL to train a humanoid robot with 20 actuated joints to play a simplified one-versus-one (1v1) soccer game. The resulting agent exhibits robust and dynamic movement skills such as rapid fall recovery, walking, turning, kicking and more; and it transitions between them in a smooth, stable, and efficient manner. The agent's locomotion and tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. The agent also developed a basic strategic understanding of the game, and learned, for instance, to anticipate ball movements and to block opponent shots. Our agent was trained in simulation and transferred to real robots zero-shot. We found that a combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training in simulation enabled good-quality transfer. Although the robots are inherently fragile, basic regularization of the behavior during training led the robots to learn safe and effective movements while still performing in a dynamic and agile way -- well beyond what is intuitively expected from the robot. Indeed, in experiments, they walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline, while efficiently combining the skills to achieve the longer term objectives. △ Less

Submitted 11 April, 2024; v1 submitted 26 April, 2023; originally announced April 2023.

Comments: Project website: https://sites.google.com/view/op3-soccer

Exponentially improved efficient machine learning for quantum many-body states with provable guarantees

Authors: Yanming Che, Clemens Gneiting, Franco Nori

Abstract: Solving the ground state and the ground-state properties of quantum many-body systems is generically a hard task for classical algorithms. For a family of Hamiltonians defined on an $m$-dimensional space of physical parameters, the ground state and its properties at an arbitrary parameter configuration can be predicted via a machine learning protocol up to a prescribed prediction error… ▽ More Solving the ground state and the ground-state properties of quantum many-body systems is generically a hard task for classical algorithms. For a family of Hamiltonians defined on an $m$-dimensional space of physical parameters, the ground state and its properties at an arbitrary parameter configuration can be predicted via a machine learning protocol up to a prescribed prediction error $\varepsilon$, provided that a sample set (of size $N$) of the states can be efficiently prepared and measured. In a recent work [Huang et al., Science 377, eabk3333 (2022)], a rigorous guarantee for such a generalization was proved. Unfortunately, an exponential scaling for the provable sample complexity, $N=m^{\cal{O}\left(\frac{1}{\varepsilon}\right)}$, was found to be universal for generic gapped Hamiltonians. This result applies to the situation where the dimension of the parameter space is large while the scaling with the accuracy is not an urgent factor. In this work, we consider an alternative scenario where $m$ is a finite, not necessarily large constant while the scaling with the prediction error becomes the central concern. By jointly preserving the fundamental properties of density matrices in the learning protocol and utilizing the continuity of quantum states in the parameter range of interest, we rigorously obtain a polynomial sample complexity for predicting quantum many-body states and their properties, with respect to the uniform prediction error $\varepsilon$ and the number of qubits $n$. Moreover, if restricted to learning local quantum-state properties, the number of samples with respect to $n$ can be further reduced exponentially. Our results provide theoretical guarantees for efficient learning of quantum many-body states and their properties, with model-independent applications not restricted to ground states of gapped Hamiltonians. △ Less

Submitted 12 August, 2024; v1 submitted 9 April, 2023; originally announced April 2023.

Comments: Published on Physical Review Research 6, 033035 (2024)

NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fields

Authors: Arunkumar Byravan, Jan Humplik, Leonard Hasenclever, Arthur Brussee, Francesco Nori, Tuomas Haarnoja, Ben Moran, Steven Bohez, Fereshteh Sadeghi, Bojan Vujatovic, Nicolas Heess

Abstract: We present a system for applying sim2real approaches to "in the wild" scenes with realistic visuals, and to policies which rely on active perception using RGB cameras. Given a short video of a static scene collected using a generic phone, we learn the scene's contact geometry and a function for novel view synthesis using a Neural Radiance Field (NeRF). We augment the NeRF rendering of the static s… ▽ More We present a system for applying sim2real approaches to "in the wild" scenes with realistic visuals, and to policies which rely on active perception using RGB cameras. Given a short video of a static scene collected using a generic phone, we learn the scene's contact geometry and a function for novel view synthesis using a Neural Radiance Field (NeRF). We augment the NeRF rendering of the static scene by overlaying the rendering of other dynamic objects (e.g. the robot's own body, a ball). A simulation is then created using the rendering engine in a physics simulator which computes contact dynamics from the static scene geometry (estimated from the NeRF volume density) and the dynamic objects' geometry and physical properties (assumed known). We demonstrate that we can use this simulation to learn vision-based whole body navigation and ball pushing policies for a 20 degrees of freedom humanoid robot with an actuated head-mounted RGB camera, and we successfully transfer these policies to a real robot. Project video is available at https://sites.google.com/view/nerf2real/home △ Less

Submitted 10 October, 2022; originally announced October 2022.

Estimating the Euclidean quantum propagator with deep generative modeling of Feynman paths

Authors: Yanming Che, Clemens Gneiting, Franco Nori

Abstract: Feynman path integrals provide an elegant, classically inspired representation for the quantum propagator and the quantum dynamics, through summing over a huge manifold of all possible paths. From computational and simulational perspectives, the ergodic tracking of the whole path manifold is a hard problem. Machine learning can help, in an efficient manner, to identify the relevant subspace and th… ▽ More Feynman path integrals provide an elegant, classically inspired representation for the quantum propagator and the quantum dynamics, through summing over a huge manifold of all possible paths. From computational and simulational perspectives, the ergodic tracking of the whole path manifold is a hard problem. Machine learning can help, in an efficient manner, to identify the relevant subspace and the intrinsic structure residing at a small fraction of the vast path manifold. In this work, we propose the Feynman path generator for quantum mechanical systems, which efficiently generates Feynman paths with fixed endpoints, from a (low-dimensional) latent space and by targeting a desired density of paths in the Euclidean space-time. With such path generators, the Euclidean propagator as well as the ground-state wave function can be estimated efficiently for a generic potential energy. Our work provides an alternative approach for calculating the quantum propagator and the ground-state wave function, paves the way toward generative modeling of quantum mechanical Feynman paths, and offers a different perspective to understand the quantum-classical correspondence through deep learning. △ Less

Submitted 18 June, 2022; v1 submitted 6 February, 2022; originally announced February 2022.

Comments: Published version; Updated references

Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes

Authors: Alex X. Lee, Coline Devin, Yuxiang Zhou, Thomas Lampe, Konstantinos Bousmalis, Jost Tobias Springenberg, Arunkumar Byravan, Abbas Abdolmaleki, Nimrod Gileadi, David Khosid, Claudio Fantacci, Jose Enrique Chen, Akhil Raju, Rae Jeong, Michael Neunert, Antoine Laurens, Stefano Saliceti, Federico Casarini, Martin Riedmiller, Raia Hadsell, Francesco Nori

Abstract: We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategies beyond a simple "pick-and-place" solution. Our method is a reinforcement learning (RL) approach combined with vision-based interactive policy distillation and simulation-to-reality transfer. Our learned policies can ef… ▽ More We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of such objects that was carefully designed to require strategies beyond a simple "pick-and-place" solution. Our method is a reinforcement learning (RL) approach combined with vision-based interactive policy distillation and simulation-to-reality transfer. Our learned policies can efficiently handle multiple object combinations in the real world and exhibit a large variety of stacking skills. In a large experimental study, we investigate what choices matter for learning such general vision-based agents in simulation, and what affects optimal transfer to the real robot. We then leverage data collected by such policies and improve upon them with offline RL. A video and a blog post of our work are provided as supplementary material. △ Less

Submitted 3 November, 2021; v1 submitted 12 October, 2021; originally announced October 2021.

Comments: CoRL 2021. Video: https://dpmd.ai/robotics-stacking-YT . Blog: https://dpmd.ai/robotics-stacking . Code: https://github.com/deepmind/rgb_stacking

iCub

Authors: Lorenzo Natale, Chiara Bartolozzi, Francesco Nori, Giulio Sandini, Giorgio Metta

Abstract: In this chapter we describe the history and evolution of the iCub humanoid platform. We start by describing the first version as it was designed during the RobotCub EU project and illustrate how it evolved to become the platform that is adopted by more than 30 laboratories world wide. We complete the chapter by illustrating some of the research activities that are currently carried out on the iCub… ▽ More In this chapter we describe the history and evolution of the iCub humanoid platform. We start by describing the first version as it was designed during the RobotCub EU project and illustrate how it evolved to become the platform that is adopted by more than 30 laboratories world wide. We complete the chapter by illustrating some of the research activities that are currently carried out on the iCub robot, i.e. visual perception, event driven sensing and dynamic control. We conclude the Chapter with a discussion of the lessons we learned and a preview of the upcoming next release of the robot, iCub 3.0. △ Less

Submitted 7 May, 2021; v1 submitted 5 May, 2021; originally announced May 2021.

Journal ref: Humanoid Robotics: A Reference, Springer, 2017

A Plenum-Based Calibration Device for Tactile Sensor Arrays

Authors: Joan Kangro, Anand Vazhapilli Sureshbabu, Silvio Traversaro, Daniele Pucci, Francesco Nori

Abstract: In modern robotic applications, tactile sensor arrays (i.e., artificial skins) are an emergent solution to determine the locations of contacts between a robot and an external agent. Localizing the point of contact is useful but determining the force applied on the skin provides many additional possibilities. This additional feature usually requires time-consuming calibration procedures to relate t… ▽ More In modern robotic applications, tactile sensor arrays (i.e., artificial skins) are an emergent solution to determine the locations of contacts between a robot and an external agent. Localizing the point of contact is useful but determining the force applied on the skin provides many additional possibilities. This additional feature usually requires time-consuming calibration procedures to relate the sensor readings to the applied forces. This letter presents a novel device that enables the calibration of tactile sensor arrays in a fast and simple way. The key idea is to design a plenum chamber where the skin is inserted, and then the calibration of the tactile sensors is achieved by relating the air pressure and the sensor readings. This general concept is tested experimentally to calibrate the skin of the iCub robot. The validation of the calibration device is achieved by placing the masses of known weight on the artificial skin and comparing the applied force against the one estimated by the sensors. △ Less

Submitted 23 March, 2021; originally announced March 2021.

Comments: 8 pages, 18 figures

Journal ref: IEEE Robotics and Automation Letters ( Volume: 3, Issue: 4, Oct. 2018)

Classification and reconstruction of optical quantum states with deep neural networks

Authors: Shahnawaz Ahmed, Carlos Sánchez Muñoz, Franco Nori, Anton Frisk Kockum

Abstract: We apply deep-neural-network-based techniques to quantum state classification and reconstruction. We demonstrate high classification accuracies and reconstruction fidelities, even in the presence of noise and with little data. Using optical quantum states as examples, we first demonstrate how convolutional neural networks (CNNs) can successfully classify several types of states distorted by, e.g.,… ▽ More We apply deep-neural-network-based techniques to quantum state classification and reconstruction. We demonstrate high classification accuracies and reconstruction fidelities, even in the presence of noise and with little data. Using optical quantum states as examples, we first demonstrate how convolutional neural networks (CNNs) can successfully classify several types of states distorted by, e.g., additive Gaussian noise or photon loss. We further show that a CNN trained on noisy inputs can learn to identify the most important regions in the data, which potentially can reduce the cost of tomography by guiding adaptive data collection. Secondly, we demonstrate reconstruction of quantum-state density matrices using neural networks that incorporate quantum-physics knowledge. The knowledge is implemented as custom neural-network layers that convert outputs from standard feedforward neural networks to valid descriptions of quantum states. Any standard feed-forward neural-network architecture can be adapted for quantum state tomography (QST) with our method. We present further demonstrations of our proposed [arXiv:2008.03240] QST technique with conditional generative adversarial networks (QST-CGAN). We motivate our choice of a learnable loss function within an adversarial framework by demonstrating that the QST-CGAN outperforms, across a range of scenarios, generative networks trained with standard loss functions. For pure states with additive or convolutional Gaussian noise, the QST-CGAN is able to adapt to the noise and reconstruct the underlying state. The QST-CGAN reconstructs states using up to two orders of magnitude fewer iterative steps than a standard iterative maximum likelihood (iMLE) method. Further, the QST-CGAN can reconstruct both pure and mixed states from two orders of magnitude fewer randomly chosen data points than iMLE. △ Less

Submitted 3 December, 2020; originally announced December 2020.

Comments: 40 pages, 20 figures, 5 tables, code will be available at https://github.com/quantshah/qst-nn

Journal ref: Phys. Rev. Research 3, 033278 (2021)

"What, not how": Solving an under-actuated insertion task from scratch

Authors: Giulia Vezzani, Michael Neunert, Markus Wulfmeier, Rae Jeong, Thomas Lampe, Noah Siegel, Roland Hafner, Abbas Abdolmaleki, Martin Riedmiller, Francesco Nori

Abstract: Robot manipulation requires a complex set of skills that need to be carefully combined and coordinated to solve a task. Yet, most ReinforcementLearning (RL) approaches in robotics study tasks which actually consist only of a single manipulation skill, such as grasping an object or inserting a pre-grasped object. As a result the skill ('how' to solve the task) but not the actual goal of a complete… ▽ More Robot manipulation requires a complex set of skills that need to be carefully combined and coordinated to solve a task. Yet, most ReinforcementLearning (RL) approaches in robotics study tasks which actually consist only of a single manipulation skill, such as grasping an object or inserting a pre-grasped object. As a result the skill ('how' to solve the task) but not the actual goal of a complete manipulation ('what' to solve) is specified. In contrast, we study a complex manipulation goal that requires an agent to learn and combine diverse manipulation skills. We propose a challenging, highly under-actuated peg-in-hole task with a free, rotational asymmetrical peg, requiring a broad range of manipulation skills. While correct peg (re-)orientation is a requirement for successful insertion, there is no reward associated with it. Hence an agent needs to understand this pre-condition and learn the skill to fulfil it. The final insertion reward is sparse, allowing freedom in the solution and leading to complex emerging behaviour not envisioned during the task design. We tackle the problem in a multi-task RL framework using Scheduled Auxiliary Control (SAC-X) combined with Regularized Hierarchical Policy Optimization (RHPO) which successfully solves the task in simulation and from scratch on a single robot where data is severely limited. △ Less

Submitted 30 October, 2020; v1 submitted 29 October, 2020; originally announced October 2020.

Learning Dexterous Manipulation from Suboptimal Experts

Authors: Rae Jeong, Jost Tobias Springenberg, Jackie Kay, Daniel Zheng, Yuxiang Zhou, Alexandre Galashov, Nicolas Heess, Francesco Nori

Abstract: Learning dexterous manipulation in high-dimensional state-action spaces is an important open challenge with exploration presenting a major bottleneck. Although in many cases the learning process could be guided by demonstrations or other suboptimal experts, current RL algorithms for continuous action spaces often fail to effectively utilize combinations of highly off-policy expert data and on-poli… ▽ More Learning dexterous manipulation in high-dimensional state-action spaces is an important open challenge with exploration presenting a major bottleneck. Although in many cases the learning process could be guided by demonstrations or other suboptimal experts, current RL algorithms for continuous action spaces often fail to effectively utilize combinations of highly off-policy expert data and on-policy exploration data. As a solution, we introduce Relative Entropy Q-Learning (REQ), a simple policy iteration algorithm that combines ideas from successful offline and conventional RL algorithms. It represents the optimal policy via importance sampling from a learned prior and is well-suited to take advantage of mixed data distributions. We demonstrate experimentally that REQ outperforms several strong baselines on robotic manipulation tasks for which suboptimal experts are available. We show how suboptimal experts can be constructed effectively by composing simple waypoint tracking controllers, and we also show how learned primitives can be combined with waypoint controllers to obtain reference behaviors to bootstrap a complex manipulation task on a simulated bimanual robot with human-like hands. Finally, we show that REQ is also effective for general off-policy RL, offline RL, and RL from demonstrations. Videos and further materials are available at sites.google.com/view/rlfse. △ Less

Submitted 5 January, 2021; v1 submitted 16 October, 2020; originally announced October 2020.

Quantum State Tomography with Conditional Generative Adversarial Networks

Authors: Shahnawaz Ahmed, Carlos Sánchez Muñoz, Franco Nori, Anton Frisk Kockum

Abstract: Quantum state tomography (QST) is a challenging task in intermediate-scale quantum devices. Here, we apply conditional generative adversarial networks (CGANs) to QST. In the CGAN framework, two duelling neural networks, a generator and a discriminator, learn multi-modal models from data. We augment a CGAN with custom neural-network layers that enable conversion of output from any standard neural n… ▽ More Quantum state tomography (QST) is a challenging task in intermediate-scale quantum devices. Here, we apply conditional generative adversarial networks (CGANs) to QST. In the CGAN framework, two duelling neural networks, a generator and a discriminator, learn multi-modal models from data. We augment a CGAN with custom neural-network layers that enable conversion of output from any standard neural network into a physical density matrix. To reconstruct the density matrix, the generator and discriminator networks train each other on data using standard gradient-based methods. We demonstrate that our QST-CGAN reconstructs optical quantum states with high fidelity orders of magnitude faster, and from less data, than a standard maximum-likelihood method. We also show that the QST-CGAN can reconstruct a quantum state in a single evaluation of the generator network if it has been pre-trained on similar quantum states. △ Less

Submitted 4 December, 2020; v1 submitted 7 August, 2020; originally announced August 2020.

Comments: 5 pages, 5 figures, code will be available at https://github.com/quantshah/qst-cgan; v2: minor updates; see also the companion paper arXiv:2012.02185

Journal ref: Phys. Rev. Lett. 127, 140502 (2021)

Modelling Generalized Forces with Reinforcement Learning for Sim-to-Real Transfer

Authors: Rae Jeong, Jackie Kay, Francesco Romano, Thomas Lampe, Tom Rothorl, Abbas Abdolmaleki, Tom Erez, Yuval Tassa, Francesco Nori

Abstract: Learning robotic control policies in the real world gives rise to challenges in data efficiency, safety, and controlling the initial condition of the system. On the other hand, simulations are a useful alternative as they provide an abundant source of data without the restrictions of the real world. Unfortunately, simulations often fail to accurately model complex real-world phenomena. Traditional… ▽ More Learning robotic control policies in the real world gives rise to challenges in data efficiency, safety, and controlling the initial condition of the system. On the other hand, simulations are a useful alternative as they provide an abundant source of data without the restrictions of the real world. Unfortunately, simulations often fail to accurately model complex real-world phenomena. Traditional system identification techniques are limited in expressiveness by the analytical model parameters, and usually are not sufficient to capture such phenomena. In this paper we propose a general framework for improving the analytical model by optimizing state dependent generalized forces. State dependent generalized forces are expressive enough to model constraints in the equations of motion, while maintaining a clear physical meaning and intuition. We use reinforcement learning to efficiently optimize the mapping from states to generalized forces over a discounted infinite horizon. We show that using only minutes of real world data improves the sim-to-real control policy transfer. We demonstrate the feasibility of our approach by validating it on a nonprehensile manipulation task on the Sawyer robot. △ Less

Submitted 21 October, 2019; originally announced October 2019.

Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation

Authors: Rae Jeong, Yusuf Aytar, David Khosid, Yuxiang Zhou, Jackie Kay, Thomas Lampe, Konstantinos Bousmalis, Francesco Nori

Abstract: Collecting and automatically obtaining reward signals from real robotic visual data for the purposes of training reinforcement learning algorithms can be quite challenging and time-consuming. Methods for utilizing unlabeled data can have a huge potential to further accelerate robotic learning. We consider here the problem of performing manipulation tasks from pixels. In such tasks, choosing an app… ▽ More Collecting and automatically obtaining reward signals from real robotic visual data for the purposes of training reinforcement learning algorithms can be quite challenging and time-consuming. Methods for utilizing unlabeled data can have a huge potential to further accelerate robotic learning. We consider here the problem of performing manipulation tasks from pixels. In such tasks, choosing an appropriate state representation is crucial for planning and control. This is even more relevant with real images where noise, occlusions and resolution affect the accuracy and reliability of state estimation. In this work, we learn a latent state representation implicitly with deep reinforcement learning in simulation, and then adapt it to the real domain using unlabeled real robot data. We propose to do so by optimizing sequence-based self supervised objectives. These exploit the temporal nature of robot experience, and can be common in both the simulated and real domains, without assuming any alignment of underlying states in simulated and unlabeled real images. We propose Contrastive Forward Dynamics loss, which combines dynamics model learning with time-contrastive techniques. The learned state representation that results from our methods can be used to robustly solve a manipulation task in simulation and to successfully transfer the learned skill on a real system. We demonstrate the effectiveness of our approaches by training a vision-based reinforcement learning agent for cube stacking. Agents trained with our method, using only 5 hours of unlabeled real robot data for adaptation, shows a clear improvement over domain randomization, and standard visual domain adaptation techniques for sim-to-real transfer. △ Less

Submitted 21 October, 2019; originally announced October 2019.

Simultaneously Learning Vision and Feature-based Control Policies for Real-world Ball-in-a-Cup

Authors: Devin Schwab, Tobias Springenberg, Murilo F. Martins, Thomas Lampe, Michael Neunert, Abbas Abdolmaleki, Tim Hertweck, Roland Hafner, Francesco Nori, Martin Riedmiller

Abstract: We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxiliary task policies to utilize task features that are available only at training-ti… ▽ More We present a method for fast training of vision based control policies on real robots. The key idea behind our method is to perform multi-task Reinforcement Learning with auxiliary tasks that differ not only in the reward to be optimized but also in the state-space in which they operate. In particular, we allow auxiliary task policies to utilize task features that are available only at training-time. This allows for fast learning of auxiliary policies, which subsequently generate good data for training the main, vision-based control policies. This method can be seen as an extension of the Scheduled Auxiliary Control (SAC-X) framework. We demonstrate the efficacy of our method by using both a simulated and real-world Ball-in-a-Cup game controlled by a robot arm. In simulation, our approach leads to significant learning speed-ups when compared to standard SAC-X. On the real robot we show that the task can be learned from-scratch, i.e., with no transfer from simulation and no imitation learning. Videos of our learned policies running on the real robot can be found at https://sites.google.com/view/rss-2019-sawyer-bic/. △ Less

Submitted 18 February, 2019; v1 submitted 12 February, 2019; originally announced February 2019.

Comments: Videos can be found at https://sites.google.com/view/rss-2019-sawyer-bic/

Model Based In Situ Calibration with Temperature compensation of 6 axis Force Torque Sensors

Authors: Francisco Javier Andrade Chavez, Gabriele Nava, Silvio Traversaro, Francesco Nori, Daniele Pucci

Abstract: It is well known that sensors using strain gauges have a potential dependency on temperature. This creates temperature drift in the measurements of six axis force torque sensors (F/T). The temperature drift can be considerable if an experiment is long or the environmental conditions are different from when the calibration of the sensor was performed. Other \textit{in situ} methods disregard the ef… ▽ More It is well known that sensors using strain gauges have a potential dependency on temperature. This creates temperature drift in the measurements of six axis force torque sensors (F/T). The temperature drift can be considerable if an experiment is long or the environmental conditions are different from when the calibration of the sensor was performed. Other \textit{in situ} methods disregard the effect of temperature on the sensor measurements. Experiments performed using the humanoid robot platform iCub show that the effect of temperature is relevant. The model based \textit{in situ} calibration of six axis force torque sensors method is extended to perform temperature compensation. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Comments: Submitted to ICRA 2019

Towards Partner-Aware Humanoid Robot Control Under Physical Interactions

Authors: Yeshasvi Tirupachuri, Gabriele Nava, Claudia Latella, Diego Ferigo, Lorenzo Rapetti, Luca Tagliapietra, Francesco Nori, Daniele Pucci

Abstract: The topic of physical human-robot interaction received a lot of attention from the robotics community because of many promising application domains. However, studying physical interaction between a robot and an external agent, like a human or another robot, without considering the dynamics of both the systems may lead to many short-comings in fully exploiting the interaction. In this paper, we pre… ▽ More The topic of physical human-robot interaction received a lot of attention from the robotics community because of many promising application domains. However, studying physical interaction between a robot and an external agent, like a human or another robot, without considering the dynamics of both the systems may lead to many short-comings in fully exploiting the interaction. In this paper, we present a coupled-dynamics formalism followed by a sound approach in exploiting helpful interaction with a humanoid robot. In particular, we propose the first attempt to define and exploit the human help for the robot to accomplish a specific task. As a result, we present a task-based partner-aware robot control techniques. The theoretical results are validated by conducting experiments with two iCub humanoid robots involved in physical interaction. △ Less

Submitted 20 March, 2019; v1 submitted 17 September, 2018; originally announced September 2018.

Comments: Accepted to Intelligent Systems 2019 Conference

Momentum-Based Topology Estimation of Articulated Objects

Authors: Yeshasvi Tirupachuri, Silvio Traversaro, Francesco Nori, Daniele Pucci

Abstract: Articulated objects like doors, drawers, valves, and tools are pervasive in our everyday unstructured dynamic environments. Articulation models describe the joint nature between the different parts of an articulated object. As most of these objects are passive, a robot has to interact with them to infer all the articulation models to understand the object topology. We present a general algorithm t… ▽ More Articulated objects like doors, drawers, valves, and tools are pervasive in our everyday unstructured dynamic environments. Articulation models describe the joint nature between the different parts of an articulated object. As most of these objects are passive, a robot has to interact with them to infer all the articulation models to understand the object topology. We present a general algorithm to estimate the inherent articulation models by exploiting the momentum of the articulated system along with the interaction wrench while manipulating the object. We validate our approach with experiments in a simulation environment. △ Less

Submitted 20 March, 2019; v1 submitted 5 August, 2018; originally announced August 2018.

Comments: Accepted to Intelligent Systems 2019 Conference

A Control Architecture with Online Predictive Planning for Position and Torque Controlled Walking of Humanoid Robots

Authors: Stefano Dafarra, Gabriele Nava, Marie Charbonneau, Nuno Guedelha, Francisco Andrade, Silvio Traversaro, Luca Fiorio, Francesco Romano, Francesco Nori, Giorgio Metta, Daniele Pucci

Abstract: A common approach to the generation of walking patterns for humanoid robots consists in adopting a layered control architecture. This paper proposes an architecture composed of three nested control loops. The outer loop exploits a robot kinematic model to plan the footstep positions. In the mid layer, a predictive controller generates a Center of Mass trajectory according to the well-known table-c… ▽ More A common approach to the generation of walking patterns for humanoid robots consists in adopting a layered control architecture. This paper proposes an architecture composed of three nested control loops. The outer loop exploits a robot kinematic model to plan the footstep positions. In the mid layer, a predictive controller generates a Center of Mass trajectory according to the well-known table-cart model. Through a whole-body inverse kinematics algorithm, we can define joint references for position controlled walking. The outcomes of these two loops are then interpreted as inputs of a stack-of-task QP-based torque controller, which represents the inner loop of the presented control architecture. This resulting architecture allows the robot to walk also in torque control, guaranteeing higher level of compliance. Real world experiments have been carried on the humanoid robot iCub. △ Less

Submitted 14 July, 2018; originally announced July 2018.

Comments: IROS 2018

Contact Force and Joint Torque Estimation Using Skin

Authors: Francisco Javier Andrade Chavez, Joan Kangro, Silvio Traversaro, Francesco Nori, Daniele Pucci

Abstract: In this paper, we present algorithms to estimate external contact forces and joint torques using only skin, i.e. distributed tactile sensors. To deal with gaps between the tactile sensors (taxels), we use interpolation techniques. The application of these interpolation techniques allows us to estimate contact forces and joint torques without the need for expensive force-torque sensors. Validation… ▽ More In this paper, we present algorithms to estimate external contact forces and joint torques using only skin, i.e. distributed tactile sensors. To deal with gaps between the tactile sensors (taxels), we use interpolation techniques. The application of these interpolation techniques allows us to estimate contact forces and joint torques without the need for expensive force-torque sensors. Validation was performed using the iCub humanoid robot. △ Less

Submitted 20 September, 2017; originally announced September 2017.

Comments: Submitted to ICRA2018

MSC Class: 93C85; 68T40

Modeling and Control of Humanoid Robots in Dynamic Environments: iCub Balancing on a Seesaw

Authors: Gabriele Nava, Daniele Pucci, Nuno Guedelha, Silvio Traversaro, Francesco Romano, Stefano Dafarra, Francesco Nori

Abstract: Forthcoming applications concerning humanoid robots may involve physical interaction between the robot and a dynamic environment. In such scenario, classical balancing and walking controllers that neglect the environment dynamics may not be sufficient for achieving a stable robot behavior. This paper presents a modeling and control framework for balancing humanoid robots in contact with a dynamic… ▽ More Forthcoming applications concerning humanoid robots may involve physical interaction between the robot and a dynamic environment. In such scenario, classical balancing and walking controllers that neglect the environment dynamics may not be sufficient for achieving a stable robot behavior. This paper presents a modeling and control framework for balancing humanoid robots in contact with a dynamic environment. We first model the robot and environment dynamics, together with the contact constraints. Then, a control strategy for stabilizing the full system is proposed. Theoretical results are verified in simulation with robot iCub balancing on a seesaw. △ Less

Submitted 9 March, 2018; v1 submitted 28 July, 2017; originally announced July 2017.

An Optimization Based Control Framework for Balancing and Walking: Implementation on the iCub Robot

Authors: Marie Charbonneau, Gabriele Nava, Francesco Nori, Daniele Pucci

Abstract: A whole-body torque control framework adapted for balancing and walking tasks is presented in this paper. In the proposed approach, centroidal momentum terms are excluded in favor of a hierarchy of high-priority position and orientation tasks and a low-priority postural task. More specifically, the controller stabilizes the position of the center of mass, the orientation of the pelvis frame, as we… ▽ More A whole-body torque control framework adapted for balancing and walking tasks is presented in this paper. In the proposed approach, centroidal momentum terms are excluded in favor of a hierarchy of high-priority position and orientation tasks and a low-priority postural task. More specifically, the controller stabilizes the position of the center of mass, the orientation of the pelvis frame, as well as the position and orientation of the feet frames. The low-priority postural task provides reference positions for each joint of the robot. Joint torques and contact forces to stabilize tasks are obtained through quadratic programming optimization. Besides the exclusion of centroidal momentum terms, part of the novelty of the approach lies in the definition of control laws in SE(3) which do not require the use of Euler parameterization. Validation of the framework was achieved in a scenario where the robot kept balance while walking in place. Experiments have been conducted with the iCub robot, in simulation and in real-world experiments. △ Less

Submitted 26 July, 2017; originally announced July 2017.

Comments: manuscript submitted to Humanoids 2017

A Receding Horizon Push Recovery Strategy for Balancing the iCub Humanoid Robot

Authors: Stefano Dafarra, Francesco Romano, Francesco Nori

Abstract: Balancing and reacting to strong and unexpected pushes is a critical requirement for humanoid robots. We recently designed a capture point based approach which interfaces with a momentum-based torque controller and we implemented and validated it on the iCub humanoid robot. In this work we implement a Receding Horizon control, also known as Model Predictive Control, to add the possibility to predi… ▽ More Balancing and reacting to strong and unexpected pushes is a critical requirement for humanoid robots. We recently designed a capture point based approach which interfaces with a momentum-based torque controller and we implemented and validated it on the iCub humanoid robot. In this work we implement a Receding Horizon control, also known as Model Predictive Control, to add the possibility to predict the future evolution of the robot, especially the constraints switching given by the hybrid nature of the system. We prove that the proposed MPC extension makes the step-recovery controller more robust and reliable when executing the recovery strategy. Experiments in simulation show the results of the proposed approach. △ Less

Submitted 30 May, 2017; originally announced May 2017.

A Predictive Momentum-Based Whole-Body Torque Controller: Theory and Simulations for the iCub Stepping

Authors: Stefano Dafarra, Francesco Romano, Gabriele Nava, Francesco Nori

Abstract: When balancing, a humanoid robot can be easily subjected to unexpected disturbances like external pushes. In these circumstances, reactive movements as steps become a necessary requirement in order to avoid potentially harmful falling states. In this paper we conceive a Model Predictive Controller which determines a desired set of contact wrenches by predicting the future evolution of the robot, w… ▽ More When balancing, a humanoid robot can be easily subjected to unexpected disturbances like external pushes. In these circumstances, reactive movements as steps become a necessary requirement in order to avoid potentially harmful falling states. In this paper we conceive a Model Predictive Controller which determines a desired set of contact wrenches by predicting the future evolution of the robot, while taking into account constraints switching in case of steps. The control inputs computed by this strategy, namely the desired contact wrenches, are directly obtained on the robot through a modification of the momentum-based whole-body torque controller currently implemented on iCub. The proposed approach is validated through simulations in a stepping scenario, revealing high robustness and reliability when executing a recovery strategy. △ Less

Submitted 28 July, 2017; v1 submitted 30 May, 2017; originally announced May 2017.

Torque-Controlled Stepping-Strategy Push Recovery: Design and Implementation on the iCub Humanoid Robot

Authors: Stefano Dafarra, Francesco Romano, Francesco Nori

Abstract: One of the challenges for the robotics community is to deploy robots which can reliably operate in real world scenarios together with humans. A crucial requirement for legged robots is the capability to properly balance on their feet, rejecting external disturbances. iCub is a state-of-the-art humanoid robot which has only recently started to balance on its feet. While the current balancing contro… ▽ More One of the challenges for the robotics community is to deploy robots which can reliably operate in real world scenarios together with humans. A crucial requirement for legged robots is the capability to properly balance on their feet, rejecting external disturbances. iCub is a state-of-the-art humanoid robot which has only recently started to balance on its feet. While the current balancing controller has proved successful in various scenarios, it still misses the capability to properly react to strong pushes by taking steps. This paper goes in this direction. It proposes and implements a control strategy based on the Capture Point concept [1]. Instead of relying on position control, like most of Capture Point related approaches, the proposed strategy generates references for the momentum-based torque controller already implemented on the iCub, thus extending its capabilities to react to external disturbances, while retaining the advantages of torque control when interacting with the environment. Experiments in the Gazebo simulator and on the iCub humanoid robot validate the proposed strategy. △ Less

Submitted 30 May, 2017; originally announced May 2017.

Inverse, forward and other dynamic computations computationally optimized with sparse matrix factorizations

Authors: Francesco Nori

Abstract: We propose an algorithm to compute the dynamics of articulated rigid-bodies with different sensor distributions. Prior to the on-line computations, the proposed algorithm performs an off-line optimisation step to simplify the computational complexity of the underlying solution. This optimisation step consists in formulating the dynamic computations as a system of linear equations. The computationa… ▽ More We propose an algorithm to compute the dynamics of articulated rigid-bodies with different sensor distributions. Prior to the on-line computations, the proposed algorithm performs an off-line optimisation step to simplify the computational complexity of the underlying solution. This optimisation step consists in formulating the dynamic computations as a system of linear equations. The computational complexity of computing the associated solution is reduced by performing a permuted LU-factorisation with off-line optimised permutations. We apply our algorithm to solve classical dynamic problems: inverse and forward dynamics. The computational complexity of the proposed solution is compared to `gold standard' algorithms: recursive Newton-Euler and articulated body algorithm. It is shown that our algorithm reduces the number of floating point operations with respect to previous approaches. We also evaluate the numerical complexity of our algorithm by performing tests on dynamic computations for which no gold standard is available. △ Less

Submitted 12 May, 2017; originally announced May 2017.

Comments: 8 pages, 2 figure, conference RCAR 2017

Momentum Control of Humanoid Robots with Series Elastic Actuators

Authors: Gabriele Nava, Daniele Pucci, Francesco Nori

Abstract: Humanoid robots may require a degree of compliance at the joint level for improving efficiency, shock tolerance, and safe interaction with humans. The presence of joint elasticity, however, complexifies the design of balancing and walking controllers. This paper proposes a control framework for extending momentum based controllers developed for stiff actuators to the case of series elastic actuato… ▽ More Humanoid robots may require a degree of compliance at the joint level for improving efficiency, shock tolerance, and safe interaction with humans. The presence of joint elasticity, however, complexifies the design of balancing and walking controllers. This paper proposes a control framework for extending momentum based controllers developed for stiff actuators to the case of series elastic actuators. The key point is to consider the motor velocities as an intermediate control input, and then apply high-gain control to stabilise the desired motor velocities achieving momentum control. Simulations carried out on a model of the robot iCub verify the soundness of the proposed approach. △ Less

Submitted 6 March, 2017; originally announced March 2017.

Momentum Control of an Underactuated Flying Humanoid Robot

Authors: Daniele Pucci, Silvio Traversaro, Francesco Nori

Abstract: The paper takes the first step towards the de- velopment of a control framework for underactuated flying humanoid robots. These robots may thus have the capacities of flight, contact locomotion, and manipulation, and benefit from technologies and methods developed for Whole-Body Control and Aerial Manipulation. As in the case of quadrotors, we as- sume that the humanoid robot is powered by four th… ▽ More The paper takes the first step towards the de- velopment of a control framework for underactuated flying humanoid robots. These robots may thus have the capacities of flight, contact locomotion, and manipulation, and benefit from technologies and methods developed for Whole-Body Control and Aerial Manipulation. As in the case of quadrotors, we as- sume that the humanoid robot is powered by four thrust forces. For convenience, these forces are placed at the robot hands and feet. The control objective is defined as the asymptotic stabilization of the robot centroidal momentum. This objective allows us to track a desired trajectory for the robot center of mass and keep small errors between a reference orientation and the robot base frame. Stability and convergence of the robot momentum are shown to be in the sense of Lyapunov. Simulations carried out on a model of the humanoid robot iCub verify the soundness of the proposed approach. △ Less

Submitted 3 June, 2017; v1 submitted 16 February, 2017; originally announced February 2017.

On Centroidal Dynamics and Integrability of Average Angular Velocity

Authors: Alessandro Saccon, Silvio Traversaro, Francesco Nori, Henk Nijmeijer

Abstract: In the literature on robotics and multibody dynamics, the concept of average angular velocity has received considerable attention in recent years. We address the question of whether the average angular velocity defines an orientation framethat depends only on the current robot configuration and provide a simple algebraic condition to check whether this holds. In the language of geometric mechanics… ▽ More In the literature on robotics and multibody dynamics, the concept of average angular velocity has received considerable attention in recent years. We address the question of whether the average angular velocity defines an orientation framethat depends only on the current robot configuration and provide a simple algebraic condition to check whether this holds. In the language of geometric mechanics, this condition corresponds to requiring the flatness of the mechanical connection associated to the robotic system. Here, however, we provide both a reinterpretation and a proof of this result accessible to readers with a background in rigid body kinematics and multibody dynamics but not necessarily acquainted with differential geometry, still providing precise links to the geometric mechanics literature. This should help spreading the algebraic condition beyond the scope of geometric mechanics,contributing to a proper utilization and understanding of the concept of average angular velocity. △ Less

Submitted 10 January, 2017; originally announced January 2017.

Comments: 8 pages, accepted for IEEE Robotics and Automation Letters (RA-L)

A Whole-Body Software Abstraction layer for Control Design of free-floating Mechanical Systems

Authors: Francesco Romano, Silvio Traversaro, Daniele Pucci, Jorhabib Eljaik, Andrea Del Prete, Francesco Nori

Abstract: In this paper, we propose a software abstraction layer to simplify the design and synthesis of whole-body controllers without making any preliminary assumptions on the control law to be implemented. The main advantage of the proposed library is the decoupling of the control software from implementation details, which are related to the robotic platform. Furthermore, the resulting code is more clea… ▽ More In this paper, we propose a software abstraction layer to simplify the design and synthesis of whole-body controllers without making any preliminary assumptions on the control law to be implemented. The main advantage of the proposed library is the decoupling of the control software from implementation details, which are related to the robotic platform. Furthermore, the resulting code is more clean and concise than ad-hoc code, as it focuses only on the implementation of the control law. In addition, we present a reference implementation of the abstraction layer together with a Simulink interface to provide support to Model-Driven based development. We also show the implementation of a simple proportional-derivative plus gravity compensation control together with a more complex momentum-based bipedal balance controller. △ Less

Submitted 4 January, 2017; originally announced January 2017.

Identification of Fully Physical Consistent Inertial Parameters using Optimization on Manifolds

Authors: Silvio Traversaro, Stanislas Brossette, Adrien Escande, Francesco Nori

Abstract: This paper presents a new condition, the fully physical consistency for a set of inertial parameters to determine if they can be generated by a physical rigid body. The proposed condition ensure both the positive definiteness and the triangular inequality of 3D inertia matrices as opposed to existing techniques in which the triangular inequality constraint is ignored. This paper presents also a ne… ▽ More This paper presents a new condition, the fully physical consistency for a set of inertial parameters to determine if they can be generated by a physical rigid body. The proposed condition ensure both the positive definiteness and the triangular inequality of 3D inertia matrices as opposed to existing techniques in which the triangular inequality constraint is ignored. This paper presents also a new parametrization that naturally ensures that the inertial parameters are fully physical consistency. The proposed parametrization is exploited to reformulate the inertial identification problem as a manifold optimization problem, that ensures that the identified parameters can always be generated by a physical body. The proposed optimization problem has been validated with a set of experiments on the iCub humanoid robot. △ Less

Submitted 27 October, 2016; originally announced October 2016.

Comments: 6 pages, published in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on

Model Based In Situ Calibration of Six Axis Force Torque Sensors

Authors: Francisco Javier Andrade Chavez, Silvio Traversaro, Daniele Pucci, Francesco Nori

Abstract: This paper proposes and validates an in situ calibration method to calibrate six axis force torque (F/T) sensors once they are mounted on the system. This procedure takes advantage of the knowledge of the model of the robot to generate the expected wrenches of the sensors during some arbitrary motions. It then uses this information to train and validate new calibration matrices, taking into accoun… ▽ More This paper proposes and validates an in situ calibration method to calibrate six axis force torque (F/T) sensors once they are mounted on the system. This procedure takes advantage of the knowledge of the model of the robot to generate the expected wrenches of the sensors during some arbitrary motions. It then uses this information to train and validate new calibration matrices, taking into account the calibration matrix obtained with a classical Workbench calibration. The proposed calibration algorithm is validated on the F/T sensors mounted on the iCub humanoid robot legs. △ Less

Submitted 27 October, 2016; v1 submitted 11 October, 2016; originally announced October 2016.

Comments: 6 pages, 10 figures, 3 tables

Automatic Gain Tuning of a Momentum Based Balancing Controller for Humanoid Robots

Authors: Daniele Pucci, Gabriele Nava, Francesco Nori

Abstract: This paper proposes a technique for automatic gain tuning of a momentum based balancing controller for humanoid robots. The controller ensures the stabilization of the centroidal dynamics and the associated zero dynamics. Then, the closed-loop, constrained joint space dynamics is linearized and the controller's gains are chosen so as to obtain desired properties of the linearized system. Symmetry… ▽ More This paper proposes a technique for automatic gain tuning of a momentum based balancing controller for humanoid robots. The controller ensures the stabilization of the centroidal dynamics and the associated zero dynamics. Then, the closed-loop, constrained joint space dynamics is linearized and the controller's gains are chosen so as to obtain desired properties of the linearized system. Symmetry and positive definiteness constraints of gain matrices are enforced by proposing a tracker for symmetric positive definite matrices. Simulation results are carried out on the humanoid robot iCub. △ Less

Submitted 21 October, 2016; v1 submitted 10 October, 2016; originally announced October 2016.

Comments: Accepted at IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS). 2016

The Static Center of Pressure Sensitivity: a further Criterion to assess Contact Stability and Balancing Controllers

Authors: Francesco Romano, Daniele Pucci, Silvio Traversaro, Francesco Nori

Abstract: Legged locomotion has received increasing attention from the robotics community. In this respect, contact stability plays a critical role in ensuring that robots maintain balance, and it is a key element for balancing and walking controllers. The Center of Pressure is a contact stability criterion that defines a point that must be kept strictly inside the support polygon in order to ensure postura… ▽ More Legged locomotion has received increasing attention from the robotics community. In this respect, contact stability plays a critical role in ensuring that robots maintain balance, and it is a key element for balancing and walking controllers. The Center of Pressure is a contact stability criterion that defines a point that must be kept strictly inside the support polygon in order to ensure postural stability. In this paper, we introduce the concept of the sensitivity of the static center of pressure: roughly speaking, the rate of change of the center of pressure with respect to the system equilibrium configurations. This new concept can be used as an additional criterion to assess the robustness of the contact stability. We show how the sensitivity of the center of pressure can also be used as a metric to assess balancing controllers by considering two state-of-the-art control strategies. The analytical analysis is performed on a simplified model, and validated during balancing tasks on the iCub humanoid robot. △ Less

Submitted 29 May, 2017; v1 submitted 5 October, 2016; originally announced October 2016.

Skin Normal Force Calibration Using Vacuum Bags

Authors: Joan Kangro, Silvio Traversaro, Daniele Pucci, Francesco Nori

Abstract: The paper presents a proof of concept to calibrate iCub's skin using vacuum bags. The method's main idea consists in inserting the skin in a vacuum bag, and then decreasing the pressure in the bag to create a uniform pressure distribution on the skin surface. Acquisition and data processing of the bag pressure and sensors' measured capacitance allow us to characterize the relationship between the… ▽ More The paper presents a proof of concept to calibrate iCub's skin using vacuum bags. The method's main idea consists in inserting the skin in a vacuum bag, and then decreasing the pressure in the bag to create a uniform pressure distribution on the skin surface. Acquisition and data processing of the bag pressure and sensors' measured capacitance allow us to characterize the relationship between the pressure and the measured capacitance of each sensor. After calibration, integration of the pressure distribution over the skin geometry provides us with the net normal force applied to the skin. Experiments are conducted using the forearm skin of the iCub humanoid robot, and validation results indicate acceptable average errors in force prediction. △ Less

Submitted 30 September, 2016; originally announced September 2016.

Comments: 7 pages

On-line Joint Limit Avoidance for Torque Controlled Robots by Joint Space Parametrization

Authors: Marie Charbonneau, Francesco Nori, Daniele Pucci

Abstract: This paper proposes control laws ensuring the stabilization of a time-varying desired joint trajectory, as well as joint limit avoidance, in the case of fully-actuated manipulators. The key idea is to perform a parametrization of the feasible joint space in terms of exogenous states. It follows that the control of these states allows for joint limit avoidance. One of the main outcomes of this pape… ▽ More This paper proposes control laws ensuring the stabilization of a time-varying desired joint trajectory, as well as joint limit avoidance, in the case of fully-actuated manipulators. The key idea is to perform a parametrization of the feasible joint space in terms of exogenous states. It follows that the control of these states allows for joint limit avoidance. One of the main outcomes of this paper is that position terms in control laws are replaced by parametrized terms, where joint limits must be avoided. Stability and convergence of time-varying reference trajectories obtained with the proposed method are demonstrated to be in the sense of Lyapunov. The introduced control laws are verified by carrying out experiments on two degrees-of-freedom of the humanoid robot iCub. △ Less

Submitted 26 July, 2017; v1 submitted 24 August, 2016; originally announced August 2016.

Comments: 8 pages, 4 figures. Submitted to the 2016 IEEE-RAS International Conference on Humanoid Robots

MSC Class: 93C85; 70E60; 70B15; 68T40

Journal ref: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids)

Walking of the iCub humanoid robot in different scenarios: implementation and performance analysis

Authors: Yue Hu, Jorhabib Eljaik, Kevin Stein, Francesco Nori, Katja Mombaur

Abstract: The humanoid robot iCub is a research platform of the Fondazione Istituto Italiano di Tecnologia (IIT), spread among different institutes around the world. In the most recent version of iCub, the robot is equipped with stronger legs and bigger feet, allowing it to perform balancing and walking motions that were not possible with the first generations. Despite the new legs hardware, walking has bee… ▽ More The humanoid robot iCub is a research platform of the Fondazione Istituto Italiano di Tecnologia (IIT), spread among different institutes around the world. In the most recent version of iCub, the robot is equipped with stronger legs and bigger feet, allowing it to perform balancing and walking motions that were not possible with the first generations. Despite the new legs hardware, walking has been rarely performed on the iCub robot. In this work the objective is to implement walking motions on the robot, from which we want to analyze its walking capabilities. We developed software modules based on extensions of classic techniques such as the ZMP based pattern generator and position control to identify which are the characteristics as well as limitations of the robot against different walking tasks in order to give the users a reference of the performance of the robot. Most of the experiments have been performed with HeiCub, a reduced version of iCub without arms and head. △ Less

Submitted 17 July, 2017; v1 submitted 28 July, 2016; originally announced July 2016.

Comments: 8 pages, presented at IEEE International Conference on Humanoid Robots (Humanoids 2016)

Walking on Partial Footholds Including Line Contacts with the Humanoid Robot Atlas

Authors: Georg Wiedebach, Sylvain Bertrand, Tingfan Wu, Luca Fiorio, Stephen McCrory, Robert Griffin, Francesco Nori, Jerry Pratt

Abstract: We present a method for humanoid robot walking on partial footholds such as small stepping stones and rocks with sharp surfaces. Our algorithm does not rely on prior knowledge of the foothold, but information about an expected foothold can be used to improve the stepping performance. After a step is taken, the robot explores the new contact surface by attempting to shift the center of pressure aro… ▽ More We present a method for humanoid robot walking on partial footholds such as small stepping stones and rocks with sharp surfaces. Our algorithm does not rely on prior knowledge of the foothold, but information about an expected foothold can be used to improve the stepping performance. After a step is taken, the robot explores the new contact surface by attempting to shift the center of pressure around the foot. The available foothold is inferred by the way in which the foot rotates about contact edges and/or by the achieved center of pressure locations on the foot during exploration. This estimated contact area is then used by a whole body momentum-based control algorithm. To walk and balance on partial footholds, we combine fast, dynamic stepping with the use of upper body angular momentum to regain balance. We applied this method to the Atlas humanoid designed by Boston Dynamics to walk over small contact surfaces, such as line and point contacts. We present experimental results and discuss performance limitations. △ Less

Submitted 12 January, 2017; v1 submitted 27 July, 2016; originally announced July 2016.

Comments: in 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids)

Stability Analysis and Design of Momentum-based Controllers for Humanoid Robots

Authors: Gabriele Nava, Francesco Romano, Francesco Nori, Daniele Pucci

Abstract: Envisioned applications for humanoid robots call for the design of balancing and walking controllers. While promising results have been recently achieved, robust and reliable controllers are still a challenge for the control community dealing with humanoid robotics. Momentum-based strategies have proven their effectiveness for controlling humanoids balancing, but the stability analysis of these co… ▽ More Envisioned applications for humanoid robots call for the design of balancing and walking controllers. While promising results have been recently achieved, robust and reliable controllers are still a challenge for the control community dealing with humanoid robotics. Momentum-based strategies have proven their effectiveness for controlling humanoids balancing, but the stability analysis of these controllers is still missing. The contribution of this paper is twofold. First, we numerically show that the application of state-of-the-art momentum-based control strategies may lead to unstable zero dynamics. Secondly, we propose simple modifications to the control architecture that avoid instabilities at the zero-dynamics level. Asymptotic stability of the closed loop system is shown by means of a Lyapunov analysis on the linearized system's joint space. The theoretical results are validated with both simulations and experiments on the iCub humanoid robot. △ Less

Submitted 16 July, 2017; v1 submitted 14 March, 2016; originally announced March 2016.

Comments: The notation used in the paper can be found at: http://wiki.icub.org/codyco/dox/html/dynamics_notation.html

Incremental Semiparametric Inverse Dynamics Learning

Authors: Raffaello Camoriano, Silvio Traversaro, Lorenzo Rosasco, Giorgio Metta, Francesco Nori

Abstract: This paper presents a novel approach for incremental semiparametric inverse dynamics learning. In particular, we consider the mixture of two approaches: Parametric modeling based on rigid body dynamics equations and nonparametric modeling based on incremental kernel methods, with no prior information on the mechanical properties of the system. This yields to an incremental semiparametric approach,… ▽ More This paper presents a novel approach for incremental semiparametric inverse dynamics learning. In particular, we consider the mixture of two approaches: Parametric modeling based on rigid body dynamics equations and nonparametric modeling based on incremental kernel methods, with no prior information on the mechanical properties of the system. This yields to an incremental semiparametric approach, leveraging the advantages of both the parametric and nonparametric models. We validate the proposed technique learning the dynamics of one arm of the iCub humanoid robot. △ Less

Submitted 18 January, 2016; originally announced January 2016.