-
Remote telepresence over large distances via robot avatars: case studies
Authors:
Mohamed Elobaid,
Stefano Dafarra,
Ehsan Ranjbari,
Giulio Romualdi,
Tomohiro Chaki,
Tomohiro Kawakami,
Takahide Yoshiike,
Daniele Pucci
Abstract:
This paper discusses the necessary considerations and adjustments that allow a recently proposed avatar system architecture to be used with different robotic avatar morphologies (both wheeled and legged robots with various types of hands and kinematic structures) for the purpose of enabling remote (intercontinental) telepresence under communication bandwidth restrictions. The case studies reported…
▽ More
This paper discusses the necessary considerations and adjustments that allow a recently proposed avatar system architecture to be used with different robotic avatar morphologies (both wheeled and legged robots with various types of hands and kinematic structures) for the purpose of enabling remote (intercontinental) telepresence under communication bandwidth restrictions. The case studies reported involve robots using both position and torque control modes, independently of their software middleware.
△ Less
Submitted 2 September, 2024;
originally announced September 2024.
-
Adaptive Non-linear Centroidal MPC with Stability Guarantees for Robust Locomotion of Legged Robots
Authors:
Mohamed Elobaid,
Giulio Turrisi,
Lorenzo Rapetti,
Giulio Romualdi,
Stefano Dafarra,
Tomohiro Kawakami,
Tomohiro Chaki,
Takahide Yoshiike,
Claudio Semini,
Daniele Pucci
Abstract:
Nonlinear model predictive locomotion controllers based on the reduced centroidal dynamics are nowadays ubiquitous in legged robots. These schemes, even if they assume an inherent simplification of the robot's dynamics, were shown to endow robots with a step-adjustment capability in reaction to small pushes, and, moreover, in the case of uncertain parameters - as unknown payloads - they were shown…
▽ More
Nonlinear model predictive locomotion controllers based on the reduced centroidal dynamics are nowadays ubiquitous in legged robots. These schemes, even if they assume an inherent simplification of the robot's dynamics, were shown to endow robots with a step-adjustment capability in reaction to small pushes, and, moreover, in the case of uncertain parameters - as unknown payloads - they were shown to be able to provide some practical, albeit limited, robustness. In this work, we provide rigorous certificates of their closed loop stability via a reformulation of the centroidal MPC controller. This is achieved thanks to a systematic procedure inspired by the machinery of adaptive control, together with ideas coming from Control Lyapunov functions. Our reformulation, in addition, provides robustness for a class of unmeasured constant disturbances. To demonstrate the generality of our approach, we validated our formulation on a new generation of humanoid robots - the 56.7 kg ergoCub, as well as on a commercially available 21 kg quadruped robot, Aliengo.
△ Less
Submitted 13 January, 2025; v1 submitted 2 September, 2024;
originally announced September 2024.
-
XBG: End-to-end Imitation Learning for Autonomous Behaviour in Human-Robot Interaction and Collaboration
Authors:
Carlos Cardenas-Perez,
Giulio Romualdi,
Mohamed Elobaid,
Stefano Dafarra,
Giuseppe L'Erario,
Silvio Traversaro,
Pietro Morerio,
Alessio Del Bue,
Daniele Pucci
Abstract:
This paper presents XBG (eXteroceptive Behaviour Generation), a multimodal end-to-end Imitation Learning (IL) system for a whole-body autonomous humanoid robot used in real-world Human-Robot Interaction (HRI) scenarios. The main contribution of this paper is an architecture for learning HRI behaviours using a data-driven approach. Through teleoperation, a diverse dataset is collected, comprising d…
▽ More
This paper presents XBG (eXteroceptive Behaviour Generation), a multimodal end-to-end Imitation Learning (IL) system for a whole-body autonomous humanoid robot used in real-world Human-Robot Interaction (HRI) scenarios. The main contribution of this paper is an architecture for learning HRI behaviours using a data-driven approach. Through teleoperation, a diverse dataset is collected, comprising demonstrations across multiple HRI scenarios, including handshaking, handwaving, payload reception, walking, and walking with a payload. After synchronizing, filtering, and transforming the data, different Deep Neural Networks (DNN) models are trained. The final system integrates different modalities comprising exteroceptive and proprioceptive sources of information to provide the robot with an understanding of its environment and its own actions. The robot takes sequence of images (RGB and depth) and joints state information during the interactions and then reacts accordingly, demonstrating learned behaviours. By fusing multimodal signals in time, we encode new autonomous capabilities into the robotic platform, allowing the understanding of context changes over time. The models are deployed on ergoCub, a real-world humanoid robot, and their performance is measured by calculating the success rate of the robot's behaviour under the mentioned scenarios.
△ Less
Submitted 22 June, 2024;
originally announced June 2024.
-
Online Non-linear Centroidal MPC for Humanoid Robots Payload Carrying with Contact-Stable Force Parametrization
Authors:
Mohamed Elobaid,
Giulio Romualdi,
Gabriele Nava,
Lorenzo Rapetti,
Hosameldin Awadalla Omer Mohamed,
Daniele Pucci
Abstract:
In this paper we consider the problem of allowing a humanoid robot that is subject to a persistent disturbance, in the form of a payload-carrying task, to follow given planned footsteps. To solve this problem, we combine an online nonlinear centroidal Model Predictive Controller - MPC with a contact stable force parametrization. The cost function of the MPC is augmented with terms handling the dis…
▽ More
In this paper we consider the problem of allowing a humanoid robot that is subject to a persistent disturbance, in the form of a payload-carrying task, to follow given planned footsteps. To solve this problem, we combine an online nonlinear centroidal Model Predictive Controller - MPC with a contact stable force parametrization. The cost function of the MPC is augmented with terms handling the disturbance and regularizing the parameter. The performance of the resulting controller is validated both in simulations and on the humanoid robot iCub. Finally, the effect of using the parametrization on the computational time of the controller is briefly studied.
△ Less
Submitted 18 May, 2023;
originally announced May 2023.
-
A Control Approach for Human-Robot Ergonomic Payload Lifting
Authors:
Lorenzo Rapetti,
Carlotta Sartore,
Mohamed Elobaid,
Yeshasvi Tirupachuri,
Francesco Draicchio,
Tomohiro Kawakami,
Takahide Yoshiike,
Daniele Pucci
Abstract:
Collaborative robots can relief human operators from excessive efforts during payload lifting activities. Modelling the human partner allows the design of safe and efficient collaborative strategies. In this paper, we present a control approach for human-robot collaboration based on human monitoring through whole-body wearable sensors, and interaction modelling through coupled rigid-body dynamics.…
▽ More
Collaborative robots can relief human operators from excessive efforts during payload lifting activities. Modelling the human partner allows the design of safe and efficient collaborative strategies. In this paper, we present a control approach for human-robot collaboration based on human monitoring through whole-body wearable sensors, and interaction modelling through coupled rigid-body dynamics. Moreover, a trajectory advancement strategy is proposed, allowing for online adaptation of the robot trajectory depending on the human motion. The resulting framework allows us to perform payload lifting tasks, taking into account the ergonomic requirements of the agents. Validation has been performed in an experimental scenario using the iCub3 humanoid robot and a human subject sensorized with the iFeel wearable system.
△ Less
Submitted 15 May, 2023;
originally announced May 2023.
-
iCub3 Avatar System: Enabling Remote Fully-Immersive Embodiment of Humanoid Robots
Authors:
Stefano Dafarra,
Ugo Pattacini,
Giulio Romualdi,
Lorenzo Rapetti,
Riccardo Grieco,
Kourosh Darvish,
Gianluca Milani,
Enrico Valli,
Ines Sorrentino,
Paolo Maria Viceconte,
Alessandro Scalzo,
Silvio Traversaro,
Carlotta Sartore,
Mohamed Elobaid,
Nuno Guedelha,
Connor Herron,
Alexander Leonessa,
Francesco Draicchio,
Giorgio Metta,
Marco Maggiali,
Daniele Pucci
Abstract:
We present an avatar system designed to facilitate the embodiment of humanoid robots by human operators, validated through iCub3, a humanoid developed at the Istituto Italiano di Tecnologia (IIT). More precisely, the contribution of the paper is twofold: first, we present the humanoid iCub3 as a robotic avatar which integrates the latest significant improvements after about fifteen years of develo…
▽ More
We present an avatar system designed to facilitate the embodiment of humanoid robots by human operators, validated through iCub3, a humanoid developed at the Istituto Italiano di Tecnologia (IIT). More precisely, the contribution of the paper is twofold: first, we present the humanoid iCub3 as a robotic avatar which integrates the latest significant improvements after about fifteen years of development of the iCub series; second, we present a versatile avatar system enabling humans to embody humanoid robots encompassing aspects such as locomotion, manipulation, voice, and face expressions with comprehensive sensory feedback including visual, auditory, haptic, weight, and touch modalities. We validate the system by implementing several avatar architecture instances, each tailored to specific requirements. First, we evaluated the optimized architecture for verbal, non-verbal, and physical interactions with a remote recipient. This testing involved the operator in Genoa and the avatar in the Biennale di Venezia, Venice - about 290 Km away - thus allowing the operator to visit remotely the Italian art exhibition. Second, we evaluated the optimised architecture for recipient physical collaboration and public engagement on-stage, live, at the We Make Future show, a prominent world digital innovation festival. In this instance, the operator was situated in Genoa while the avatar operates in Rimini - about 300 Km away - interacting with a recipient who entrusted the avatar a payload to carry on stage before an audience of approximately 2000 spectators. Third, we present the architecture implemented by the iCub Team for the ANA Avatar XPrize competition.
△ Less
Submitted 25 January, 2024; v1 submitted 14 March, 2022;
originally announced March 2022.
-
Telexistence and Teleoperation for Walking Humanoid Robots
Authors:
Mohamed Elobaid,
Yue Hu,
Giulio Romualdi,
Stefano Dafarra,
Jan Babic,
Daniele Pucci
Abstract:
This paper proposes an architecture for achieving telexistence and teleoperation of humanoid robots. The architecture combines several technological set-ups, methodologies, locomotion and manipulation algorithms in a novel manner, thus building upon and extending works available in literature. The approach allows a human operator to command and telexist with the robot. Therefore, in this work we t…
▽ More
This paper proposes an architecture for achieving telexistence and teleoperation of humanoid robots. The architecture combines several technological set-ups, methodologies, locomotion and manipulation algorithms in a novel manner, thus building upon and extending works available in literature. The approach allows a human operator to command and telexist with the robot. Therefore, in this work we treat aspects pertaining not only to the proposed architecture structure and implementation, but also the human operator experience in terms of ability to adapt to the robot and to the architecture. Also the proprioception aspects and embodiment of the robot are studied through specific experimental results, which are also treated in a somewhat formal, albeit high-level manner. Application of the proposed architecture and experiments incorporating user training and experience are addressed using an illustrative bipedal humanoid robot, namely the iCub robot.
△ Less
Submitted 4 April, 2019; v1 submitted 5 September, 2018;
originally announced September 2018.
