Showing 1–50 of 94 results for author: Kawaharazuka, K

KLEIYN : A Quadruped Robot with an Active Waist for Both Locomotion and Wall Climbing

Authors: Keita Yoneda, Kento Kawaharazuka, Temma Suzuki, Takahiro Hattori, Kei Okada

Abstract: In recent years, advancements in hardware have enabled quadruped robots to operate with high power and speed, while robust locomotion control using reinforcement learning (RL) has also been realized. As a result, expectations are rising for the automation of tasks such as material transport and exploration in unknown environments. However, autonomous locomotion in rough terrains with significant h… ▽ More In recent years, advancements in hardware have enabled quadruped robots to operate with high power and speed, while robust locomotion control using reinforcement learning (RL) has also been realized. As a result, expectations are rising for the automation of tasks such as material transport and exploration in unknown environments. However, autonomous locomotion in rough terrains with significant height variations requires vertical movement, and robots capable of performing such movements stably, along with their control methods, have not yet been fully established. In this study, we developed the quadruped robot KLEIYN, which features a waist joint, and aimed to expand quadruped locomotion by enabling chimney climbing through RL. To facilitate the learning of vertical motion, we introduced Contact-Guided Curriculum Learning (CGCL). As a result, KLEIYN successfully climbed walls ranging from 800 mm to 1000 mm in width at an average speed of 150 mm/s, 50 times faster than conventional robots. Furthermore, we demonstrated that the introduction of a waist joint improves climbing performance, particularly enhancing tracking ability on narrow walls. △ Less

Submitted 10 July, 2025; v1 submitted 9 July, 2025; originally announced July 2025.

Comments: Accepted at IROS2025, website - https://keitayoneda.github.io/kleiyn-chimney-climbing/, YouTube - https://www.youtube.com/watch?v=cLfUhyNFOeY

Design Optimization of Three-Dimensional Wire Arrangement Considering Wire Crossings for Tendon-driven Robots

Authors: Kento Kawaharazuka, Shintaro Inoue, Yuta Sahara, Keita Yoneda, Temma Suzuki, Kei Okada

Abstract: Tendon-driven mechanisms are useful from the perspectives of variable stiffness, redundant actuation, and lightweight design, and they are widely used, particularly in hands, wrists, and waists of robots. The design of these wire arrangements has traditionally been done empirically, but it becomes extremely challenging when dealing with complex structures. Various studies have attempted to optimiz… ▽ More Tendon-driven mechanisms are useful from the perspectives of variable stiffness, redundant actuation, and lightweight design, and they are widely used, particularly in hands, wrists, and waists of robots. The design of these wire arrangements has traditionally been done empirically, but it becomes extremely challenging when dealing with complex structures. Various studies have attempted to optimize wire arrangement, but many of them have oversimplified the problem by imposing conditions such as restricting movements to a 2D plane, keeping the moment arm constant, or neglecting wire crossings. Therefore, this study proposes a three-dimensional wire arrangement optimization that takes wire crossings into account. We explore wire arrangements through a multi-objective black-box optimization method that ensures wires do not cross while providing sufficient joint torque along a defined target trajectory. For a 3D link structure, we optimize the wire arrangement under various conditions, demonstrate its effectiveness, and discuss the obtained design solutions. △ Less

Submitted 5 July, 2025; originally announced July 2025.

Comments: Accepted at IROS2025, Website - https://haraduka.github.io/muscle-3d-opt/ , YouTube - https://www.youtube.com/watch?v=cy510s-kOaY

DIJE: Dense Image Jacobian Estimation for Robust Robotic Self-Recognition and Visual Servoing

Authors: Yasunori Toshimitsu, Kento Kawaharazuka, Akihiro Miki, Kei Okada, Masayuki Inaba

Abstract: For robots to move in the real world, they must first correctly understand the state of its own body and the tools that it holds. In this research, we propose DIJE, an algorithm to estimate the image Jacobian for every pixel. It is based on an optical flow calculation and a simplified Kalman Filter that can be efficiently run on the whole image in real time. It does not rely on markers nor knowled… ▽ More For robots to move in the real world, they must first correctly understand the state of its own body and the tools that it holds. In this research, we propose DIJE, an algorithm to estimate the image Jacobian for every pixel. It is based on an optical flow calculation and a simplified Kalman Filter that can be efficiently run on the whole image in real time. It does not rely on markers nor knowledge of the robotic structure. We use the DIJE in a self-recognition process which can robustly distinguish between movement by the robot and by external entities, even when the motion overlaps. We also propose a visual servoing controller based on DIJE, which can learn to control the robot's body to conduct reaching movements or bimanual tool-tip control. The proposed algorithms were implemented on a physical musculoskeletal robot and its performance was verified. We believe that such global estimation of the visuomotor policy has the potential to be extended into a more general framework for manipulation. △ Less

Submitted 1 July, 2025; originally announced July 2025.

Comments: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Journal ref: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 2022, pp. 2219-2226

PIMBS: Efficient Body Schema Learning for Musculoskeletal Humanoids with Physics-Informed Neural Networks

Authors: Kento Kawaharazuka, Takahiro Hattori, Keita Yoneda, Kei Okada

Abstract: Musculoskeletal humanoids are robots that closely mimic the human musculoskeletal system, offering various advantages such as variable stiffness control, redundancy, and flexibility. However, their body structure is complex, and muscle paths often significantly deviate from geometric models. To address this, numerous studies have been conducted to learn body schema, particularly the relationships… ▽ More Musculoskeletal humanoids are robots that closely mimic the human musculoskeletal system, offering various advantages such as variable stiffness control, redundancy, and flexibility. However, their body structure is complex, and muscle paths often significantly deviate from geometric models. To address this, numerous studies have been conducted to learn body schema, particularly the relationships among joint angles, muscle tension, and muscle length. These studies typically rely solely on data collected from the actual robot, but this data collection process is labor-intensive, and learning becomes difficult when the amount of data is limited. Therefore, in this study, we propose a method that applies the concept of Physics-Informed Neural Networks (PINNs) to the learning of body schema in musculoskeletal humanoids, enabling high-accuracy learning even with a small amount of data. By utilizing not only data obtained from the actual robot but also the physical laws governing the relationship between torque and muscle tension under the assumption of correct joint structure, more efficient learning becomes possible. We apply the proposed method to both simulation and an actual musculoskeletal humanoid and discuss its effectiveness and characteristics. △ Less

Submitted 5 July, 2025; v1 submitted 25 June, 2025; originally announced June 2025.

Comments: Accepted at IEEE Robotics and Automation Letters, Website - https://haraduka.github.io/pinn-body-schema/

Development of a Five-Fingerd Biomimetic Soft Robotic Hand by 3D Printing the Skin and Skeleton as One Unit

Authors: Kazuhiro Miyama, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: Robot hands that imitate the shape of the human body have been actively studied, and various materials and mechanisms have been proposed to imitate the human body. Although the use of soft materials is advantageous in that it can imitate the characteristics of the human body's epidermis, it increases the number of parts and makes assembly difficult in order to perform complex movements. In this st… ▽ More Robot hands that imitate the shape of the human body have been actively studied, and various materials and mechanisms have been proposed to imitate the human body. Although the use of soft materials is advantageous in that it can imitate the characteristics of the human body's epidermis, it increases the number of parts and makes assembly difficult in order to perform complex movements. In this study, we propose a skin-skeleton integrated robot hand that has 15 degrees of freedom and consists of four parts. The developed robotic hand is mostly composed of a single flexible part produced by a 3D printer, and while it can be easily assembled, it can perform adduction, flexion, and opposition of the thumb, as well as flexion of four fingers. △ Less

Submitted 2 March, 2025; originally announced March 2025.

Comments: Accepted at IROS2023

Stability Recognition with Active Vibration for Bracing Behaviors and Motion Extensions Using Environment in Musculoskeletal Humanoids

Authors: Kento Kawaharazuka, Manabu Nishiura, Shinsuke Nakashima, Yasunori Toshimitsu, Yusuke Omura, Yuya Koga, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: Although robots with flexible bodies are superior in terms of the contact and adaptability, it is difficult to control them precisely. On the other hand, human beings make use of the surrounding environments to stabilize their bodies and control their movements. In this study, we propose a method for the bracing motion and extension of the range of motion using the environment for the musculoskele… ▽ More Although robots with flexible bodies are superior in terms of the contact and adaptability, it is difficult to control them precisely. On the other hand, human beings make use of the surrounding environments to stabilize their bodies and control their movements. In this study, we propose a method for the bracing motion and extension of the range of motion using the environment for the musculoskeletal humanoid. Here, it is necessary to recognize the stability of the body when contacting the environment, and we develop a method to measure it by using the change in sensor values of the body when actively vibrating a part of the body. Experiments are conducted using the musculoskeletal humanoid Musashi, and the effectiveness of this method is confirmed. △ Less

Submitted 22 February, 2025; originally announced February 2025.

Comments: Accepted at RoboSoft2021

Reflex-based Motion Strategy of Musculoskeletal Humanoids under Environmental Contact Using Muscle Relaxation Control

Authors: Kento Kawaharazuka, Kei Tsuzuki, Moritaka Onitsuka, Yuya Koga, Yusuke Omura, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: The musculoskeletal humanoid can move well under environmental contact thanks to its body softness. However, there are few studies that actively make use of the environment to rest its flexible musculoskeletal body. Also, its complex musculoskeletal structure is difficult to modelize and high internal muscle tension sometimes occurs. To solve these problems, we develop a muscle relaxation control… ▽ More The musculoskeletal humanoid can move well under environmental contact thanks to its body softness. However, there are few studies that actively make use of the environment to rest its flexible musculoskeletal body. Also, its complex musculoskeletal structure is difficult to modelize and high internal muscle tension sometimes occurs. To solve these problems, we develop a muscle relaxation control which can minimize the muscle tension by actively using the environment and inhibit useless internal muscle tension. We apply this control to some basic movements, the motion of resting the arms on the desk, and handle operation, and verify its effectiveness. △ Less

Submitted 22 February, 2025; originally announced February 2025.

Comments: Accepted at Humanoids2019

Online Learning of Danger Avoidance for Complex Structures of Musculoskeletal Humanoids and Its Applications

Authors: Kento Kawaharazuka, Naoki Hiraoka, Yuya Koga, Manabu Nishiura, Yusuke Omura, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: The complex structure of musculoskeletal humanoids makes it difficult to model them, and the inter-body interference and high internal muscle force are unavoidable. Although various safety mechanisms have been developed to solve this problem, it is important not only to deal with the dangers when they occur but also to prevent them from happening. In this study, we propose a method to learn a netw… ▽ More The complex structure of musculoskeletal humanoids makes it difficult to model them, and the inter-body interference and high internal muscle force are unavoidable. Although various safety mechanisms have been developed to solve this problem, it is important not only to deal with the dangers when they occur but also to prevent them from happening. In this study, we propose a method to learn a network outputting danger probability corresponding to the muscle length online so that the robot can gradually prevent dangers from occurring. Applications of this network for control are also described. The method is applied to the musculoskeletal humanoid, Musashi, and its effectiveness is verified. △ Less

Submitted 22 February, 2025; originally announced February 2025.

Comments: Accepted at Humanoids2020

Applications of Stretch Reflex for the Upper Limb of Musculoskeletal Humanoids: Protective Behavior, Postural Stability, and Active Induction

Authors: Kento Kawaharazuka, Yuya Koga, Kei Tsuzuki, Moritaka Onitsuka, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: The musculoskeletal humanoid has various biomimetic benefits, and it is important that we can embed and evaluate human reflexes in the actual robot. Although stretch reflex has been implemented in lower limbs of musculoskeletal humanoids, we apply it to the upper limb to discover its useful applications. We consider the implementation of stretch reflex in the actual robot, its active/passive appli… ▽ More The musculoskeletal humanoid has various biomimetic benefits, and it is important that we can embed and evaluate human reflexes in the actual robot. Although stretch reflex has been implemented in lower limbs of musculoskeletal humanoids, we apply it to the upper limb to discover its useful applications. We consider the implementation of stretch reflex in the actual robot, its active/passive applications, and the change in behavior according to the difference of parameters. △ Less

Submitted 18 February, 2025; originally announced February 2025.

Comments: Accepted at IROS2020

Exceeding the Maximum Speed Limit of the Joint Angle for the Redundant Tendon-driven Structures of Musculoskeletal Humanoids

Authors: Kento Kawaharazuka, Yuya Koga, Kei Tsuzuki, Moritaka Onitsuka, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: The musculoskeletal humanoid has various biomimetic benefits, and the redundant muscle arrangement is one of its most important characteristics. This redundancy can achieve fail-safe redundant actuation and variable stiffness control. However, there is a problem that the maximum joint angle velocity is limited by the slowest muscle among the redundant muscles. In this study, we propose two methods… ▽ More The musculoskeletal humanoid has various biomimetic benefits, and the redundant muscle arrangement is one of its most important characteristics. This redundancy can achieve fail-safe redundant actuation and variable stiffness control. However, there is a problem that the maximum joint angle velocity is limited by the slowest muscle among the redundant muscles. In this study, we propose two methods that can exceed the limited maximum joint angle velocity, and verify the effectiveness with actual robot experiments. △ Less

Submitted 18 February, 2025; originally announced February 2025.

Comments: Accepted at IROS2020

Design Optimization of Musculoskeletal Humanoids with Maximization of Redundancy to Compensate for Muscle Rupture

Authors: Kento Kawaharazuka, Yasunori Toshimitsu, Manabu Nishiura, Yuya Koga, Yusuke Omura, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: Musculoskeletal humanoids have various biomimetic advantages, and the redundant muscle arrangement allowing for variable stiffness control is one of the most important. In this study, we focus on one feature of the redundancy, which enables the humanoid to keep moving even if one of its muscles breaks, an advantage that has not been dealt with in many studies. In order to make the most of this adv… ▽ More Musculoskeletal humanoids have various biomimetic advantages, and the redundant muscle arrangement allowing for variable stiffness control is one of the most important. In this study, we focus on one feature of the redundancy, which enables the humanoid to keep moving even if one of its muscles breaks, an advantage that has not been dealt with in many studies. In order to make the most of this advantage, the design of muscle arrangement is optimized by considering the maximization of minimum available torque that can be exerted when one muscle breaks. This method is applied to the elbow of a musculoskeletal humanoid Musashi with simulations, the design policy is extracted from the optimization results, and its effectiveness is confirmed with the actual robot. △ Less

Submitted 18 February, 2025; originally announced February 2025.

Comments: Accepted at IROS2021

Learning Quiet Walking for a Small Home Robot

Authors: Ryo Watanabe, Takahiro Miki, Fan Shi, Yuki Kadokawa, Filip Bjelonic, Kento Kawaharazuka, Andrei Cramariuc, Marco Hutter

Abstract: As home robotics gains traction, robots are increasingly integrated into households, offering companionship and assistance. Quadruped robots, particularly those resembling dogs, have emerged as popular alternatives for traditional pets. However, user feedback highlights concerns about the noise these robots generate during walking at home, particularly the loud footstep sound. To address this issu… ▽ More As home robotics gains traction, robots are increasingly integrated into households, offering companionship and assistance. Quadruped robots, particularly those resembling dogs, have emerged as popular alternatives for traditional pets. However, user feedback highlights concerns about the noise these robots generate during walking at home, particularly the loud footstep sound. To address this issue, we propose a sim-to-real based reinforcement learning (RL) approach to minimize the foot contact velocity highly related to the footstep sound. Our framework incorporates three key elements: learning varying PD gains to actively dampen and stiffen each joint, utilizing foot contact sensors, and employing curriculum learning to gradually enforce penalties on foot contact velocity. Experiments demonstrate that our learned policy achieves superior quietness compared to a RL baseline and the carefully handcrafted Sony commercial controllers. Furthermore, the trade-off between robustness and quietness is shown. This research contributes to developing quieter and more user-friendly robotic companions in home environments. △ Less

Submitted 28 February, 2025; v1 submitted 15 February, 2025; originally announced February 2025.

Comments: accepted at ICRA 2025

Front Hair Styling Robot System Using Path Planning for Root-Centric Strand Adjustment

Authors: Soonhyo Kim, Naoaki Kanazawa, Shun Hasegawa, Kento Kawaharazuka, Kei Okada

Abstract: Hair styling is a crucial aspect of personal grooming, significantly influenced by the appearance of front hair. While brushing is commonly used both to detangle hair and for styling purposes, existing research primarily focuses on robotic systems for detangling hair, with limited exploration into robotic hair styling. This research presents a novel robotic system designed to automatically adjust… ▽ More Hair styling is a crucial aspect of personal grooming, significantly influenced by the appearance of front hair. While brushing is commonly used both to detangle hair and for styling purposes, existing research primarily focuses on robotic systems for detangling hair, with limited exploration into robotic hair styling. This research presents a novel robotic system designed to automatically adjust front hairstyles, with an emphasis on path planning for root-centric strand adjustment. The system utilizes images to compare the current hair state with the desired target state through an orientation map of hair strands. By concentrating on the differences in hair orientation and specifically targeting adjustments at the root of each strand, the system performs detailed styling tasks. The path planning approach ensures effective alignment of the hairstyle with the target, and a closed-loop mechanism refines these adjustments to accurately evolve the hairstyle towards the desired outcome. Experimental results demonstrate that the proposed system achieves a high degree of similarity and consistency in front hair styling, showing promising results for automated, precise hairstyle adjustments. △ Less

Submitted 28 January, 2025; v1 submitted 19 January, 2025; originally announced January 2025.

Comments: Accepted at IEEE/SICE SII2025

Environmentally Adaptive Control Including Variance Minimization Using Stochastic Predictive Network with Parametric Bias: Application to Mobile Robots

Authors: Kento Kawaharazuka, Koki Shinjo, Yoichiro Kawamura, Kei Okada, Masayuki Inaba

Abstract: In this study, we propose a predictive model composed of a recurrent neural network including parametric bias and stochastic elements, and an environmentally adaptive robot control method including variance minimization using the model. Robots which have flexible bodies or whose states can only be partially observed are difficult to modelize, and their predictive models often have stochastic behav… ▽ More In this study, we propose a predictive model composed of a recurrent neural network including parametric bias and stochastic elements, and an environmentally adaptive robot control method including variance minimization using the model. Robots which have flexible bodies or whose states can only be partially observed are difficult to modelize, and their predictive models often have stochastic behaviors. In addition, the physical state of the robot and the surrounding environment change sequentially, and so the predictive model can change online. Therefore, in this study, we construct a learning-based stochastic predictive model implemented in a neural network embedded with such information from the experience of the robot, and develop a control method for the robot to avoid unstable motion with large variance while adapting to the current environment. This method is verified through a mobile robot in simulation and to the actual robot Fetch. △ Less

Submitted 11 December, 2024; originally announced December 2024.

Comments: Accepted at IROS2021

Task-specific Self-body Controller Acquisition by Musculoskeletal Humanoids: Application to Pedal Control in Autonomous Driving

Authors: Kento Kawaharazuka, Kei Tsuzuki, Shogo Makino, Moritaka Onitsuka, Koki Shinjo, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: The musculoskeletal humanoid has many benefits that human beings have, but the modeling of its complex flexible body is difficult. Although we have developed an online acquisition method of the nonlinear relationship between joints and muscles, we could not completely match the actual robot and its self-body image. When realizing a certain task, the direct relationship between the control input an… ▽ More The musculoskeletal humanoid has many benefits that human beings have, but the modeling of its complex flexible body is difficult. Although we have developed an online acquisition method of the nonlinear relationship between joints and muscles, we could not completely match the actual robot and its self-body image. When realizing a certain task, the direct relationship between the control input and task state needs to be learned. So, we construct a neural network representing the time-series relationship between the control input and task state, and realize the intended task state by applying the network to a real-time control. In this research, we conduct accelerator pedal control experiments as one application, and verify the effectiveness of this study. △ Less

Submitted 11 December, 2024; originally announced December 2024.

Comments: Accepted at IROS2019

Modification of muscle antagonistic relations and hand trajectory on the dynamic motion of Musculoskeletal Humanoid

Authors: Yuya Koga, Kento Kawaharazuka, Moritaka Onitsuka, Tasuku Makabe, Kei Tsuzuki, Yusuke Omura, Yuki Asano, Kei Okada, Masayuki Inaba

Abstract: In recent years, some research on musculoskeletal humanoids is in progress. However, there are some challenges such as unmeasurable transformation of body structure and muscle path, and difficulty in measuring own motion because of lack of joint angle sensor. In this study, we suggest two motion acquisition methods. One is a method to acquire antagonistic relations of muscles by tension sensing, a… ▽ More In recent years, some research on musculoskeletal humanoids is in progress. However, there are some challenges such as unmeasurable transformation of body structure and muscle path, and difficulty in measuring own motion because of lack of joint angle sensor. In this study, we suggest two motion acquisition methods. One is a method to acquire antagonistic relations of muscles by tension sensing, and the other is a method to acquire correct hand trajectory by vision sensing. Finally, we realize badminton shuttlecock-hitting motion of Kengoro with these two acquisition methods. △ Less

Submitted 1 December, 2024; originally announced December 2024.

Comments: Accepted at Humanoids2019

Design of a Five-Fingered Hand with Full-Fingered Tactile Sensors Using Conductive Filaments and Its Application to Bending after Insertion Motion

Authors: Kazuhiro Miyama, Shun Hasegawa, Kento Kawaharazuka, Naoya Yamaguchi, Kei Okada, Masayuki Inaba

Abstract: The purpose of this study is to construct a contact point estimation system for the both side of a finger, and to realize a motion of bending the finger after inserting the finger into a tool (hereinafter referred to as the bending after insertion motion). In order to know the contact points of the full finger including the joints, we propose to fabricate a nerve inclusion flexible epidermis by co… ▽ More The purpose of this study is to construct a contact point estimation system for the both side of a finger, and to realize a motion of bending the finger after inserting the finger into a tool (hereinafter referred to as the bending after insertion motion). In order to know the contact points of the full finger including the joints, we propose to fabricate a nerve inclusion flexible epidermis by combining a flexible epidermis and a nerve line made of conductive filaments, and estimate the contact position from the change of resistance of the nerve line. A nerve inclusion flexible epidermis attached to a thin fingered robotic hand was combined with a twin-armed robot and tool use experiments were conducted. The contact information can be used for tool use, confirming the effectiveness of the proposed method. △ Less

Submitted 1 December, 2024; originally announced December 2024.

Comments: Accepted at Humanoids2022

Remote Life Support Robot Interface System for Global Task Planning and Local Action Expansion Using Foundation Models

Authors: Yoshiki Obinata, Haoyu Jia, Kento Kawaharazuka, Naoaki Kanazawa, Kei Okada

Abstract: Robot systems capable of executing tasks based on language instructions have been actively researched. It is challenging to convey uncertain information that can only be determined on-site with a single language instruction to the robot. In this study, we propose a system that includes ambiguous parts as template variables in language instructions to communicate the information to be collected and… ▽ More Robot systems capable of executing tasks based on language instructions have been actively researched. It is challenging to convey uncertain information that can only be determined on-site with a single language instruction to the robot. In this study, we propose a system that includes ambiguous parts as template variables in language instructions to communicate the information to be collected and the options to be presented to the robot for predictable uncertain events. This study implements prompt generation for each robot action function based on template variables to collect information, and a feedback system for presenting and selecting options based on template variables for user-to-robot communication. The effectiveness of the proposed system was demonstrated through its application to real-life support tasks performed by the robot. △ Less

Submitted 15 November, 2024; originally announced November 2024.

Comments: Accepted to 2024 IEEE-RAS International Conference on Humanoids Robots (Humanoids 2024)

Vlimb: A Wire-Driven Wearable Robot for Bodily Extension, Balancing Powerfulness and Reachability

Authors: Shogo Sawaguchi, Temma Suzuki, Akihiro Miki, Kento Kawaharazuka, Sota Yuzaki, Shunnosuke Yoshimura, Yoshimoto Ribayashi, Kei Okada, Masayuki Inaba

Abstract: Numerous wearable robots have been developed to meet the demands of physical assistance and entertainment. These wearable robots range from body-enhancing types that assist human arms and legs to body-extending types that have extra arms. This study focuses specifically on wearable robots of the latter category, aimed at bodily extension. However, they have not yet achieved the level of powerfulne… ▽ More Numerous wearable robots have been developed to meet the demands of physical assistance and entertainment. These wearable robots range from body-enhancing types that assist human arms and legs to body-extending types that have extra arms. This study focuses specifically on wearable robots of the latter category, aimed at bodily extension. However, they have not yet achieved the level of powerfulness and reachability equivalent to that of human limbs, limiting their application to entertainment and manipulation tasks involving lightweight objects. Therefore, in this study, we develop an body-extending wearable robot, Vlimb, which has enough powerfulness to lift a human and can perform manipulation. Leveraging the advantages of tendon-driven mechanisms, Vlimb incorporates a wire routing mechanism capable of accommodating both delicate manipulations and robust lifting tasks. Moreover, by introducing a passive ring structure to overcome the limited reachability inherent in tendon-driven mechanisms, Vlimb achieves both the powerfulness and reachability comparable to that of humans. This paper outlines the design methodology of Vlimb, conducts preliminary manipulation and lifting tasks, and verifies its effectiveness. △ Less

Submitted 14 November, 2024; originally announced November 2024.

Integrative Wrapping System for a Dual-Arm Humanoid Robot

Authors: Yukina Iwata, Shun Hasegawa, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: Flexible object manipulation of paper and cloth is a major research challenge in robot manipulation. Although there have been efforts to develop hardware that enables specific actions and to realize a single action of paper folding using sim-to-real and learning, there have been few proposals for humanoid robots and systems that enable continuous, multi-step actions of flexible materials. Wrapping… ▽ More Flexible object manipulation of paper and cloth is a major research challenge in robot manipulation. Although there have been efforts to develop hardware that enables specific actions and to realize a single action of paper folding using sim-to-real and learning, there have been few proposals for humanoid robots and systems that enable continuous, multi-step actions of flexible materials. Wrapping an object with paper and tape is more complex and diverse than traditional manipulation research due to the increased number of objects that need to be handled, as well as the three-dimensionality of the operation. In this research, necessary information is organized and coded based on the characteristics of each object handled in wrapping. We also generalize the hardware configuration, manipulation method, and recognition system that enable humanoid wrapping operations. The system will include manipulation with admittance control focusing on paper tension and state evaluation using point clouds to handle three-dimensional flexible objects. Finally, wrapping objects with different shapes is experimented with to show the generality and effectiveness of the proposed system. △ Less

Submitted 13 November, 2024; originally announced November 2024.

Comments: Accepted Humanoids2024

Motion Modification Method of Musculoskeletal Humanoids by Human Teaching Using Muscle-Based Compensation Control

Authors: Kento Kawaharazuka, Yuya Koga, Manabu Nishiura, Yusuke Omura, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: While musculoskeletal humanoids have the advantages of various biomimetic structures, it is difficult to accurately control the body, which is challenging to model. Although various learning-based control methods have been developed so far, they cannot completely absorb model errors, and recognition errors are also bound to occur. In this paper, we describe a method to modify the movement of the m… ▽ More While musculoskeletal humanoids have the advantages of various biomimetic structures, it is difficult to accurately control the body, which is challenging to model. Although various learning-based control methods have been developed so far, they cannot completely absorb model errors, and recognition errors are also bound to occur. In this paper, we describe a method to modify the movement of the musculoskeletal humanoid by applying external force during the movement, taking advantage of its flexible body. Considering the fact that the joint angles cannot be measured, and that the external force greatly affects the nonlinear elastic element and not the actuator, the modified motion is reproduced by the proposed muscle-based compensation control. This method is applied to a musculoskeletal humanoid, Musashi, and its effectiveness is confirmed. △ Less

Submitted 9 November, 2024; originally announced November 2024.

Comments: Acceptedt at Humanoids2020

Adaptive Body Schema Learning System Considering Additional Muscles for Musculoskeletal Humanoids

Authors: Kento Kawaharazuka, Akihiro Miki, Yasunori Toshimitsu, Kei Okada, Masayuki Inaba

Abstract: One of the important advantages of musculoskeletal humanoids is that the muscle arrangement can be easily changed and the number of muscles can be increased according to the situation. In this study, we describe an overall system of muscle addition for musculoskeletal humanoids and the adaptive body schema learning while taking into account the additional muscles. For hardware, we describe a modul… ▽ More One of the important advantages of musculoskeletal humanoids is that the muscle arrangement can be easily changed and the number of muscles can be increased according to the situation. In this study, we describe an overall system of muscle addition for musculoskeletal humanoids and the adaptive body schema learning while taking into account the additional muscles. For hardware, we describe a modular body design that can be fitted with additional muscles, and for software, we describe a method that can learn the changes in body schema associated with additional muscles from a small amount of motion data. We apply our method to a simple 1-DOF tendon-driven robot simulation and the arm of the musculoskeletal humanoid Musashi, and show the effectiveness of muscle tension relaxation by adding muscles for a high-load task. △ Less

Submitted 9 November, 2024; originally announced November 2024.

Comments: Accepted at IEEE Robotics and Automation Letters

Self-Body Image Acquisition and Posture Generation with Redundancy using Musculoskeletal Humanoid Shoulder Complex for Object Manipulation

Authors: Yuya Koga, Kento Kawaharazuka, Yasunori Toshimitsu, Manabu Nishiura, Yusuke Omura, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: We proposed a method for learning the actual body image of a musculoskeletal humanoid for posture generation and object manipulation using inverse kinematics with redundancy in the shoulder complex. The effectiveness of this method was confirmed by realizing automobile steering wheel operation. The shoulder complex has a scapula that glides over the rib cage and an open spherical joint, and is sup… ▽ More We proposed a method for learning the actual body image of a musculoskeletal humanoid for posture generation and object manipulation using inverse kinematics with redundancy in the shoulder complex. The effectiveness of this method was confirmed by realizing automobile steering wheel operation. The shoulder complex has a scapula that glides over the rib cage and an open spherical joint, and is supported by numerous muscle groups, enabling a wide range of motion. As a development of the human mimetic shoulder complex, we have increased the muscle redundancy by implementing deep muscles and stabilize the joint drive. As a posture generation method to utilize the joint redundancy of the shoulder complex, we consider inverse kinematics based on the scapular drive strategy suggested by the scapulohumeral rhythm of the human body. In order to control a complex robot imitating a human body, it is essential to learn its own body image, but it is difficult to know its own state accurately due to its deformation which is difficult to measure. To solve this problem, we developed a method to acquire a self-body image that can be updated appropriately by recognizing the hand position relative to an object for the purpose of object manipulation. We apply the above methods to a full-body musculoskeletal humanoid, Kengoro, and confirm its effectiveness by conducting an experiment to operate a car steering wheel, which requires the appropriate use of both arms. △ Less

Submitted 9 November, 2024; originally announced November 2024.

Comments: Accepted at IEEE Robotics and Automation Letters

Fundamental Three-Dimensional Configuration of Wire-Wound Muscle-Tendon Complex Drive

Authors: Yoshimoto Ribayashi, Yuta Sahara, Shogo Sawaguchi, Kazuhiro Miyama, Akihiro Miki, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: For robots to become more versatile and expand their areas of application, their bodies need to be suitable for contact with the environment. When the human body comes into contact with the environment, it is possible for it to continue to move even if the positional relationship between muscles or the shape of the muscles changes. We have already focused on the effect of geometric deformation of… ▽ More For robots to become more versatile and expand their areas of application, their bodies need to be suitable for contact with the environment. When the human body comes into contact with the environment, it is possible for it to continue to move even if the positional relationship between muscles or the shape of the muscles changes. We have already focused on the effect of geometric deformation of muscles and proposed a drive system called wire-wound Muscle-Tendon Complex (ww-MTC), an extension of the wire drive system. Our previous study using a robot with a two-dimensional configuration demonstrated several advantages: reduced wire loosening, interference, and wear; improved robustness during environmental contact; and a muscular appearance. However, this design had some problems, such as excessive muscle expansion that hindered inter-muscle movement, and confinement to planar motion. In this study, we develop the ww-MTC into a three-dimensional shape. We present a fundamental construction method for a muscle exterior that expands gently and can be contacted over its entire surface. We also apply the three-dimensional ww-MTC to a 2-axis 3-muscle robot, and confirm that the robot can continue to move while adapting to its environment. △ Less

Submitted 6 November, 2024; originally announced November 2024.

Comments: Accepted at Humanoids2024, website - https://sites.google.com/view/yoshimoto-ribayashi/projects, YouTube - https://youtu.be/EDeAqg7aAb4

CubiXMusashi: Fusion of Wire-Driven CubiX and Musculoskeletal Humanoid Musashi toward Unlimited Performance

Authors: Shintaro Inoue, Kento Kawaharazuka, Temma Suzuki, Sota Yuzaki, Yoshimoto Ribayashi, Yuta Sahara, Kei Okada

Abstract: Humanoids exhibit a wide variety in terms of joint configuration, actuators, and degrees of freedom, resulting in different achievable movements and tasks for each type. Particularly, musculoskeletal humanoids are developed to closely emulate human body structure and movement functions, consisting of a skeletal framework driven by numerous muscle actuators. The redundant arrangement of muscles rel… ▽ More Humanoids exhibit a wide variety in terms of joint configuration, actuators, and degrees of freedom, resulting in different achievable movements and tasks for each type. Particularly, musculoskeletal humanoids are developed to closely emulate human body structure and movement functions, consisting of a skeletal framework driven by numerous muscle actuators. The redundant arrangement of muscles relative to the skeletal degrees of freedom has been used to represent the flexible and complex body movements observed in humans. However, due to this flexible body and high degrees of freedom, modeling, simulation, and control become extremely challenging, limiting the feasible movements and tasks. In this study, we integrate the musculoskeletal humanoid Musashi with the wire-driven robot CubiX, capable of connecting to the environment, to form CubiXMusashi. This combination addresses the shortcomings of traditional musculoskeletal humanoids and enables movements beyond the capabilities of other humanoids. CubiXMusashi connects to the environment with wires and drives by winding them, successfully achieving movements such as pull-up, rising from a lying pose, and mid-air kicking, which are difficult for Musashi alone. This concept demonstrates that various humanoids, not limited to musculoskeletal humanoids, can mitigate their physical constraints and acquire new abilities by connecting to the environment and driving through wires. △ Less

Submitted 31 October, 2024; originally announced October 2024.

Comments: Accepted Humanoids2024, website - https://shin0805.github.io/cubixmusashi/, YouTube - https://youtu.be/IvzP98-r_mo

Robotic State Recognition with Image-to-Text Retrieval Task of Pre-Trained Vision-Language Model and Black-Box Optimization

Authors: Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba

Abstract: State recognition of the environment and objects, such as the open/closed state of doors and the on/off of lights, is indispensable for robots that perform daily life support and security tasks. Until now, state recognition methods have been based on training neural networks from manual annotations, preparing special sensors for the recognition, or manually programming to extract features from poi… ▽ More State recognition of the environment and objects, such as the open/closed state of doors and the on/off of lights, is indispensable for robots that perform daily life support and security tasks. Until now, state recognition methods have been based on training neural networks from manual annotations, preparing special sensors for the recognition, or manually programming to extract features from point clouds or raw images. In contrast, we propose a robotic state recognition method using a pre-trained vision-language model, which is capable of Image-to-Text Retrieval (ITR) tasks. We prepare several kinds of language prompts in advance, calculate the similarity between these prompts and the current image by ITR, and perform state recognition. By applying the optimal weighting to each prompt using black-box optimization, state recognition can be performed with higher accuracy. Experiments show that this theory enables a variety of state recognitions by simply preparing multiple prompts without retraining neural networks or manual programming. In addition, since only prompts and their weights need to be prepared for each recognizer, there is no need to prepare multiple models, which facilitates resource management. It is possible to recognize the open/closed state of transparent doors, the state of whether water is running or not from a faucet, and even the qualitative state of whether a kitchen is clean or not, which have been challenging so far, through language. △ Less

Submitted 30 October, 2024; originally announced October 2024.

Comments: Accepted at Humanoids2024

Component Modularized Design of Musculoskeletal Humanoid Platform Musashi to Investigate Learning Control Systems

Authors: Kento Kawaharazuka, Shogo Makino, Kei Tsuzuki, Moritaka Onitsuka, Yuya Nagamatsu, Koki Shinjo, Tasuku Makabe, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: To develop Musashi as a musculoskeletal humanoid platform to investigate learning control systems, we aimed for a body with flexible musculoskeletal structure, redundant sensors, and easily reconfigurable structure. For this purpose, we develop joint modules that can directly measure joint angles, muscle modules that can realize various muscle routes, and nonlinear elastic units with soft structur… ▽ More To develop Musashi as a musculoskeletal humanoid platform to investigate learning control systems, we aimed for a body with flexible musculoskeletal structure, redundant sensors, and easily reconfigurable structure. For this purpose, we develop joint modules that can directly measure joint angles, muscle modules that can realize various muscle routes, and nonlinear elastic units with soft structures, etc. Next, we develop MusashiLarm, a musculoskeletal platform composed of only joint modules, muscle modules, generic bone frames, muscle wire units, and a few attachments. Finally, we develop Musashi, a musculoskeletal humanoid platform which extends MusashiLarm to the whole body design, and conduct several basic experiments and learning control experiments to verify the effectiveness of its concept. △ Less

Submitted 29 October, 2024; originally announced October 2024.

Comments: Accepted at IROS2019

Design Method of a Kangaroo Robot with High Power Legs and an Articulated Soft Tail

Authors: Shunnosuke Yoshimura, Temma Suzuki, Masahiro Bando, Sota Yuzaki, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: In this paper, we focus on the kangaroo, which has powerful legs capable of jumping and a soft and strong tail. To incorporate these unique structure into a robot for utilization, we propose a design method that takes into account both the feasibility as a robot and the kangaroo-mimetic structure. Based on the kangaroo's musculoskeletal structure, we determine the structure of the robot that enabl… ▽ More In this paper, we focus on the kangaroo, which has powerful legs capable of jumping and a soft and strong tail. To incorporate these unique structure into a robot for utilization, we propose a design method that takes into account both the feasibility as a robot and the kangaroo-mimetic structure. Based on the kangaroo's musculoskeletal structure, we determine the structure of the robot that enables it to jump by analyzing the muscle arrangement and prior verification in simulation. Also, to realize a tail capable of body support, we use an articulated, elastic structure as a tail. In order to achieve both softness and high power output, the robot is driven by a direct-drive, high-power wire-winding mechanism, and weight of legs and the tail is reduced by placing motors in the torso. The developed kangaroo robot can jump with its hind legs, moving its tail, and supporting its body using its hind legs and tail. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Comments: accepted at IROS2023

Patterned Structure Muscle : Arbitrary Shaped Wire-driven Artificial Muscle Utilizing Anisotropic Flexible Structure for Musculoskeletal Robots

Authors: Shunnosuke Yoshimura, Akihiro Miki, Kazuhiro Miyama, Yuta Sahara, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: Muscles of the human body are composed of tiny actuators made up of myosin and actin filaments. They can exert force in various shapes such as curved or flat, under contact forces and deformations from the environment. On the other hand, muscles in musculoskeletal robots so far have faced challenges in generating force in such shapes and environments. To address this issue, we propose Patterned St… ▽ More Muscles of the human body are composed of tiny actuators made up of myosin and actin filaments. They can exert force in various shapes such as curved or flat, under contact forces and deformations from the environment. On the other hand, muscles in musculoskeletal robots so far have faced challenges in generating force in such shapes and environments. To address this issue, we propose Patterned Structure Muscle (PSM), artificial muscles for musculoskeletal robots. PSM utilizes patterned structures with anisotropic characteristics, wire-driven mechanisms, and is made of flexible material Thermoplastic Polyurethane (TPU) using FDM 3D printing. This method enables the creation of various shapes of muscles, such as simple 1 degree-of-freedom (DOF) muscles, Multi-DOF wide area muscles, joint-covering muscles, and branched muscles. We created an upper arm structure using these muscles to demonstrate wide range of motion, lifting heavy objects, and movements through environmental contact. These experiments show that the proposed PSM is capable of operating in various shapes and environments, and is suitable for the muscles of musculoskeletal robots. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Comments: accepted at IROS2024

CubiX: Portable Wire-Driven Parallel Robot Connecting to and Utilizing the Environment

Authors: Shintaro Inoue, Kento Kawaharazuka, Temma Suzuki, Sota Yuzaki, Kei Okada, Masayuki Inaba

Abstract: A wire-driven parallel robot is a type of robotic system where multiple wires are used to control the movement of a end-effector. The wires are attached to the end-effector and anchored to fixed points on external structures. This configuration allows for the separation of actuators and end-effectors, enabling lightweight and simplified movable parts in the robot. However, its range of motion rema… ▽ More A wire-driven parallel robot is a type of robotic system where multiple wires are used to control the movement of a end-effector. The wires are attached to the end-effector and anchored to fixed points on external structures. This configuration allows for the separation of actuators and end-effectors, enabling lightweight and simplified movable parts in the robot. However, its range of motion remains confined within the space formed by the wires, limiting the wire-driven capability to only within the pre-designed operational range. Here, in this study, we develop a wire-driven robot, CubiX, capable of connecting to and utilizing the environment. CubiX connects itself to the environment using up to 8 wires and drives itself by winding these wires. By integrating actuators for winding the wires into CubiX, a portable wire-driven parallel robot is realized without limitations on its workspace. Consequently, the robot can form parallel wire-driven structures by connecting wires to the environment at any operational location. △ Less

Submitted 8 October, 2024; originally announced October 2024.

Comments: Accepted at IROS2024, website - https://shin0805.github.io/cubix-hardware/ , YouTube - https://youtu.be/R5ZrzMPEFZs

Construction of Musculoskeletal Simulation for Shoulder Complex with Ligaments and Its Validation via Model Predictive Control

Authors: Yuta Sahara, Akihiro Miki, Yoshimoto Ribayashi, Shunnosuke Yoshimura, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: The complex ways in which humans utilize their bodies in sports and martial arts are remarkable, and human motion analysis is one of the most effective tools for robot body design and control. On the other hand, motion analysis is not easy, and it is difficult to measure complex body motions in detail due to the influence of numerous muscles and soft tissues, mainly ligaments. In response, various… ▽ More The complex ways in which humans utilize their bodies in sports and martial arts are remarkable, and human motion analysis is one of the most effective tools for robot body design and control. On the other hand, motion analysis is not easy, and it is difficult to measure complex body motions in detail due to the influence of numerous muscles and soft tissues, mainly ligaments. In response, various musculoskeletal simulators have been developed and applied to motion analysis and robotics. However, none of them reproduce the ligaments but only the muscles, nor do they focus on the shoulder complex, including the clavicle and scapula, which is one of the most complex parts of the body. Therefore, in this study, a detailed simulation model of the shoulder complex including ligaments is constructed. The model will mimic not only the skeletal structure and muscle arrangement but also the ligament arrangement and maximum muscle strength. Through model predictive control based on the constructed simulation, we confirmed that the ligaments contribute to joint stabilization in the first movement and that the proper distribution of maximum muscle force contributes to the equalization of the load on each muscle, demonstrating the effectiveness of this simulation. △ Less

Submitted 8 October, 2024; originally announced October 2024.

Comments: accepted at IROS2024, websites - https://sahara-yuta.github.io/projects/shoulder-complex-simulation

Real-World Cooking Robot System from Recipes Based on Food State Recognition Using Foundation Models and PDDL

Authors: Naoaki Kanazawa, Kento Kawaharazuka, Yoshiki Obinata, Kei Okada, Masayuki Inaba

Abstract: Although there is a growing demand for cooking behaviours as one of the expected tasks for robots, a series of cooking behaviours based on new recipe descriptions by robots in the real world has not yet been realised. In this study, we propose a robot system that integrates real-world executable robot cooking behaviour planning using the Large Language Model (LLM) and classical planning of PDDL de… ▽ More Although there is a growing demand for cooking behaviours as one of the expected tasks for robots, a series of cooking behaviours based on new recipe descriptions by robots in the real world has not yet been realised. In this study, we propose a robot system that integrates real-world executable robot cooking behaviour planning using the Large Language Model (LLM) and classical planning of PDDL descriptions, and food ingredient state recognition learning from a small number of data using the Vision-Language model (VLM). We succeeded in experiments in which PR2, a dual-armed wheeled robot, performed cooking from arranged new recipes in a real-world environment, and confirmed the effectiveness of the proposed system. △ Less

Submitted 6 October, 2024; v1 submitted 3 October, 2024; originally announced October 2024.

Comments: Accepted at Advanced Robotics, website - https://kanazawanaoaki.github.io/cook-from-recipe-pddl/

Robot Design Optimization with Rotational and Prismatic Joints using Black-Box Multi-Objective Optimization

Authors: Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: Robots generally have a structure that combines rotational joints and links in a serial fashion. On the other hand, various joint mechanisms are being utilized in practice, such as prismatic joints, closed links, and wire-driven systems. Previous research have focused on individual mechanisms, proposing methods to design robots capable of achieving given tasks by optimizing the length of links and… ▽ More Robots generally have a structure that combines rotational joints and links in a serial fashion. On the other hand, various joint mechanisms are being utilized in practice, such as prismatic joints, closed links, and wire-driven systems. Previous research have focused on individual mechanisms, proposing methods to design robots capable of achieving given tasks by optimizing the length of links and the arrangement of the joints. In this study, we propose a method for the design optimization of robots that combine different types of joints, specifically rotational and prismatic joints. The objective is to automatically generate a robot that minimizes the number of joints and link lengths while accomplishing a desired task, by utilizing a black-box multi-objective optimization approach. This enables the simultaneous observation of a diverse range of body designs through the obtained Pareto solutions. Our findings confirm the emergence of practical and known combinations of rotational and prismatic joints, as well as the discovery of novel joint combinations. △ Less

Submitted 30 September, 2024; originally announced September 2024.

Comments: Accepted at IROS2024, website - https://haraduka.github.io/prismatic-joint-opt/

Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization

Authors: Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba

Abstract: In order for robots to autonomously navigate and operate in diverse environments, it is essential for them to recognize the state of their environment. On the other hand, the environmental state recognition has traditionally involved distinct methods tailored to each state to be recognized. In this study, we perform a unified environmental state recognition for robots through the spoken language w… ▽ More In order for robots to autonomously navigate and operate in diverse environments, it is essential for them to recognize the state of their environment. On the other hand, the environmental state recognition has traditionally involved distinct methods tailored to each state to be recognized. In this study, we perform a unified environmental state recognition for robots through the spoken language with pre-trained large-scale vision-language models. We apply Visual Question Answering and Image-to-Text Retrieval, which are tasks of Vision-Language Models. We show that with our method, it is possible to recognize not only whether a room door is open/closed, but also whether a transparent door is open/closed and whether water is running in a sink, without training neural networks or manual programming. In addition, the recognition accuracy can be improved by selecting appropriate texts from the set of prepared texts based on black-box optimization. For each state recognition, only the text set and its weighting need to be changed, eliminating the need to prepare multiple different models and programs, and facilitating the management of source code and computer resource. We experimentally demonstrate the effectiveness of our method and apply it to the recognition behavior on a mobile robot, Fetch. △ Less

Submitted 26 September, 2024; originally announced September 2024.

Comments: Accepted at Advanced Robotics, website - https://haraduka.github.io/vlm-bbo/

Dynamic Cloth Manipulation Considering Variable Stiffness and Material Change Using Deep Predictive Model with Parametric Bias

Authors: Kento Kawaharazuka, Akihiro Miki, Masahiro Bando, Kei Okada, Masayuki Inaba

Abstract: Dynamic manipulation of flexible objects such as fabric, which is difficult to modelize, is one of the major challenges in robotics. With the development of deep learning, we are beginning to see results in simulations and in some actual robots, but there are still many problems that have not yet been tackled. Humans can move their arms at high speed using their flexible bodies skillfully, and eve… ▽ More Dynamic manipulation of flexible objects such as fabric, which is difficult to modelize, is one of the major challenges in robotics. With the development of deep learning, we are beginning to see results in simulations and in some actual robots, but there are still many problems that have not yet been tackled. Humans can move their arms at high speed using their flexible bodies skillfully, and even when the material to be manipulated changes, they can manipulate the material after moving it several times and understanding its characteristics. Therefore, in this research, we focus on the following two points: (1) body control using a variable stiffness mechanism for more dynamic manipulation, and (2) response to changes in the material of the manipulated object using parametric bias. By incorporating these two approaches into a deep predictive model, we show through simulation and actual robot experiments that Musashi-W, a musculoskeletal humanoid with variable stiffness mechanism, can dynamically manipulate cloth while detecting changes in the physical properties of the manipulated object. △ Less

Submitted 23 September, 2024; originally announced September 2024.

Comments: Accepted at Frontiers in Neurorobotics

Robust Continuous Motion Strategy Against Muscle Rupture using Online Learning of Redundant Intersensory Networks for Musculoskeletal Humanoids

Authors: Kento Kawaharazuka, Manabu Nishiura, Yasunori Toshimitsu, Yusuke Omura, Yuya Koga, Yuki Asano, Koji Kawasaki, Masayuki Inaba

Abstract: Musculoskeletal humanoids have various biomimetic advantages, of which redundant muscle arrangement is one of the most important features. This feature enables variable stiffness control and allows the robot to keep moving its joints even if one of the redundant muscles breaks, but this has been rarely explored. In this study, we construct a neural network that represents the relationship among se… ▽ More Musculoskeletal humanoids have various biomimetic advantages, of which redundant muscle arrangement is one of the most important features. This feature enables variable stiffness control and allows the robot to keep moving its joints even if one of the redundant muscles breaks, but this has been rarely explored. In this study, we construct a neural network that represents the relationship among sensors in the flexible and difficult-to-modelize body of the musculoskeletal humanoid, and by learning this neural network, accurate motions can be achieved. In order to take advantage of the redundancy of muscles, we discuss the use of this network for muscle rupture detection, online update of the intersensory relationship considering the muscle rupture, and body control and state estimation using the muscle rupture information. This study explains a method of constructing a musculoskeletal humanoid that continues to move and perform tasks robustly even when one muscle breaks. △ Less

Submitted 23 September, 2024; originally announced September 2024.

Comments: Accepted at Robotics and Autonomous Systems

MEVIUS: A Quadruped Robot Easily Constructed through E-Commerce with Sheet Metal Welding and Machining

Authors: Kento Kawaharazuka, Shintaro Inoue, Temma Suzuki, Sota Yuzaki, Shogo Sawaguchi, Kei Okada, Masayuki Inaba

Abstract: Quadruped robots that individual researchers can build by themselves are crucial for expanding the scope of research due to their high scalability and customizability. These robots must be easily ordered and assembled through e-commerce or DIY methods, have a low number of components for easy maintenance, and possess durability to withstand experiments in diverse environments. Various quadruped ro… ▽ More Quadruped robots that individual researchers can build by themselves are crucial for expanding the scope of research due to their high scalability and customizability. These robots must be easily ordered and assembled through e-commerce or DIY methods, have a low number of components for easy maintenance, and possess durability to withstand experiments in diverse environments. Various quadruped robots have been developed so far, but most robots that can be built by research institutions are relatively small and made of plastic using 3D printers. These robots cannot withstand experiments in external environments such as mountain trails or rubble, and they will easily break with intense movements. Although there is the advantage of being able to print parts by yourself, the large number of components makes replacing broken parts and maintenance very cumbersome. Therefore, in this study, we develop a metal quadruped robot MEVIUS, that can be constructed and assembled using only materials ordered through e-commerce. We have considered the minimum set of components required for a quadruped robot, employing metal machining, sheet metal welding, and off-the-shelf components only. Also, we have achieved a simple circuit and software configuration. Considering the communication delay due to its simple configuration, we experimentally demonstrate that MEVIUS, utilizing reinforcement learning and Sim2Real, can traverse diverse rough terrains and withstand outside experiments. All hardware and software components can be obtained from https://github.com/haraduka/mevius. △ Less

Submitted 23 September, 2024; originally announced September 2024.

Comments: Accepted at Humanoids2024, website - https://haraduka.github.io/mevius-hardware/

Human-mimetic binaural ear design and sound source direction estimation for task realization of musculoskeletal humanoids

Authors: Yusuke Omura, Kento Kawaharazuka, Yuya Nagamatsu, Yuya Koga, Manabu Nishiura, Yasunori Toshimitsu, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: Human-like environment recognition by musculoskeletal humanoids is important for task realization in real complex environments and for use as dummies for test subjects. Humans integrate various sensory information to perceive their surroundings, and hearing is particularly useful for recognizing objects out of view or out of touch. In this research, we aim to realize human-like auditory environmen… ▽ More Human-like environment recognition by musculoskeletal humanoids is important for task realization in real complex environments and for use as dummies for test subjects. Humans integrate various sensory information to perceive their surroundings, and hearing is particularly useful for recognizing objects out of view or out of touch. In this research, we aim to realize human-like auditory environmental recognition and task realization for musculoskeletal humanoids by equipping them with a human-like auditory processing system. Humans realize sound-based environmental recognition by estimating directions of the sound sources and detecting environmental sounds based on changes in the time and frequency domain of incoming sounds and the integration of auditory information in the central nervous system. We propose a human mimetic auditory information processing system, which consists of three components: the human mimetic binaural ear unit, which mimics human ear structure and characteristics, the sound source direction estimation system, and the environmental sound detection system, which mimics processing in the central nervous system. We apply it to Musashi, a human mimetic musculoskeletal humanoid, and have it perform tasks that require sound information outside of view in real noisy environments to confirm the usefulness of the proposed methods. △ Less

Submitted 10 September, 2024; originally announced September 2024.

Comments: Accepted at ROBOMECH Journal

GeMuCo: Generalized Multisensory Correlational Model for Body Schema Learning

Authors: Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: Humans can autonomously learn the relationship between sensation and motion in their own bodies, estimate and control their own body states, and move while continuously adapting to the current environment. On the other hand, current robots control their bodies by learning the network structure described by humans from their experiences, making certain assumptions on the relationship between sensor… ▽ More Humans can autonomously learn the relationship between sensation and motion in their own bodies, estimate and control their own body states, and move while continuously adapting to the current environment. On the other hand, current robots control their bodies by learning the network structure described by humans from their experiences, making certain assumptions on the relationship between sensors and actuators. In addition, the network model does not adapt to changes in the robot's body, the tools that are grasped, or the environment, and there is no unified theory, not only for control but also for state estimation, anomaly detection, simulation, and so on. In this study, we propose a Generalized Multisensory Correlational Model (GeMuCo), in which the robot itself acquires a body schema describing the correlation between sensors and actuators from its own experience, including model structures such as network input/output. The robot adapts to the current environment by updating this body schema model online, estimates and controls its body state, and even performs anomaly detection and simulation. We demonstrate the effectiveness of this method by applying it to tool-use considering changes in grasping state for an axis-driven robot, to joint-muscle mapping learning for a musculoskeletal robot, and to full-body tool manipulation for a low-rigidity plastic-made humanoid. △ Less

Submitted 10 September, 2024; originally announced September 2024.

Comments: Accepted at IEEE Robotics and Automation Magazine

Antagonist Inhibition Control in Redundant Tendon-driven Structures Based on Human Reciprocal Innervation for Wide Range Limb Motion of Musculoskeletal Humanoids

Authors: Kento Kawaharazuka, Masaya Kawamura, Shogo Makino, Yuki Asano, Kei Okada, Masayuki Inaba

Abstract: The body structure of an anatomically correct tendon-driven musculoskeletal humanoid is complex, and the difference between its geometric model and the actual robot is very large because expressing the complex routes of tendon wires in a geometric model is very difficult. If we move a tendon-driven musculoskeletal humanoid by the tendon wire lengths of the geometric model, unintended muscle tensio… ▽ More The body structure of an anatomically correct tendon-driven musculoskeletal humanoid is complex, and the difference between its geometric model and the actual robot is very large because expressing the complex routes of tendon wires in a geometric model is very difficult. If we move a tendon-driven musculoskeletal humanoid by the tendon wire lengths of the geometric model, unintended muscle tension and slack will emerge. In some cases, this can lead to the wreckage of the actual robot. To solve this problem, we focused on reciprocal innervation in the human nervous system, and then implemented antagonist inhibition control (AIC) based on the reflex. This control makes it possible to avoid unnecessary internal muscle tension and slack of tendon wires caused by model error, and to perform wide range motion safely for a long time. To verify its effectiveness, we applied AIC to the upper limb of the tendon-driven musculoskeletal humanoid, Kengoro, and succeeded in dangling for 14 minutes and doing pull-ups. △ Less

Submitted 1 September, 2024; originally announced September 2024.

Comments: Accepted at IEEE Robotics and Automation Letters

Automatic Grouping of Redundant Sensors and Actuators Using Functional and Spatial Connections: Application to Muscle Grouping for Musculoskeletal Humanoids

Authors: Kento Kawaharazuka, Manabu Nishiura, Yuya Koga, Yusuke Omura, Yasunori Toshimitsu, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: For a robot with redundant sensors and actuators distributed throughout its body, it is difficult to construct a controller or a neural network using all of them due to computational cost and complexity. Therefore, it is effective to extract functionally related sensors and actuators, group them, and construct a controller or a network for each of these groups. In this study, the functional and sp… ▽ More For a robot with redundant sensors and actuators distributed throughout its body, it is difficult to construct a controller or a neural network using all of them due to computational cost and complexity. Therefore, it is effective to extract functionally related sensors and actuators, group them, and construct a controller or a network for each of these groups. In this study, the functional and spatial connections among sensors and actuators are embedded into a graph structure and a method for automatic grouping is developed. Taking a musculoskeletal humanoid with a large number of redundant muscles as an example, this method automatically divides all the muscles into regions such as the forearm, upper arm, scapula, neck, etc., which has been done by humans based on a geometric model. The functional relationship among the muscles and the spatial relationship of the neural connections are calculated without a geometric model. △ Less

Submitted 1 September, 2024; originally announced September 2024.

Comments: Accepted at IEEE Robotics and Automation Letters

Reflex-Based Open-Vocabulary Navigation without Prior Knowledge Using Omnidirectional Camera and Multiple Vision-Language Models

Authors: Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Naoto Tsukamoto, Kei Okada, Masayuki Inaba

Abstract: Various robot navigation methods have been developed, but they are mainly based on Simultaneous Localization and Mapping (SLAM), reinforcement learning, etc., which require prior map construction or learning. In this study, we consider the simplest method that does not require any map construction or learning, and execute open-vocabulary navigation of robots without any prior knowledge to do this.… ▽ More Various robot navigation methods have been developed, but they are mainly based on Simultaneous Localization and Mapping (SLAM), reinforcement learning, etc., which require prior map construction or learning. In this study, we consider the simplest method that does not require any map construction or learning, and execute open-vocabulary navigation of robots without any prior knowledge to do this. We applied an omnidirectional camera and pre-trained vision-language models to the robot. The omnidirectional camera provides a uniform view of the surroundings, thus eliminating the need for complicated exploratory behaviors including trajectory generation. By applying multiple pre-trained vision-language models to this omnidirectional image and incorporating reflective behaviors, we show that navigation becomes simple and does not require any prior setup. Interesting properties and limitations of our method are discussed based on experiments with the mobile robot Fetch. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: Accepted at Advanced Robotics, website - https://haraduka.github.io/omnidirectional-vlm/

Human Mimetic Forearm Design with Radioulnar Joint using Miniature Bone-Muscle Modules and Its Applications

Authors: Kento Kawaharazuka, Shogo Makino, Masaya Kawamura, Yuki Asano, Yohei Kakiuchi, Kei Okada, Masayuki Inaba

Abstract: The human forearm is composed of two long, thin bones called the radius and the ulna, and rotates using two axle joints. We aimed to develop a forearm based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body in order to bring out its benefits. For this, we need to miniaturize the muscle modules. To approach this task, we arranged two muscle motors ins… ▽ More The human forearm is composed of two long, thin bones called the radius and the ulna, and rotates using two axle joints. We aimed to develop a forearm based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body in order to bring out its benefits. For this, we need to miniaturize the muscle modules. To approach this task, we arranged two muscle motors inside one muscle module, and used the space effectively by utilizing common parts. In addition, we enabled the muscle module to also be used as the bone structure. Moreover, we used miniature motors and developed a way to dissipate the motor heat to the bone structure. Through these approaches, we succeeded in developing a forearm with a radioulnar joint based on the body proportion, weight ratio, muscle arrangement, and joint performance of the human body, while keeping maintainability and reliability. Also, we performed some motions such as soldering, opening a book, turning a screw, and badminton swinging using the benefits of the radioulnar structure, which have not been discussed before, and verified that Kengoro can realize skillful motions using the radioulnar joint like a human. △ Less

Submitted 19 August, 2024; originally announced August 2024.

Comments: Accepted at IROS2017

Behavioral Learning of Dish Rinsing and Scrubbing based on Interruptive Direct Teaching Considering Assistance Rate

Authors: Shumpei Wakabayashi, Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: Robots are expected to manipulate objects in a safe and dexterous way. For example, washing dishes is a dexterous operation that involves scrubbing the dishes with a sponge and rinsing them with water. It is necessary to learn it safely without splashing water and without dropping the dishes. In this study, we propose a safe and dexterous manipulation system. The robot learns a dynamics model of t… ▽ More Robots are expected to manipulate objects in a safe and dexterous way. For example, washing dishes is a dexterous operation that involves scrubbing the dishes with a sponge and rinsing them with water. It is necessary to learn it safely without splashing water and without dropping the dishes. In this study, we propose a safe and dexterous manipulation system. The robot learns a dynamics model of the object by estimating the state of the object and the robot itself, the control input, and the amount of human assistance required (assistance rate) after the human corrects the initial trajectory of the robot's hands by interruptive direct teaching. By backpropagating the error between the estimated and the reference value using the acquired dynamics model, the robot can generate a control input that approaches the reference value, for example, so that human assistance is not required and the dish does not move excessively. This allows for adaptive rinsing and scrubbing of dishes with unknown shapes and properties. As a result, it is possible to generate safe actions that require less human assistance. △ Less

Submitted 21 December, 2024; v1 submitted 18 August, 2024; originally announced August 2024.

Comments: Accepted at Advanced Robotics

Journal ref: Advanced Robotics 38(15) (2024) 1052-1065

Tool Shape Optimization through Backpropagation of Neural Network

Authors: Kento Kawaharazuka, Toru Ogawa, Cota Nabeshima

Abstract: When executing a certain task, human beings can choose or make an appropriate tool to achieve the task. This research especially addresses the optimization of tool shape for robotic tool-use. We propose a method in which a robot obtains an optimized tool shape, tool trajectory, or both, depending on a given task. The feature of our method is that a transition of the task state when the robot moves… ▽ More When executing a certain task, human beings can choose or make an appropriate tool to achieve the task. This research especially addresses the optimization of tool shape for robotic tool-use. We propose a method in which a robot obtains an optimized tool shape, tool trajectory, or both, depending on a given task. The feature of our method is that a transition of the task state when the robot moves a certain tool along a certain trajectory is represented by a deep neural network. We applied this method to object manipulation tasks on a 2D plane, and verified that appropriate tool shapes are generated by using this novel method. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Comments: Accepted at IROS2020

Dynamic Task Control Method of a Flexible Manipulator Using a Deep Recurrent Neural Network

Authors: Kento Kawaharazuka, Toru Ogawa, Cota Nabeshima

Abstract: The flexible body has advantages over the rigid body in terms of environmental contact thanks to its underactuation. On the other hand, when applying conventional control methods to realize dynamic tasks with the flexible body, there are two difficulties: accurate modeling of the flexible body and the derivation of intermediate postures to achieve the tasks. Learning-based methods are considered t… ▽ More The flexible body has advantages over the rigid body in terms of environmental contact thanks to its underactuation. On the other hand, when applying conventional control methods to realize dynamic tasks with the flexible body, there are two difficulties: accurate modeling of the flexible body and the derivation of intermediate postures to achieve the tasks. Learning-based methods are considered to be more effective than accurate modeling, but they require explicit intermediate postures. To solve these two difficulties at the same time, we developed a real-time task control method with a deep recurrent neural network named Dynamic Task Execution Network (DTXNET), which acquires the relationship among the control command, robot state including image information, and task state. Once the network is trained, only the target event and its timing are needed to realize a given task. To demonstrate the effectiveness of our method, we applied it to the task of Wadaiko (traditional Japanese drum) drumming as an example, and verified the best configuration of DTXNET. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Comments: Accepted at IROS2019

Imitation Learning with Additional Constraints on Motion Style using Parametric Bias

Authors: Kento Kawaharazuka, Yoichiro Kawamura, Kei Okada, Masayuki Inaba

Abstract: Imitation learning is one of the methods for reproducing human demonstration adaptively in robots. So far, it has been found that generalization ability of the imitation learning enables the robots to perform tasks adaptably in untrained environments. However, motion styles such as motion trajectory and the amount of force applied depend largely on the dataset of human demonstration, and settle do… ▽ More Imitation learning is one of the methods for reproducing human demonstration adaptively in robots. So far, it has been found that generalization ability of the imitation learning enables the robots to perform tasks adaptably in untrained environments. However, motion styles such as motion trajectory and the amount of force applied depend largely on the dataset of human demonstration, and settle down to an average motion style. In this study, we propose a method that adds parametric bias to the conventional imitation learning network and can add constraints to the motion style. By experiments using PR2 and the musculoskeletal humanoid MusashiLarm, we show that it is possible to perform tasks by changing its motion style as intended with constraints on joint velocity, muscle length velocity, and muscle tension. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: Accepted at IEEE Robotics and Automation Letters

Estimation and Control of Motor Core Temperature with Online Learning of Thermal Model Parameters: Application to Musculoskeletal Humanoids

Authors: Kento Kawaharazuka, Naoki Hiraoka, Kei Tsuzuki, Moritaka Onitsuka, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: The estimation and management of motor temperature are important for the continuous movements of robots. In this study, we propose an online learning method of thermal model parameters of motors for an accurate estimation of motor core temperature. Also, we propose a management method of motor core temperature using the updated model and anomaly detection method of motors. Finally, we apply this m… ▽ More The estimation and management of motor temperature are important for the continuous movements of robots. In this study, we propose an online learning method of thermal model parameters of motors for an accurate estimation of motor core temperature. Also, we propose a management method of motor core temperature using the updated model and anomaly detection method of motors. Finally, we apply this method to the muscles of the musculoskeletal humanoid and verify the ability of continuous movements. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: Accepted at IEEE Robotics and Automation Letters

Adaptive Robotic Tool-Tip Control Learning Considering Online Changes in Grasping State

Authors: Kento Kawaharazuka, Kei Okada, Masayuki Inaba

Abstract: Various robotic tool manipulation methods have been developed so far. However, to our knowledge, none of them have taken into account the fact that the grasping state such as grasping position and tool angle can change at any time during the tool manipulation. In addition, there are few studies that can handle deformable tools. In this study, we develop a method for estimating the position of a to… ▽ More Various robotic tool manipulation methods have been developed so far. However, to our knowledge, none of them have taken into account the fact that the grasping state such as grasping position and tool angle can change at any time during the tool manipulation. In addition, there are few studies that can handle deformable tools. In this study, we develop a method for estimating the position of a tool-tip, controlling the tool-tip, and handling online adaptation to changes in the relationship between the body and the tool, using a neural network including parametric bias. We demonstrate the effectiveness of our method for online change in grasping state and for deformable tools, in experiments using two different types of robots: axis-driven robot PR2 and tendon-driven robot MusashiLarm. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: Accepted at IEEE Robotics and Automation Letters

Object Recognition, Dynamic Contact Simulation, Detection, and Control of the Flexible Musculoskeletal Hand Using a Recurrent Neural Network with Parametric Bias

Authors: Kento Kawaharazuka, Kei Tsuzuki, Moritaka Onitsuka, Yuki Asano, Kei Okada, Koji Kawasaki, Masayuki Inaba

Abstract: The flexible musculoskeletal hand is difficult to modelize, and its model can change constantly due to deterioration over time, irreproducibility of initialization, etc. Also, for object recognition, contact detection, and contact control using the hand, it is desirable not to use a neural network trained for each task, but to use only one integrated network. Therefore, we develop a method to acqu… ▽ More The flexible musculoskeletal hand is difficult to modelize, and its model can change constantly due to deterioration over time, irreproducibility of initialization, etc. Also, for object recognition, contact detection, and contact control using the hand, it is desirable not to use a neural network trained for each task, but to use only one integrated network. Therefore, we develop a method to acquire a sensor state equation of the musculoskeletal hand using a recurrent neural network with parametric bias. By using this network, the hand can realize recognition of the grasped object, contact simulation, detection, and control, and can cope with deterioration over time, irreproducibility of initialization, etc. by updating parametric bias. We apply this study to the hand of the musculoskeletal humanoid Musashi and show its effectiveness. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: Accepted at IEEE Robotics and Automation Letters