Skip to main content

Showing 1–20 of 20 results for author: Au, K W S

.
  1. arXiv:2505.16249  [pdf, ps, other

    cs.RO cs.AI

    Manipulating Elasto-Plastic Objects With 3D Occupancy and Learning-Based Predictive Control

    Authors: Zhen Zhang, Xiangyu Chu, Yunxi Tang, Lulu Zhao, Jing Huang, Zhongliang Jiang, K. W. Samuel Au

    Abstract: Manipulating elasto-plastic objects remains a significant challenge due to severe self-occlusion, difficulties of representation, and complicated dynamics. This work proposes a novel framework for elasto-plastic object manipulation with a quasi-static assumption for motions, leveraging 3D occupancy to represent such objects, a learned dynamics model trained with 3D occupancy, and a learning-based… ▽ More

    Submitted 22 May, 2025; v1 submitted 22 May, 2025; originally announced May 2025.

    Comments: 8 Pages, 13 figures, accepted for publication in IEEE Robotics and Automation Letters (RA-L)

  2. arXiv:2501.11945  [pdf, other

    cs.RO

    Learning to Hop for a Single-Legged Robot with Parallel Mechanism

    Authors: Hongbo Zhang, Xiangyu Chu, Yanlin Chen, Yunxi Tang, Linzhu Yue, Yun-Hui Liu, Kwok Wai Samuel Au

    Abstract: This work presents the application of reinforcement learning to improve the performance of a highly dynamic hopping system with a parallel mechanism. Unlike serial mechanisms, parallel mechanisms can not be accurately simulated due to the complexity of their kinematic constraints and closed-loop structures. Besides, learning to hop suffers from prolonged aerial phase and the sparse nature of the r… ▽ More

    Submitted 21 January, 2025; originally announced January 2025.

  3. arXiv:2410.21758  [pdf, other

    cs.CV cs.RO

    DOFS: A Real-world 3D Deformable Object Dataset with Full Spatial Information for Dynamics Model Learning

    Authors: Zhen Zhang, Xiangyu Chu, Yunxi Tang, K. W. Samuel Au

    Abstract: This work proposes DOFS, a pilot dataset of 3D deformable objects (DOs) (e.g., elasto-plastic objects) with full spatial information (i.e., top, side, and bottom information) using a novel and low-cost data collection platform with a transparent operating plane. The dataset consists of active manipulation action, multi-view RGB-D images, well-registered point clouds, 3D deformed mesh, and 3D occup… ▽ More

    Submitted 29 October, 2024; originally announced October 2024.

    Comments: 5 pages, 6 figures, 2024 CoRL Workshop on Learning Robot Fine and Dexterous Manipulation: Perception and Control

  4. arXiv:2410.20374  [pdf, other

    cs.RO eess.SY

    A CT-guided Control Framework of a Robotic Flexible Endoscope for the Diagnosis of the Maxillary Sinusitis

    Authors: Puchen Zhu, Huayu Zhang, Xin Ma, Xiaoyin Zheng, Xuchen Wang, Kwok Wai Samuel Au

    Abstract: Flexible endoscopes are commonly adopted in narrow and confined anatomical cavities due to their higher reachability and dexterity. However, prolonged and unintuitive manipulation of these endoscopes leads to an increased workload on surgeons and risks of collision. To address these challenges, this paper proposes a CT-guided control framework for the diagnosis of maxillary sinusitis by using a ro… ▽ More

    Submitted 27 October, 2024; originally announced October 2024.

  5. arXiv:2407.18357  [pdf, other

    cs.RO

    Needle Segmentation Using GAN: Restoring Thin Instrument Visibility in Robotic Ultrasound

    Authors: Zhongliang Jiang, Xuesong Li, Xiangyu Chu, Angelos Karlas, Yuan Bi, Yingsheng Cheng, K. W. Samuel Au, Nassir Navab

    Abstract: Ultrasound-guided percutaneous needle insertion is a standard procedure employed in both biopsy and ablation in clinical practices. However, due to the complex interaction between tissue and instrument, the needle may deviate from the in-plane view, resulting in a lack of close monitoring of the percutaneous needle. To address this challenge, we introduce a robot-assisted ultrasound (US) imaging s… ▽ More

    Submitted 25 July, 2024; originally announced July 2024.

    Comments: accepted by IEEE TIM. code: https://github.com/noseefood/NeedleSegmentation-GAN; video: https://youtu.be/4WuEP9PACs0

  6. arXiv:2406.02885  [pdf, other

    cs.RO

    Homotopic Path Set Planning for Robot Manipulation and Navigation

    Authors: Jing Huang, Yunxi Tang, Kwok Wai Samuel Au

    Abstract: This paper addresses path set planning that yields important applications in robot manipulation and navigation such as path generation for deformable object keypoints and swarms. A path set refers to the collection of finite agent paths to represent the overall spatial path of a group of keypoints or a swarm, whose collective properties meet spatial and topological constraints. As opposed to plann… ▽ More

    Submitted 11 July, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

    Comments: 16 pages, 19 figures, conference

  7. arXiv:2405.17940  [pdf, other

    cs.RO cs.AI

    World Models for General Surgical Grasping

    Authors: Hongbin Lin, Bin Li, Chun Wai Wong, Juan Rojas, Xiangyu Chu, Kwok Wai Samuel Au

    Abstract: Intelligent vision control systems for surgical robots should adapt to unknown and diverse objects while being robust to system disturbances. Previous methods did not meet these requirements due to mainly relying on pose estimation and feature tracking. We propose a world-model-based deep reinforcement learning framework "Grasp Anything for Surgery" (GAS), that learns a pixel-level visuomotor poli… ▽ More

    Submitted 28 May, 2024; originally announced May 2024.

    Journal ref: Robotics: Science and Systems 2024

  8. arXiv:2404.19242  [pdf, other

    cs.CV eess.IV stat.ME

    A Minimal Set of Parameters Based Depth-Dependent Distortion Model and Its Calibration Method for Stereo Vision Systems

    Authors: Xin Ma, Puchen Zhu, Xiao Li, Xiaoyin Zheng, Jianshu Zhou, Xuchen Wang, Kwok Wai Samuel Au

    Abstract: Depth position highly affects lens distortion, especially in close-range photography, which limits the measurement accuracy of existing stereo vision systems. Moreover, traditional depth-dependent distortion models and their calibration methods have remained complicated. In this work, we propose a minimal set of parameters based depth-dependent distortion model (MDM), which considers the radial an… ▽ More

    Submitted 1 May, 2024; v1 submitted 29 April, 2024; originally announced April 2024.

    Comments: This paper has been accepted for publication in IEEE Transactions on Instrumentation and Measurement

  9. VibNet: Vibration-Boosted Needle Detection in Ultrasound Images

    Authors: Dianye Huang, Chenyang Li, Angelos Karlas, Xiangyu Chu, K. W. Samuel Au, Nassir Navab, Zhongliang Jiang

    Abstract: Precise percutaneous needle detection is crucial for ultrasound (US)-guided interventions. However, inherent limitations such as speckles, needle-like artifacts, and low resolution make it challenging to robustly detect needles, especially when their visibility is reduced or imperceptible. To address this challenge, we propose VibNet, a learning-based framework designed to enhance the robustness a… ▽ More

    Submitted 22 February, 2025; v1 submitted 21 March, 2024; originally announced March 2024.

    Comments: Accepted by IEEE TMI

  10. Deformable Object Manipulation With Constraints Using Path Set Planning and Tracking

    Authors: Jing Huang, Xiangyu Chu, Xin Ma, Kwok Wai Samuel Au

    Abstract: In robotic deformable object manipulation (DOM) applications, constraints arise commonly from environments and task-specific requirements. Enabling DOM with constraints is therefore crucial for its deployment in practice. However, dealing with constraints turns out to be challenging due to many inherent factors such as inaccessible deformation models of deformable objects (DOs) and varying environ… ▽ More

    Submitted 17 February, 2024; originally announced February 2024.

    Comments: 20 pages, 25 figures, journal

    Journal ref: IEEE Transactions on Robotics, 2023

  11. arXiv:2311.06543  [pdf, other

    cs.RO

    Bootstrapping Robotic Skill Learning With Intuitive Teleoperation: Initial Feasibility Study

    Authors: Xiangyu Chu, Yunxi Tang, Lam Him Kwok, Yuanpei Cai, Kwok Wai Samuel Au

    Abstract: Robotic skill learning has been increasingly studied but the demonstration collections are more challenging compared to collecting images/videos in computer vision and texts in natural language processing. This paper presents a skill learning paradigm by using intuitive teleoperation devices to generate high-quality human demonstrations efficiently for robotic skill learning in a data-driven manne… ▽ More

    Submitted 11 November, 2023; originally announced November 2023.

    Comments: 10 pages, 4 figures, accepted by ISER2023

  12. arXiv:2310.08873  [pdf, other

    cs.RO cs.AI

    Interactive Navigation in Environments with Traversable Obstacles Using Large Language and Vision-Language Models

    Authors: Zhen Zhang, Anran Lin, Chun Wai Wong, Xiangyu Chu, Qi Dou, K. W. Samuel Au

    Abstract: This paper proposes an interactive navigation framework by using large language and vision-language models, allowing robots to navigate in environments with traversable obstacles. We utilize the large language model (GPT-3.5) and the open-set Vision-language Model (Grounding DINO) to create an action-aware costmap to perform effective path planning without fine-tuning. With the large models, we ca… ▽ More

    Submitted 12 March, 2024; v1 submitted 13 October, 2023; originally announced October 2023.

    Comments: Accepted by 2024 IEEE International Conference on Robotics and Automation (ICRA), 7 pages, 8 figures

  13. arXiv:2308.10401  [pdf, other

    cs.RO eess.SY

    Model-Free Large-Scale Cloth Spreading With Mobile Manipulation: Initial Feasibility Study

    Authors: Xiangyu Chu+, Shengzhi Wang+, Minjian Feng, Jiaxi Zheng, Yuxuan Zhao, Jing Huang, K. W. Samuel Au

    Abstract: Cloth manipulation is common in domestic and service tasks, and most studies use fixed-base manipulators to manipulate objects whose sizes are relatively small with respect to the manipulators' workspace, such as towels, shirts, and rags. In contrast, manipulation of large-scale cloth, such as bed making and tablecloth spreading, poses additional challenges of reachability and manipulation control… ▽ More

    Submitted 20 August, 2023; originally announced August 2023.

    Comments: 6 pages, 6 figures, submit to CASE2023

    Journal ref: 2023 IEEE International Conference on Automation Science and Engineering (CASE)

  14. arXiv:2303.03675  [pdf, other

    cs.RO

    End-to-End Learning of Deep Visuomotor Policy for Needle Picking

    Authors: Hongbin Lin, Bin Li, Xiangyu Chu, Qi Dou, Yunhui Liu, Kwok Wai Samuel Au

    Abstract: Needle picking is a challenging manipulation task in robot-assisted surgery due to the characteristics of small slender shapes of needles, needles' variations in shapes and sizes, and demands for millimeter-level control. Prior works, heavily relying on the prior of needles (e.g., geometric models), are hard to scale to unseen needles' variations. In this paper, we present the first end-to-end lea… ▽ More

    Submitted 26 July, 2023; v1 submitted 7 March, 2023; originally announced March 2023.

    Comments: Accepted by IROS 2023

  15. arXiv:2210.10238  [pdf, other

    cs.RO eess.SY

    Towards Exact Interaction Force Control for Underactuated Quadrupedal Systems with Orthogonal Projection and Quadratic Programming

    Authors: Shengzhi Wang, Xiangyu Chu, K. W. Samuel Au

    Abstract: Projected Inverse Dynamics Control (PIDC) is commonly used in robots subject to contact, especially in quadrupedal systems. Many methods based on such dynamics have been developed for quadrupedal locomotion tasks, and only a few works studied simple interactions between the robot and environment, such as pressing an E-stop button. To facilitate the interaction requiring exact force control for saf… ▽ More

    Submitted 18 October, 2022; originally announced October 2022.

    Comments: 7 pages, 4 figures, submitted to 2023 IEEE International Conference on Robotics and Automation (ICRA)

  16. arXiv:2210.01406  [pdf, other

    cs.RO

    Open-source High-precision Autonomous Suturing Framework With Visual Guidance

    Authors: Hongbin Lin, Bin Li, Yunhui Liu, Kwok Wai Samuel Au

    Abstract: Autonomous surgery has attracted increasing attention for revolutionizing robotic patient care, yet remains a distant and challenging goal. In this paper, we propose an image-based framework for high-precision autonomous suturing operation. We first build an algebraic geometric algorithm to achieve accurate needle pose estimation, then design the corresponding keypoint-based calibration network fo… ▽ More

    Submitted 28 May, 2024; v1 submitted 4 October, 2022; originally announced October 2022.

    Journal ref: IROS 2022 Workshop on "A Panacea Or An Alchemy? Benefits And Risks Of Robot Learning In Medical Applications"

  17. Learning Deep Nets for Gravitational Dynamics with Unknown Disturbance through Physical Knowledge Distillation: Initial Feasibility Study

    Authors: Hongbin Lin, Qian Gao, Xiangyu Chu, Qi Dou, Anton Deguet, Peter Kazanzides, K. W. Samuel Au

    Abstract: Learning high-performance deep neural networks for dynamic modeling of high Degree-Of-Freedom (DOF) robots remains challenging due to the sampling complexity. Typical unknown system disturbance caused by unmodeled dynamics (such as internal compliance, cables) further exacerbates the problem. In this paper, a novel framework characterized by both high data efficiency and disturbance-adapting capab… ▽ More

    Submitted 4 October, 2022; originally announced October 2022.

    Journal ref: IEEE ROBOTICS AND AUTOMATION LETTERS, VOL. 6, NO. 2, APRIL 2021

  18. Towards Safe Landing of Falling Quadruped Robots Using a 3-DoF Morphable Inertial Tail

    Authors: Yunxi Tang, Jiajun An, Xiangyu Chu, Shengzhi Wang, Ching Yan Wong, K. W. Samuel Au

    Abstract: Falling cat problem is well-known where cats show their super aerial reorientation capability and can land safely. For their robotic counterparts, a similar falling quadruped robot problem, has not been fully addressed, although achieving safe landing as the cats has been increasingly investigated. Unlike imposing the burden on landing control, we approach to safe landing of falling quadruped robo… ▽ More

    Submitted 30 September, 2022; originally announced September 2022.

    Comments: 7 pages, 8 figures, submit to ICRA2023

    Journal ref: 2023 IEEE International Conference on Robotics and Automation (ICRA)

  19. arXiv:2109.07131  [pdf, other

    cs.RO

    Inequality Constrained Trajectory Optimization with A Hybrid Multiple-shooting iLQR

    Authors: Yunxi Tang, Xiangyu Chu, Wanxin Jin, K. W. Samuel Au

    Abstract: Trajectory optimization has been used extensively in robotic systems. In particular, iterative Linear Quadratic Regulator (iLQR) has performed well as an off-line planner and online nonlinear model predictive control solver, with a lower computational cost. However, standard iLQR cannot handle any constraints or perform reasonable initialization of a state trajectory. In this paper, we propose a h… ▽ More

    Submitted 18 March, 2023; v1 submitted 15 September, 2021; originally announced September 2021.

    Comments: 8 pages, 5 figures

  20. A Reliable Gravity Compensation Control Strategy for dVRK Robotic Arms With Nonlinear Disturbance Forces

    Authors: Hongbin Lin, C. W. Vincent Hui, Yan Wang, Anton Deguet, Peter Kazanzides, K. W. Samuel Au

    Abstract: External disturbance forces caused by nonlinear springy electrical cables in the Master Tool Manipulator (MTM) of the da Vinci Research Kit (dVRK) limits the usage of the existing gravity compensation methods. Significant motion drifts at the MTM tip are often observed when the MTM is located far from its identification trajectory, preventing the usage of these methods for the entire workspace rel… ▽ More

    Submitted 16 January, 2020; originally announced January 2020.

    Journal ref: IEEE Robotics and Automation Letters 4.4 (2019): 3892-3899