-
Optical Waveguide-based Spider Web Enables Resilient Impact Detection and Localization
Authors:
Dylan Wilson,
Marco Pontin,
Peter Walters,
Perla Maiolino
Abstract:
Spiders use their webs as multifunctional tools that enable capturing and localizing prey and more general environmental sensing through vibrations. Inspired by their biological function, we present a spider web-inspired optical waveguide system for resilient impulse detection and localization. The structure consists of six clear thermoplastic polyurethane (TPU) waveguides arranged radially and in…
▽ More
Spiders use their webs as multifunctional tools that enable capturing and localizing prey and more general environmental sensing through vibrations. Inspired by their biological function, we present a spider web-inspired optical waveguide system for resilient impulse detection and localization. The structure consists of six clear thermoplastic polyurethane (TPU) waveguides arranged radially and interconnected by a spiral TPU thread, mimicking orb spider webs. Light transmission losses, induced by vibrations, are measured via coupled LEDs and photo-diodes, allowing real-time detection. We systematically characterize individual waveguides, analyzing key parameters such as tension, impulse position, and break angle to optimize vibrational response. The complete system is validated through controlled experiments, revealing a 5 ms propagation delay in vibration transfer between adjacent radii, enhancing localization capabilities. We demonstrate a robust impulse detection and localization algorithm leveraging time delay analysis, achieving reliable event identification even in cases of sensor failure. This study highlights the potential of bioinspired optical waveguide structures for adaptive sensing, with applications in soft robotics, structural monitoring, and environmental sensing.
△ Less
Submitted 20 June, 2025;
originally announced June 2025.
-
Diffusion-based Inverse Observation Model for Artificial Skin
Authors:
Ante Maric,
Julius Jankowski,
Giammarco Caroleo,
Alessandro Albini,
Perla Maiolino,
Sylvain Calinon
Abstract:
Contact-based estimation of object pose is challenging due to discontinuities and ambiguous observations that can correspond to multiple possible system states. This multimodality makes it difficult to efficiently sample valid hypotheses while respecting contact constraints. Diffusion models can learn to generate samples from such multimodal probability distributions through denoising algorithms.…
▽ More
Contact-based estimation of object pose is challenging due to discontinuities and ambiguous observations that can correspond to multiple possible system states. This multimodality makes it difficult to efficiently sample valid hypotheses while respecting contact constraints. Diffusion models can learn to generate samples from such multimodal probability distributions through denoising algorithms. We leverage these probabilistic modeling capabilities to learn an inverse observation model conditioned on tactile measurements acquired from a distributed artificial skin. We present simulated experiments demonstrating efficient sampling of contact hypotheses for object pose estimation through touch.
△ Less
Submitted 16 June, 2025;
originally announced June 2025.
-
Estimating Scene Flow in Robot Surroundings with Distributed Miniaturized Time-of-Flight Sensors
Authors:
Jack Sander,
Giammarco Caroleo,
Alessandro Albini,
Perla Maiolino
Abstract:
Tracking motions of humans or objects in the surroundings of the robot is essential to improve safe robot motions and reactions. In this work, we present an approach for scene flow estimation from low-density and noisy point clouds acquired from miniaturized Time of Flight (ToF) sensors distributed on the robot body. The proposed method clusters points from consecutive frames and applies Iterative…
▽ More
Tracking motions of humans or objects in the surroundings of the robot is essential to improve safe robot motions and reactions. In this work, we present an approach for scene flow estimation from low-density and noisy point clouds acquired from miniaturized Time of Flight (ToF) sensors distributed on the robot body. The proposed method clusters points from consecutive frames and applies Iterative Closest Point (ICP) to estimate a dense motion flow, with additional steps introduced to mitigate the impact of sensor noise and low-density data points. Specifically, we employ a fitness-based classification to distinguish between stationary and moving points and an inlier removal strategy to refine geometric correspondences. The proposed approach is validated in an experimental setup where 24 ToF are used to estimate the velocity of an object moving at different controlled speeds. Experimental results show that the method consistently approximates the direction of the motion and its magnitude with an error which is in line with sensor noise.
△ Less
Submitted 3 April, 2025;
originally announced April 2025.
-
Tiny Lidars for Manipulator Self-Awareness: Sensor Characterization and Initial Localization Experiments
Authors:
Giammarco Caroleo,
Alessandro Albini,
Daniele De Martini,
Timothy D. Barfoot,
Perla Maiolino
Abstract:
For several tasks, ranging from manipulation to inspection, it is beneficial for robots to localize a target object in their surroundings. In this paper, we propose an approach that utilizes coarse point clouds obtained from miniaturized VL53L5CX Time-of-Flight (ToF) sensors (tiny lidars) to localize a target object in the robot's workspace. We first conduct an experimental campaign to calibrate t…
▽ More
For several tasks, ranging from manipulation to inspection, it is beneficial for robots to localize a target object in their surroundings. In this paper, we propose an approach that utilizes coarse point clouds obtained from miniaturized VL53L5CX Time-of-Flight (ToF) sensors (tiny lidars) to localize a target object in the robot's workspace. We first conduct an experimental campaign to calibrate the dependency of sensor readings on relative range and orientation to targets. We then propose a probabilistic sensor model that is validated in an object pose estimation task using a Particle Filter (PF). The results show that the proposed sensor model improves the performance of the localization of the target object with respect to two baselines: one that assumes measurements are free from uncertainty and one in which the confidence is provided by the sensor datasheet.
△ Less
Submitted 5 March, 2025;
originally announced March 2025.
-
Compliant Beaded-String Jamming For Variable Stiffness Anthropomorphic Fingers
Authors:
Maximilian Westermann,
Marco Pontin,
Leone Costi,
Alessandro Albini,
Perla Maiolino
Abstract:
Achieving human-like dexterity in robotic grippers remains an open challenge, particularly in ensuring robust manipulation in uncertain environments. Soft robotic hands try to address this by leveraging passive compliance, a characteristic that is crucial to the adaptability of the human hand, to achieve more robust manipulation while reducing reliance on high-resolution sensing and complex contro…
▽ More
Achieving human-like dexterity in robotic grippers remains an open challenge, particularly in ensuring robust manipulation in uncertain environments. Soft robotic hands try to address this by leveraging passive compliance, a characteristic that is crucial to the adaptability of the human hand, to achieve more robust manipulation while reducing reliance on high-resolution sensing and complex control. Further improvements in terms of precision and postural stability in manipulation tasks are achieved through the integration of variable stiffness mechanisms, but these tend to lack residual compliance, be bulky and have slow response times. To address these limitations, this work introduces a Compliant Joint Jamming mechanism for anthropomorphic fingers that exhibits passive residual compliance and adjustable stiffness, while achieving a range of motion in line with that of human interphalangeal joints. The stiffness range provided by the mechanism is controllable from 0.48 Nm/rad to 1.95 Nm/rad (a 4x increase). Repeatability, hysteresis and stiffness were also characterized as a function of the jamming force. To demonstrate the importance of the passive residual compliance afforded by the proposed system, a peg-in-hole task was conducted, which showed a 60% higher success rate for a gripper integrating our joint design when compared to a rigid one.
△ Less
Submitted 6 February, 2025;
originally announced February 2025.
-
Soft and Highly-Integrated Optical Fiber Bending Sensors for Proprioception in Multi-Material 3D Printed Fingers
Authors:
Ellis Capp,
Marco Pontin,
Peter Walters,
Perla Maiolino
Abstract:
Accurate shape sensing, only achievable through distributed proprioception, is a key requirement for closed-loop control of soft robots. Low-cost power efficient optoelectronic sensors manufactured from flexible materials represent a natural choice as they can cope with the large deformations of soft robots without loss of performance. However, existing integration approaches are cumbersome and re…
▽ More
Accurate shape sensing, only achievable through distributed proprioception, is a key requirement for closed-loop control of soft robots. Low-cost power efficient optoelectronic sensors manufactured from flexible materials represent a natural choice as they can cope with the large deformations of soft robots without loss of performance. However, existing integration approaches are cumbersome and require manual steps and complex assembly. We propose a semi-automated printing process where plastic optical fibers are embedded with readout electronics in 3D printed flexures. The fibers become locked in place and the readout electronics remain optically coupled to them while the flexures undergo large bending deformations, creating a repeatable, monolithically manufactured bending transducer with only 10 minutes required in total for the manual embedding steps. We demonstrate the process by manufacturing multi-material 3D printed fingers and extensively evaluating the performance of each proprioceptive joint. The sensors achieve 70% linearity and 4.81° RMS error on average. Furthermore, the distributed architecture allows for maintaining an average fingertip position estimation accuracy of 12 mm in the presence of external static forces. To demonstrate the potential of the distributed sensor architecture in robotics applications, we build a data-driven model independent of actuation feedback to detect contact with objects in the environment.
△ Less
Submitted 6 February, 2025;
originally announced February 2025.
-
JAMMit! Monolithic 3D-Printing of a Bead Jamming Soft Pneumatic Arm
Authors:
Yao Yao,
Maximilian Westermann,
Marco Pontin,
Alessandro Albini,
Perla Maiolino
Abstract:
3D-printed bellow soft pneumatic arms are widely adopted for their flexible design, ease of fabrication, and large deformation capabilities. However, their low stiffness limits their real-world applications. Although several methods exist to enhance the stiffness of soft actuators, many involve complex manufacturing processes not in line with modern goals of monolithic and automated additive manuf…
▽ More
3D-printed bellow soft pneumatic arms are widely adopted for their flexible design, ease of fabrication, and large deformation capabilities. However, their low stiffness limits their real-world applications. Although several methods exist to enhance the stiffness of soft actuators, many involve complex manufacturing processes not in line with modern goals of monolithic and automated additive manufacturing. With its simplicity, bead-jamming represents a simple and effective solution to these challenges. This work introduces a method for monolithic printing of a bellow soft pneumatic arm, integrating a tendon-driven central spine of bowl-shaped beads. We experimentally characterized the arm's range of motion in both unjammed and jammed states, as well as its stiffness under various actuation and jamming conditions. As a result, we provide an optimal jamming policy as a trade-off between preserving the range of motion and maximizing stiffness. The proposed design was further demonstrated in a switch-toggling task, showing its potential for practical applications.
△ Less
Submitted 5 February, 2025;
originally announced February 2025.
-
Soft Robot Localization Using Distributed Miniaturized Time-of-Flight Sensors
Authors:
Giammarco Caroleo,
Alessandro Albini,
Perla Maiolino
Abstract:
Thanks to their compliance and adaptability, soft robots can be deployed to perform tasks in constrained or complex environments. In these scenarios, spatial awareness of the surroundings and the ability to localize the robot within the environment represent key aspects. While state-of-the-art localization techniques are well-explored in autonomous vehicles and walking robots, they rely on data re…
▽ More
Thanks to their compliance and adaptability, soft robots can be deployed to perform tasks in constrained or complex environments. In these scenarios, spatial awareness of the surroundings and the ability to localize the robot within the environment represent key aspects. While state-of-the-art localization techniques are well-explored in autonomous vehicles and walking robots, they rely on data retrieved with lidar or depth sensors which are bulky and thus difficult to integrate into small soft robots. Recent developments in miniaturized Time of Flight (ToF) sensors show promise as a small and lightweight alternative to bulky sensors. These sensors can be potentially distributed on the soft robot body, providing multi-point depth data of the surroundings. However, the small spatial resolution and the noisy measurements pose a challenge to the success of state-of-the-art localization algorithms, which are generally applied to much denser and more reliable measurements. In this paper, we enforce distributed VL53L5CX ToF sensors, mount them on the tip of a soft robot, and investigate their usage for self-localization tasks. Experimental results show that the soft robot can effectively be localized with respect to a known map, with an error comparable to the uncertainty on the measures provided by the miniaturized ToF sensors.
△ Less
Submitted 3 February, 2025;
originally announced February 2025.
-
Touch-to-Touch Translation -- Learning the Mapping Between Heterogeneous Tactile Sensing Technologies
Authors:
Francesco Grella,
Alessandro Albini,
Giorgio Cannata,
Perla Maiolino
Abstract:
The use of data-driven techniques for tactile data processing and classification has recently increased. However, collecting tactile data is a time-expensive and sensor-specific procedure. Indeed, due to the lack of hardware standards in tactile sensing, data is required to be collected for each different sensor. This paper considers the problem of learning the mapping between two tactile sensor o…
▽ More
The use of data-driven techniques for tactile data processing and classification has recently increased. However, collecting tactile data is a time-expensive and sensor-specific procedure. Indeed, due to the lack of hardware standards in tactile sensing, data is required to be collected for each different sensor. This paper considers the problem of learning the mapping between two tactile sensor outputs with respect to the same physical stimulus -- we refer to this problem as touch-to-touch translation. In this respect, we proposed two data-driven approaches to address this task and we compared their performance. The first one exploits a generative model developed for image-to-image translation and adapted for this context. The second one uses a ResNet model trained to perform a regression task. We validated both methods using two completely different tactile sensors -- a camera-based, Digit and a capacitance-based, CySkin. In particular, we used Digit images to generate the corresponding CySkin data. We trained the models on a set of tactile features that can be found in common larger objects and we performed the testing on a previously unseen set of data. Experimental results show the possibility of translating Digit images into the CySkin output by preserving the contact shape and with an error of 15.18% in the magnitude of the sensor responses.
△ Less
Submitted 4 November, 2024;
originally announced November 2024.
-
Reinforcement Learning Controllers for Soft Robots using Learned Environments
Authors:
Uljad Berdica,
Matthew Jackson,
Niccolò Enrico Veronese,
Jakob Foerster,
Perla Maiolino
Abstract:
Soft robotic manipulators offer operational advantage due to their compliant and deformable structures. However, their inherently nonlinear dynamics presents substantial challenges. Traditional analytical methods often depend on simplifying assumptions, while learning-based techniques can be computationally demanding and limit the control policies to existing data. This paper introduces a novel ap…
▽ More
Soft robotic manipulators offer operational advantage due to their compliant and deformable structures. However, their inherently nonlinear dynamics presents substantial challenges. Traditional analytical methods often depend on simplifying assumptions, while learning-based techniques can be computationally demanding and limit the control policies to existing data. This paper introduces a novel approach to soft robotic control, leveraging state-of-the-art policy gradient methods within parallelizable synthetic environments learned from data. We also propose a safety oriented actuation space exploration protocol via cascaded updates and weighted randomness. Specifically, our recurrent forward dynamics model is learned by generating a training dataset from a physically safe \textit{mean reverting} random walk in actuation space to explore the partially-observed state-space. We demonstrate a reinforcement learning approach towards closed-loop control through state-of-the-art actor-critic methods, which efficiently learn high-performance behaviour over long horizons. This approach removes the need for any knowledge regarding the robot's operation or capabilities and sets the stage for a comprehensive benchmarking tool in soft robotics control.
△ Less
Submitted 25 October, 2024; v1 submitted 24 October, 2024;
originally announced October 2024.
-
SPADA: A Toolbox of Designing Soft Pneumatic Actuators for Shape Matching based on Surrogate Modeling
Authors:
Yao Yao,
Liang He,
Perla Maiolino
Abstract:
Soft pneumatic actuators (SPAs) produce motions for soft robots with simple pressure input, however they require to be appropriately designed to fit the target application. Available design methods employ kinematic models and optimization to estimate the actuator response and the optimal design parameters, to achieve a target actuator's shape. Within SPAs, Bellow-SPAs excel in rapid prototyping an…
▽ More
Soft pneumatic actuators (SPAs) produce motions for soft robots with simple pressure input, however they require to be appropriately designed to fit the target application. Available design methods employ kinematic models and optimization to estimate the actuator response and the optimal design parameters, to achieve a target actuator's shape. Within SPAs, Bellow-SPAs excel in rapid prototyping and large deformation, yet their kinematic models often lack accuracy due to the geometry complexity and the material nonlinearity. Furthermore, existing shape-matching algorithms are not providing an end-to-end solution from the desired shape to the actuator. In addition, despite the availability of computational design pipelines, an accessible and user-friendly toolbox for direct application remains elusive. This paper addresses these challenges, offering an end-to-end shape-matching design framework for bellow-SPAs to streamline the design process, and the open-source toolbox SPADA (Soft Pneumatic Actuator Design frAmework) implementing the framework with a GUI for easy access. It provides a kinematic model grounded on a modular design to improve accuracy, Finite Element Method (FEM) simulations, and piecewise constant curvature (PCC) approximation. An Artificial Neural Network-trained surrogate model, based on FEM simulation data, is trained for fast computation in optimization. A shape-matching algorithm, merging 3D PCC segmentation and a surrogate model-based genetic algorithm, identifies optimal actuator design parameters for desired shapes. The toolbox, implementing the proposed design framework, has proven its end-to-end capability in designing actuators to precisely match 2D shapes with root-mean-square errors of 4.16, 2.70, and 2.51mm, and demonstrating its potential by designing a 3D deformable actuator.
△ Less
Submitted 21 August, 2023; v1 submitted 30 May, 2023;
originally announced May 2023.
-
Design and Characterization of a 3D-printed Pneumatically-driven Bistable Valve with Tunable Characteristics
Authors:
Sihan Wang,
Liang He,
Perla Maiolino
Abstract:
Although research studies in pneumatic soft robots develop rapidly, most pneumatic actuators are still controlled by rigid valves and conventional electronics. The existence of these rigid, electronic components sacrifices the compliance and adaptability of soft robots.} Current electronics-free valve designs based on soft materials are facing challenges in behaviour consistency, design flexibilit…
▽ More
Although research studies in pneumatic soft robots develop rapidly, most pneumatic actuators are still controlled by rigid valves and conventional electronics. The existence of these rigid, electronic components sacrifices the compliance and adaptability of soft robots.} Current electronics-free valve designs based on soft materials are facing challenges in behaviour consistency, design flexibility, and fabrication complexity. Taking advantages of soft material 3D printing, this paper presents a new design of a bi-stable pneumatic valve, which utilises two soft, pneumatically-driven, and symmetrically-oriented conical shells with structural bistability to stabilise and regulate the airflow. The critical pressure required to operate the valve can be adjusted by changing the design features of the soft bi-stable structure. Multi-material printing simplifies the valve fabrication, enhances the flexibility in design feature optimisations, and improves the system repeatability. In this work, both a theoretical model and physical experiments are introduced to examine the relationships between the critical operating pressure and the key design features. Results with valve characteristic tuning via material stiffness changing show better effectiveness compared to the change of geometry design features (demonstrated largest tunable critical pressure range from 15.3 to 65.2 kPa and fastest response time $\leq$ 1.8 s.
△ Less
Submitted 4 October, 2021;
originally announced October 2021.
-
Contact modelling and tactile data processing for robot skin
Authors:
Wojciech Wasko,
Alessandro Albini,
Perla Maiolino,
Fulvio Mastrogiovanni,
Giorgio Cannata
Abstract:
Tactile sensing is a key enabling technology to develop complex behaviours for robots interacting with humans or the environment. This paper discusses computational aspects playing a significant role when extracting information about contact events. Considering a large-scale, capacitance-based robot skin technology we developed in the past few years, we analyse the classical Boussinesq-Cerruti's s…
▽ More
Tactile sensing is a key enabling technology to develop complex behaviours for robots interacting with humans or the environment. This paper discusses computational aspects playing a significant role when extracting information about contact events. Considering a large-scale, capacitance-based robot skin technology we developed in the past few years, we analyse the classical Boussinesq-Cerruti's solution and the Love's approach for solving a distributed inverse contact problem, both from a qualitative and a computational perspective. Our contribution is the characterisation of algorithms performance using a freely available dataset and data originating from surfaces provided with robot skin.
△ Less
Submitted 21 September, 2018;
originally announced September 2018.
-
A Flexible and Robust Large Scale Capacitive Tactile System for Robots
Authors:
Perla Maiolino,
Marco Maggiali,
Giorgio Cannata,
Giorgio Metta,
Lorenzo Natale
Abstract:
Capacitive technology allows building sensors that are small, compact and have high sensitivity. For this reason it has been widely adopted in robotics. In a previous work we presented a compliant skin system based on capacitive technology consisting of triangular modules interconnected to form a system of sensors that can be deployed on non-flat surfaces. This solution has been successfully adopt…
▽ More
Capacitive technology allows building sensors that are small, compact and have high sensitivity. For this reason it has been widely adopted in robotics. In a previous work we presented a compliant skin system based on capacitive technology consisting of triangular modules interconnected to form a system of sensors that can be deployed on non-flat surfaces. This solution has been successfully adopted to cover various humanoid robots. The main limitation of this and all the approaches based on capacitive technology is that they require to embed a deformable dielectric layer (usually made using an elastomer) covered by a conductive layer. This complicates the production process considerably, introduces hysteresis and limits the durability of the sensors due to ageing and mechanical stress.
In this paper we describe a novel solution in which the dielectric is made using a thin layer of 3D fabric which is glued to conductive and protective layers using techniques adopted in the clothing industry. As such, the sensor is easier to produce and has better mechanical properties. Furthermore, the sensor proposed in this paper embeds transducers for thermal compensation of the pressure measurements. We report experimental analysis that demonstrates that the sensor has good properties in terms of sensitivity and resolution. Remarkably we show that the sensor has very low hysteresis and effectively allows compensating drifts due to temperature variations.
△ Less
Submitted 25 November, 2014;
originally announced November 2014.
