-
RASMD: RGB And SWIR Multispectral Driving Dataset for Robust Perception in Adverse Conditions
Authors:
Youngwan Jin,
Michal Kovac,
Yagiz Nalcakan,
Hyeongjin Ju,
Hanbin Song,
Sanghyeop Yeo,
Shiho Kim
Abstract:
Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions like fog, rain, snow, glare, and high contrast. Although other spectral bands like near-infrared (NIR) and long-wave infrared (LWIR) can enhance vision perception in such situations, they have limitations and lack large-scale datasets and benchmarks. Short-wav…
▽ More
Current autonomous driving algorithms heavily rely on the visible spectrum, which is prone to performance degradation in adverse conditions like fog, rain, snow, glare, and high contrast. Although other spectral bands like near-infrared (NIR) and long-wave infrared (LWIR) can enhance vision perception in such situations, they have limitations and lack large-scale datasets and benchmarks. Short-wave infrared (SWIR) imaging offers several advantages over NIR and LWIR. However, no publicly available large-scale datasets currently incorporate SWIR data for autonomous driving. To address this gap, we introduce the RGB and SWIR Multispectral Driving (RASMD) dataset, which comprises 100,000 synchronized and spatially aligned RGB-SWIR image pairs collected across diverse locations, lighting, and weather conditions. In addition, we provide a subset for RGB-SWIR translation and object detection annotations for a subset of challenging traffic scenarios to demonstrate the utility of SWIR imaging through experiments on both object detection and RGB-to-SWIR image translation. Our experiments show that combining RGB and SWIR data in an ensemble framework significantly improves detection accuracy compared to RGB-only approaches, particularly in conditions where visible-spectrum sensors struggle. We anticipate that the RASMD dataset will advance research in multispectral imaging for autonomous driving and robust perception systems.
△ Less
Submitted 10 April, 2025;
originally announced April 2025.
-
Low-Cost 3D printed, Biocompatible Ionic Polymer Membranes for Soft Actuators
Authors:
Nils Trümpler,
Ryo Kanno,
Niu David,
Anja Huch,
Pham Huy Nguyen,
Maksims Jurinovs,
Gustav Nyström,
Sergejs Gaidukovs,
Mirko Kovac
Abstract:
Ionic polymer actuators, in essence, consist of ion exchange polymers sandwiched between layers of electrodes. They have recently gained recognition as promising candidates for soft actuators due to their lightweight nature, noise-free operation, and low-driving voltages. However, the materials traditionally utilized to develop them are often not human/environmentally friendly. Thus, to address th…
▽ More
Ionic polymer actuators, in essence, consist of ion exchange polymers sandwiched between layers of electrodes. They have recently gained recognition as promising candidates for soft actuators due to their lightweight nature, noise-free operation, and low-driving voltages. However, the materials traditionally utilized to develop them are often not human/environmentally friendly. Thus, to address this issue, researchers have been focusing on developing biocompatible versions of this actuator. Despite this, such actuators still face challenges in achieving high performance, in payload capacity, bending capabilities, and response time. In this paper, we present a biocompatible ionic polymer actuator whose membrane is fully 3D printed utilizing a direct ink writing method. The structure of the printed membranes consists of biodegradable ionic fluid encapsulated within layers of activated carbon polymers. From the microscopic observations of its structure, we confirmed that the ionic polymer is well encapsulated. The actuators can achieve a bending performance of up to 124$^\circ$ (curvature of 0.82 $\text{cm}^{-1}$), which, to our knowledge, is the highest curvature attained by any bending ionic polymer actuator to date. It can operate comfortably up to a 2 Hz driving frequency and can achieve blocked forces of up to 0.76 mN. Our results showcase a promising, high-performing biocompatible ionic polymer actuator, whose membrane can be easily manufactured in a single step using a standard FDM 3D printer. This approach paves the way for creating customized designs for functional soft robotic applications, including human-interactive devices, in the near future.
△ Less
Submitted 21 January, 2025;
originally announced January 2025.
-
Features characterizing safe aerial-aquatic robots
Authors:
Andrea Giordano,
Luca Romanello,
Diego Perez Gonzalez,
Mirko Kovac,
Sophie F. Armanini
Abstract:
This paper underscores the importance of environmental monitoring, and specifically of freshwater ecosystems, which play a critical role in sustaining life and global economy. Despite their importance, insufficient data availability prevents a comprehensive understanding of these ecosystems, thereby impeding informed decision-making concerning their preservation. Aerial-aquatic robots are identifi…
▽ More
This paper underscores the importance of environmental monitoring, and specifically of freshwater ecosystems, which play a critical role in sustaining life and global economy. Despite their importance, insufficient data availability prevents a comprehensive understanding of these ecosystems, thereby impeding informed decision-making concerning their preservation. Aerial-aquatic robots are identified as effective tools for freshwater sensing, offering rapid deployment and avoiding the need of using ships and manned teams.
To advance the field of aerial aquatic robots, this paper conducts a comprehensive review of air-water transitions focusing on the water entry strategy of existing prototypes. This analysis also highlights the safety risks associated with each transition and proposes a set of design requirements relating to robots' tasks, mission objectives, and safety measures. To further explore the proposed design requirements, we present a novel robot with VTOL capability, enabling seamless air water transitions.
△ Less
Submitted 31 October, 2024;
originally announced October 2024.
-
TumblerBots: Tumbling Robotic sensors for Minimally-invasive Benthic Monitoring
Authors:
L. Romanello,
A. Teboul,
F. Wiesemuller,
P. H. Nguyen,
M. Kovac,
S. F. Armanini
Abstract:
Robotic systems show significant promise for water environmental sensing applications such as water quality monitoring, pollution mapping and biodiversity data collection.
Conventional deployment methods often disrupt fragile ecosystems, preventing depiction of the undisturbed environmental condition. In response to this challenge, we propose a novel framework utilizing a lightweight tumbler sys…
▽ More
Robotic systems show significant promise for water environmental sensing applications such as water quality monitoring, pollution mapping and biodiversity data collection.
Conventional deployment methods often disrupt fragile ecosystems, preventing depiction of the undisturbed environmental condition. In response to this challenge, we propose a novel framework utilizing a lightweight tumbler system equipped with a sensing unit, deployed via a drone. This design minimizes disruption to the water habitat by maintaining a slow descent. The sensing unit is detached once on the water surface, enabling precise and non-invasive data collection from the benthic zone.
The tumbler is designed to be lightweight and compact, enabling deployment via a drone. The sensing pod, which detaches from the tumbler and descends to the bottom of the water body, is equipped with temperature and pressure sensors, as well as a buoyancy system. The later, activated upon task completion, utilizes a silicon membrane inflated via a chemical reaction. The reaction generates a pressure of 70 kPa, causing the silicon membrane to expand by 30\%, which exceeds the 5.7\% volume increase required for positive buoyancy. The tumblers, made from ecofriendly materials to minimize environmental impact when lost during the mission, were tested for their gliding ratio and descent rate. They exhibit a low descent rate, in the range of 0.8 to 2.5 meters per seconds, which minimizes disturbance to the ecosystem upon water landing. Additionally, the system demonstrated robustness in moderate to strong wind conditions during outdoor tests, validating the overall framework.
△ Less
Submitted 30 October, 2024;
originally announced October 2024.
-
Exploring the Potential of Multi-modal Sensing Framework for Forest Ecology
Authors:
Luca Romanello,
Tian Lan,
Mirko Kovac,
Sophie F. Armanini,
Basaran Bahadir Kocer
Abstract:
Forests offer essential resources and services to humanity, yet preserving and restoring them presents challenges, particularly due to the limited availability of actionable data, especially in hard-to-reach areas like forest canopies. Accessibility continues to pose a challenge for biologists collecting data in forest environments, often requiring them to invest significant time and energy in cli…
▽ More
Forests offer essential resources and services to humanity, yet preserving and restoring them presents challenges, particularly due to the limited availability of actionable data, especially in hard-to-reach areas like forest canopies. Accessibility continues to pose a challenge for biologists collecting data in forest environments, often requiring them to invest significant time and energy in climbing trees to place sensors. This operation not only consumes resources but also exposes them to danger. Efforts in robotics have been directed towards accessing the tree canopy using robots. A swarm of drones has showcased autonomous navigation through the canopy, maneuvering with agility and evading tree collisions, all aimed at mapping the area and collecting data. However, relying solely on free-flying drones has proven insufficient for data collection. Flying drones within the canopy generates loud noise, disturbing animals and potentially corrupting the data. Additionally, commercial drones often have limited autonomy for dexterous tasks where aerial physical interaction could be required, further complicating data acquisition efforts. Aerial deployed sensor placement methods such as bio-gliders and sensor shooting have proven effective for data collection within the lower canopy. However, these methods face challenges related to retrieving the data and sensors, often necessitating human intervention.
△ Less
Submitted 30 October, 2024;
originally announced October 2024.
-
Camber-changing flapping hydrofoils for efficient and environmental-safe water propulsion system
Authors:
Luca Romanello,
Leonard Hohaus,
David-Marian Schmitt,
Mirko Kovac,
Sophie F. Armanini
Abstract:
This research introduces a novel hydrofoil-based propulsion framework for unmanned aquatic robots, inspired by the undulating locomotion observed in select aquatic species. The proposed system incorporates a camber-modulating mechanism to enhance hydrofoil propulsive force generation and eventually efficiency. Through dynamic simulations, we validate the effectiveness of the camber-adjusting hydro…
▽ More
This research introduces a novel hydrofoil-based propulsion framework for unmanned aquatic robots, inspired by the undulating locomotion observed in select aquatic species. The proposed system incorporates a camber-modulating mechanism to enhance hydrofoil propulsive force generation and eventually efficiency. Through dynamic simulations, we validate the effectiveness of the camber-adjusting hydrofoil compared to a symmetric counterpart. The results demonstrate a significant improvement in horizontal thrust, emphasizing the potential of the cambering approach to enhance propulsive performance. Additionally, a prototype flipper design is presented, featuring individual control of heave and pitch motions, as well as a camber-adjustment mechanism. The integrated system not only provides efficient water-based propulsion but also offers the capacity for generating vertical forces during take-off maneuvers for seaplanes. The design is tailored to harness wave energy, contributing to the exploration of alternative energy resources. This work advances the understanding of bionic oscillatory principles for aquatic robots and provides a foundation for future developments in environmentally safe and agile underwater exploration.
△ Less
Submitted 30 October, 2024;
originally announced October 2024.
-
Hybrid Soft Electrostatic Metamaterial Gripper for Multi-surface, Multi-object Adaptation
Authors:
Ryo Kanno,
Pham H. Nguyen,
Joshua Pinskier,
David Howard,
Sukho Song,
Mirko Kovac
Abstract:
One of the trendsetting themes in soft robotics has been the goal of developing the ultimate universal soft robotic gripper. One that is capable of manipulating items of various shapes, sizes, thicknesses, textures, and weights. All the while still being lightweight and scalable in order to adapt to use cases. In this work, we report a soft gripper that enables delicate and precise grasps of fragi…
▽ More
One of the trendsetting themes in soft robotics has been the goal of developing the ultimate universal soft robotic gripper. One that is capable of manipulating items of various shapes, sizes, thicknesses, textures, and weights. All the while still being lightweight and scalable in order to adapt to use cases. In this work, we report a soft gripper that enables delicate and precise grasps of fragile, deformable, and flexible objects but also excels in lifting heavy objects of up to 1617x its own body weight. The principle behind the soft gripper is based on extending the capabilities of electroadhesion soft grippers through the enhancement principles found in metamaterial adhesion cut and patterning. This design amplifies the adhesion and grasping payload in one direction while reducing the adhesion capabilities in the other direction. This counteracts the residual forces during peeling (a common problem with electroadhesive grippers), thus increasing its speed of release. In essence, we are able to tune the maximum strength and peeling speed, beyond the capabilities of previous electroadhesive grippers. We study the capabilities of the system through a wide range of experiments with single and multiple-fingered peel tests. We also demonstrate its modular and adaptive capabilities in the real-world with a two-finger gripper, by performing grasping tests of up to $5$ different multi-surfaced objects.
△ Less
Submitted 10 March, 2024;
originally announced March 2024.
-
Gotta catch 'em all, safely! Aerial-deployed soft underwater gripper
Authors:
Luca Romanello,
Daniel Joseph Amir,
Heinrich Stengel,
Mirko Kovac,
Sophie F. Armanini
Abstract:
Underwater soft grippers exhibit potential for applications such as monitoring, research, and object retrieval. However, existing underwater gripping techniques frequently cause disturbances to ecosystems. In response to this challenge, we present a novel underwater gripping framework comprising a lightweight gripper affixed to a custom submarine pod deployable via drone. This approach minimizes w…
▽ More
Underwater soft grippers exhibit potential for applications such as monitoring, research, and object retrieval. However, existing underwater gripping techniques frequently cause disturbances to ecosystems. In response to this challenge, we present a novel underwater gripping framework comprising a lightweight gripper affixed to a custom submarine pod deployable via drone. This approach minimizes water disturbance and enables efficient navigation to target areas, enhancing overall mission effectiveness. The pod allows for underwater motion and is characterized by four degrees of freedom. It is provided with a custom buoyancy system, two water pumps for differential thrust and two for pitching. The system allows for buoyancy adjustments up to a depth of 6 meters, as well as motion in the plane. The 3-fingered gripper is manufactured out of silicone and was successfully tested on objects with different shapes and sizes, demonstrating a maximum pulling force of up to 8 N when underwater. The reliability of the submarine pod was tested in a water tank by tracking its attitude and energy consumption during grasping maneuvers. The system also accomplished a successful mission in a lake, where it was deployed on a hexacopter. Overall, the integration of this system expands the operational capabilities of underwater grasping, makes grasping missions more efficient and easy to automate, as well as causing less disturbance to the water ecosystem.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Aerial Tensile Perching and Disentangling Mechanism for Long-Term Environmental Monitoring
Authors:
Tian Lan,
Luca Romanello,
Mirko Kovac,
Sophie F. Armanini,
Basaran Bahadir Kocer
Abstract:
Aerial robots show significant potential for forest canopy research and environmental monitoring by providing data collection capabilities at high spatial and temporal resolutions. However, limited flight endurance hinders their application. Inspired by natural perching behaviours, we propose a multi-modal aerial robot system that integrates tensile perching for energy conservation and a suspended…
▽ More
Aerial robots show significant potential for forest canopy research and environmental monitoring by providing data collection capabilities at high spatial and temporal resolutions. However, limited flight endurance hinders their application. Inspired by natural perching behaviours, we propose a multi-modal aerial robot system that integrates tensile perching for energy conservation and a suspended actuated pod for data collection. The system consists of a quadrotor drone, a slewing ring mechanism allowing 360° tether rotation, and a streamlined pod with two ducted propellers connected via a tether. Winding and unwinding the tether allows the pod to move within the canopy, and activating the propellers allows the tether to be wrapped around branches for perching or disentangling. We experimentally determined the minimum counterweights required for stable perching under various conditions. Building on this, we devised and evaluated multiple perching and disentangling strategies. Comparisons of perching and disentangling manoeuvres demonstrate energy savings that could be further maximized with the use of the pod or tether winding. These approaches can reduce energy consumption to only 22\% and 1.5\%, respectively, compared to a drone disentangling manoeuvre. We also calculated the minimum idle time required by the proposed system after the system perching and motor shut down to save energy on a mission, which is 48.9\% of the operating time. Overall, the integrated system expands the operational capabilities and enhances the energy efficiency of aerial robots for long-term monitoring tasks.
△ Less
Submitted 5 March, 2024; v1 submitted 4 March, 2024;
originally announced March 2024.
-
Mask-Mediator-Wrapper architecture as a Data Mesh driver
Authors:
Juraj Dončević,
Krešimir Fertalj,
Mario Brčić,
Mihael Kovač
Abstract:
The data mesh is a novel data management concept that emphasises the importance of a domain before technology. The concept is still in the early stages of development and many efforts to implement and use it are expected to have negative consequences for organizations due to a lack of technological guidelines and best practices. To mitigate the risk of negative outcomes this paper proposes the use…
▽ More
The data mesh is a novel data management concept that emphasises the importance of a domain before technology. The concept is still in the early stages of development and many efforts to implement and use it are expected to have negative consequences for organizations due to a lack of technological guidelines and best practices. To mitigate the risk of negative outcomes this paper proposes the use of the mask-mediator-wrapper architecture as a data mesh driver. The mask-mediator-wrapper architecture provides a set of prefabricated configurable components that provide basic functionalities which a data mesh requires. This paper shows how the two concepts are compatible in terms of functionality, data modelling, evolvability and aligned capabilities. A mask-mediator-wrapper driven data mesh facilitates: low-risk adoption trials, rapid prototyping, standardization, and a guarantee of evolvability.
△ Less
Submitted 10 September, 2022;
originally announced September 2022.
-
Maximum Power Point Tracking Circuit for an Energy Harvester in 130 nm CMOS Technology
Authors:
Adam Hudec,
Lukas Nagy,
Martin Kovac,
Viera Stopjakova
Abstract:
This paper presents design of a Maximum Power Point Tracking (MPPT) circuit and its functionality for tuning the maximum power transfer from an energy harvester (EH) unit. Simple and practical Perturb and Observe algorithm is investigated and implemented. We describe the circuit functionality and the improvements that have been introduced to the original algorithm. The proposed MPPT design is divi…
▽ More
This paper presents design of a Maximum Power Point Tracking (MPPT) circuit and its functionality for tuning the maximum power transfer from an energy harvester (EH) unit. Simple and practical Perturb and Observe algorithm is investigated and implemented. We describe the circuit functionality and the improvements that have been introduced to the original algorithm. The proposed MPPT design is divided into three main blocks. The output signal is being generated by the PWM or PFM block. The tracking speed has been enhanced by implementing a variable step size in the Tracking Block. Finally, the overall power consumption of the MPPT circuit itself is controlled by the Power Management Block, which manages delivering the clock signal to the rest of the circuit. The RTL code of the proposed MPPT has been described in Verilog, then has been synthesized and placed-and-routed in a general purpose 130nm CMOS technology.
△ Less
Submitted 23 September, 2021;
originally announced October 2021.
-
Error State Extended Kalman Filter Multi-Sensor Fusion for Unmanned Aerial Vehicle Localization in GPS and Magnetometer Denied Indoor Environments
Authors:
Lovro Markovic,
Marin Kovac,
Robert Milijas,
Marko Car,
Stjepan Bogdan
Abstract:
This paper addresses the issues of unmanned aerial vehicle (UAV) indoor navigation, specifically in areas where GPS and magnetometer sensor measurements are unavailable or unreliable. The proposed solution is to use an error state extended Kalman filter (ES -EKF) in the context of multi-sensor fusion. Its implementation is adapted to fuse measurements from multiple sensor sources and the state mod…
▽ More
This paper addresses the issues of unmanned aerial vehicle (UAV) indoor navigation, specifically in areas where GPS and magnetometer sensor measurements are unavailable or unreliable. The proposed solution is to use an error state extended Kalman filter (ES -EKF) in the context of multi-sensor fusion. Its implementation is adapted to fuse measurements from multiple sensor sources and the state model is extended to account for sensor drift and possible calibration inaccuracies. Experimental validation is performed by fusing IMU data obtained from the PixHawk 2.1 flight controller with pose measurements from LiDAR Cartographer SLAM, visual odometry provided by the Intel T265 camera and position measurements from the Pozyx UWB indoor positioning system. The estimated odometry from ES-EKF is validated against ground truth data from the Optitrack motion capture system and its use in a position control loop to stabilize the UAV is demonstrated.
△ Less
Submitted 10 September, 2021;
originally announced September 2021.
-
Robotic Electrospinning Actuated by Non-Circular Joint Continuum Manipulator for Endoluminal Therapy
Authors:
Zicong Wu,
Chuqian Lou,
Zhu Jin,
Shaoping Huang,
Ning Liu,
Yun Zou,
Mirko Kovac,
Anzhu Gao,
Guang-Zhong Yang
Abstract:
Electrospinning has exhibited excellent benefits to treat the trauma for tissue engineering due to its produced micro/nano fibrous structure. It can effectively adhere to the tissue surface for long-term continuous therapy. This paper develops a robotic electrospinning platform for endoluminal therapy. The platform consists of a continuum manipulator, the electrospinning device, and the actuation…
▽ More
Electrospinning has exhibited excellent benefits to treat the trauma for tissue engineering due to its produced micro/nano fibrous structure. It can effectively adhere to the tissue surface for long-term continuous therapy. This paper develops a robotic electrospinning platform for endoluminal therapy. The platform consists of a continuum manipulator, the electrospinning device, and the actuation unit. The continuum manipulator has two bending sections to facilitate the steering of the tip needle for a controllable spinning direction. Non-circular joint profile is carefully designed to enable a constant length of the centreline of a continuum manipulator for stable fluid transmission inside it. Experiments are performed on a bronchus phantom, and the steering ability and bending limitation in each direction are also investigated. The endoluminal electrospinning is also fulfilled by a trajectory following and points targeting experiments. The effective adhesive area of the produced fibre is also illustrated. The proposed robotic electrospinning shows its feasibility to precisely spread more therapeutic drug to construct fibrous structure for potential endoluminal treatment.
△ Less
Submitted 7 June, 2021;
originally announced June 2021.
