-
Leveraging Tactile Sensors for Low Latency Embedded Smart Hands for Prosthetic and Robotic Applications
Authors:
Xiaying Wang,
Fabian Geiger,
Vlad Niculescu,
Michele Magno,
Luca Benini
Abstract:
Tactile sensing is a crucial perception mode for robots and human amputees in need of controlling a prosthetic device. Today robotic and prosthetic systems are still missing the important feature of accurate tactile sensing. This lack is mainly due to the fact that the existing tactile technologies have limited spatial and temporal resolution and are either expensive or not scalable. In this paper…
▽ More
Tactile sensing is a crucial perception mode for robots and human amputees in need of controlling a prosthetic device. Today robotic and prosthetic systems are still missing the important feature of accurate tactile sensing. This lack is mainly due to the fact that the existing tactile technologies have limited spatial and temporal resolution and are either expensive or not scalable. In this paper, we present the design and the implementation of a hardware-software embedded system called SmartHand. It is specifically designed to enable the acquisition and the real-time processing of high-resolution tactile information from a hand-shaped multi-sensor array for prosthetic and robotic applications. During data collection, our system can deliver a high throughput of 100 frames per second, which is 13.7x higher than previous related work. We collected a new tactile dataset while interacting with daily-life objects during five different sessions. We propose a compact yet accurate convolutional neural network that requires one order of magnitude less memory and 15.6x fewer computations compared to related work without degrading classification accuracy. The top-1 and top-3 cross-validation accuracies are respectively 98.86% and 99.83%. We further analyze the inter-session variability and obtain the best top-3 leave-one-out-validation accuracy of 77.84%. We deploy the trained model on a high-performance ARM Cortex-M7 microcontroller achieving an inference time of only 100 ms minimizing the response latency. The overall measured power consumption is 505 mW. Finally, we fabricate a new control sensor and perform additional experiments to provide analyses on sensor degradation and slip detection. This work is a step forward in giving robotic and prosthetic devices a sense of touch and demonstrates the practicality of a smart embedded system empowered by tiny machine learning.
△ Less
Submitted 28 March, 2022;
originally announced March 2022.
-
SmartHand: Towards Embedded Smart Hands for Prosthetic and Robotic Applications
Authors:
Xiaying Wang,
Fabian Geiger,
Vlad Niculescu,
Michele Magno,
Luca Benini
Abstract:
The sophisticated sense of touch of the human hand significantly contributes to our ability to safely, efficiently, and dexterously manipulate arbitrary objects in our environment. Robotic and prosthetic devices lack refined, tactile feedback from their end-effectors, leading to counterintuitive and complex control strategies. To address this lack, tactile sensors have been designed and developed,…
▽ More
The sophisticated sense of touch of the human hand significantly contributes to our ability to safely, efficiently, and dexterously manipulate arbitrary objects in our environment. Robotic and prosthetic devices lack refined, tactile feedback from their end-effectors, leading to counterintuitive and complex control strategies. To address this lack, tactile sensors have been designed and developed, but they often offer an insufficient spatial and temporal resolution. This paper focuses on overcoming these issues by designing a smart embedded system, called SmartHand, enabling the acquisition and real-time processing of high-resolution tactile information from a hand-shaped multi-sensor array for prosthetic and robotic applications. We acquire a new tactile dataset consisting of 340,000 frames while interacting with 16 everyday objects and the empty hand, i.e., a total of 17 classes. The design of the embedded system minimizes response latency in classification, by deploying a small yet accurate convolutional neural network on a high-performance ARM Cortex-M7 microcontroller. Compared to related work, our model requires one order of magnitude less memory and 15.6x fewer computations while achieving similar inter-session accuracy and up to 98.86% and 99.83% top-1 and top-3 cross-validation accuracy, respectively. Experimental results show a total power consumption of 505mW and a latency of only 100ms.
△ Less
Submitted 23 July, 2021;
originally announced July 2021.
-
Constrained motion design with distinct actuators and motion stabilization
Authors:
Renate Sachse,
Florian Geiger,
Manfred Bischoff
Abstract:
The design of adaptive structures is one method to improve sustainability of buildings. Adaptive structures are able to adapt to different loading and environmental conditions or to changing requirements by either small or large shape changes. In the latter case, also the mechanics and properties of the deformation process play a role for the structure's energy efficiency. The method of variationa…
▽ More
The design of adaptive structures is one method to improve sustainability of buildings. Adaptive structures are able to adapt to different loading and environmental conditions or to changing requirements by either small or large shape changes. In the latter case, also the mechanics and properties of the deformation process play a role for the structure's energy efficiency. The method of variational motion design, previously developed in the group of the authors, allows to identify deformation paths between two given geometrical configurations that are optimal with respect to a defined quality function. In a preliminary, academic setting this method assumes that every single degree of freedom is accessible to arbitrary external actuation forces that realize the optimized motion. These (nodal) forces can be recovered a posteriori. The present contribution deals with an extension of the method of motion design by the constraint that the motion is to be realized by a predefined set of actuation forces. These can be either external forces or prescribed length chances of discrete, internal actuator elements. As an additional constraint, static stability of each intermediate configuration during the motion is taken into account. It can be accomplished by enforcing a positive determinant of the stiffness matrix.
△ Less
Submitted 5 December, 2020;
originally announced December 2020.
-
Drawing with AI -- Exploring Collaborative Inking Experiences Based on Mid-air Pointing and Reinforcement Learning
Authors:
Franziska Geiger,
Michelle Martin,
Monika Pichlmair,
Ilhan Aslan,
Hannes Ritschel,
Björn Bittner,
Elisabeth André
Abstract:
Digitalization is changing the nature of tools and materials, which are used in artistic practices in professional and non-professional settings. For example, today it is common that even children express their ideas and explore their creativity by drawing on tablets as digital canvases. While there are many software-based tools, which resemble traditional tools, such as various forms of virtual b…
▽ More
Digitalization is changing the nature of tools and materials, which are used in artistic practices in professional and non-professional settings. For example, today it is common that even children express their ideas and explore their creativity by drawing on tablets as digital canvases. While there are many software-based tools, which resemble traditional tools, such as various forms of virtual brushes, erasers, etc. in contrast to traditional materials there is potential in augmenting software-based tools and digital canvases with artificial intelligence. Curious about how it would feel to interact with a digital canvas, which would be in contrast to a traditional canvas dynamic, responsive, and potentially able to continuously adapt to its user's input, we developed a drawing application and conducted a qualitative study with 14 users. In this paper, we describe details of our design process, which lead up to using a k-armed bandit as a simple form of reinforcement learning and a LeapMotion sensor to allow people from all walks of like, old and young to draw on pervasive displays, small and large, positioned near or far.
△ Less
Submitted 10 October, 2020;
originally announced October 2020.
-
Datasets of Android Applications: a Literature Review
Authors:
Franz-Xaver Geiger,
Ivano Malavolta
Abstract:
Mobile phones and tablets have become the most widely used computing devices, with a large predominance of the Android platform. As a natural evolution, the development of Android applications has surged and has become a major field of study, with research efforts ranging from energy efficiency, to code smells, performance, maintainability, security, etc. These kind of challenges ask for dedicated…
▽ More
Mobile phones and tablets have become the most widely used computing devices, with a large predominance of the Android platform. As a natural evolution, the development of Android applications has surged and has become a major field of study, with research efforts ranging from energy efficiency, to code smells, performance, maintainability, security, etc. These kind of challenges ask for dedicated solutions, tools, and datasets. This survey identifies and reviews 31 existing datasets of Android applications and classifies each of them according to key features, such as the total number of apps it contains, whether the commit history of the apps is available, whether it focusses on the source code or on the executable binaries of the apps, the sources used for building the dataset, etc. This study can benefit both the experienced and the novice researcher interested on doing research on Android apps, which can use the results of our study as a map for identifying the most suitable datasets for their research objectives.
△ Less
Submitted 26 September, 2018;
originally announced September 2018.
