-
A multi-modal approach to continuous material identification through tactile sensing
Authors:
Augusto Gómez Eguíluz,
Ignacio Rañó,
Sonya A. Coleman,
T. Martin McGinnity
Abstract:
Tactile sensing has recently been used in robotics for object identification, grasping, and material recognition. Most material recognition approaches use vibration information from a tactile exploration, typically above one second long, to identify the material. This work proposes a tactile multi-modal (vibration and thermal) material identification approach based on recursive Bayesian estimation…
▽ More
Tactile sensing has recently been used in robotics for object identification, grasping, and material recognition. Most material recognition approaches use vibration information from a tactile exploration, typically above one second long, to identify the material. This work proposes a tactile multi-modal (vibration and thermal) material identification approach based on recursive Bayesian estimation. Through the frequency response of the vibration induced by the material and thermal features, like an estimate of the thermal power loss of the finger, we show that it is possible to identify materials in less than half a second. Moreover, a comparison between the use of vibration only and multi-modal identification shows that both recognition time and classification errors are reduced by adding thermal information.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
Exploiting High Quality Tactile Sensors for Simplified Grasping
Authors:
Pedro Machado,
T. M. McGinnity
Abstract:
Robots are expected to grasp a wide range of objects varying in shape, weight or material type. Providing robots with tactile capabilities similar to humans is thus essential for applications involving human-to-robot or robot-to-robot interactions, particularly in those situations where a robot is expected to grasp and manipulate complex objects not previously encountered. A critical aspect for su…
▽ More
Robots are expected to grasp a wide range of objects varying in shape, weight or material type. Providing robots with tactile capabilities similar to humans is thus essential for applications involving human-to-robot or robot-to-robot interactions, particularly in those situations where a robot is expected to grasp and manipulate complex objects not previously encountered. A critical aspect for successful object grasp and manipulation is the use of high-quality fingertips equipped with multiple high-performance sensors, distributed appropriately across a specific contact surface.
In this paper, we present a detailed analysis of the use of two different types of commercially available robotic fingertips (BioTac and WTS-FT), each of which is equipped with multiple sensors distributed across the fingertips' contact surface. We further demonstrate that, due to the high performance of the fingertips, a complex adaptive grasping algorithm is not required for grasping of everyday objects. We conclude that a simple algorithm based on a proportional controller will suffice for many grasping applications, provided the relevant fingertips exhibit high sensitivity. In a quantified assessment, we also demonstrate that, due in part to the sensor distribution, the BioTac-based fingertip performs better than the WTS-FT device, in enabling lifting of loads up to 850g, and that the simple proportional controller can adapt the grasp even when the object is exposed to significant external vibrational challenges.
△ Less
Submitted 25 July, 2022;
originally announced July 2022.
-
NeuroHSMD: Neuromorphic Hybrid Spiking Motion Detector
Authors:
Pedro Machado,
Joao Filipe Ferreira,
Andreas Oikonomou,
T. M. McGinnity
Abstract:
Vertebrate retinas are highly-efficient in processing trivial visual tasks such as detecting moving objects, yet a complex challenges for modern computers. In vertebrates, the detection of object motion is performed by specialised retinal cells named Object Motion Sensitive Ganglion Cells (OMS-GC). OMS-GC process continuous visual signals and generate spike patterns that are post-processed by the…
▽ More
Vertebrate retinas are highly-efficient in processing trivial visual tasks such as detecting moving objects, yet a complex challenges for modern computers. In vertebrates, the detection of object motion is performed by specialised retinal cells named Object Motion Sensitive Ganglion Cells (OMS-GC). OMS-GC process continuous visual signals and generate spike patterns that are post-processed by the Visual Cortex. Our previous Hybrid Sensitive Motion Detector (HSMD) algorithm was the first hybrid algorithm to enhance Background subtraction (BS) algorithms with a customised 3-layer Spiking Neural Network (SNN) that generates OMS-GC spiking-like responses. In this work, we present a Neuromorphic Hybrid Sensitive Motion Detector (NeuroHSMD) algorithm that accelerates our HSMD algorithm using Field-Programmable Gate Arrays (FPGAs). The NeuroHSMD was compared against the HSMD algorithm, using the same 2012 Change Detection (CDnet2012) and 2014 Change Detection (CDnet2014) benchmark datasets. When tested against the CDnet2012 and CDnet2014 datasets, NeuroHSMD performs object motion detection at 720x480 at 28.06 Frames Per Second (fps) and 720x480 at 28.71 fps, respectively, with no degradation of quality. Moreover, the NeuroHSMD proposed in this paper was completely implemented in Open Computer Language (OpenCL) and therefore is easily replicated in other devices such as Graphical Processing Units (GPUs) and clusters of Central Processing Units (CPUs).
△ Less
Submitted 14 February, 2023; v1 submitted 11 December, 2021;
originally announced December 2021.
-
Object recognition for robotics from tactile time series data utilising different neural network architectures
Authors:
Wolfgang Bottcher,
Pedro Machado,
Nikesh Lama,
T. M. McGinnity
Abstract:
Robots need to exploit high-quality information on grasped objects to interact with the physical environment. Haptic data can therefore be used for supplementing the visual modality. This paper investigates the use of Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) neural network architectures for object classification on Spatio-temporal tactile grasping data. Furthermore, we…
▽ More
Robots need to exploit high-quality information on grasped objects to interact with the physical environment. Haptic data can therefore be used for supplementing the visual modality. This paper investigates the use of Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) neural network architectures for object classification on Spatio-temporal tactile grasping data. Furthermore, we compared these methods using data from two different fingertip sensors (namely the BioTac SP and WTS-FT) in the same physical setup, allowing for a realistic comparison across methods and sensors for the same tactile object classification dataset. Additionally, we propose a way to create more training examples from the recorded data. The results show that the proposed method improves the maximum accuracy from 82.4% (BioTac SP fingertips) and 90.7% (WTS-FT fingertips) with complete time-series data to about 94% for both sensor types.
△ Less
Submitted 9 September, 2021;
originally announced September 2021.
-
HSMD: An object motion detection algorithm using a Hybrid Spiking Neural Network Architecture
Authors:
Pedro Machado,
Andreas Oikonomou,
Joao Filipe Ferreira,
T. M. McGinnity
Abstract:
The detection of moving objects is a trivial task performed by vertebrate retinas, yet a complex computer vision task. Object-motion-sensitive ganglion cells (OMS-GC) are specialised cells in the retina that sense moving objects. OMS-GC take as input continuous signals and produce spike patterns as output, that are transmitted to the Visual Cortex via the optic nerve. The Hybrid Sensitive Motion D…
▽ More
The detection of moving objects is a trivial task performed by vertebrate retinas, yet a complex computer vision task. Object-motion-sensitive ganglion cells (OMS-GC) are specialised cells in the retina that sense moving objects. OMS-GC take as input continuous signals and produce spike patterns as output, that are transmitted to the Visual Cortex via the optic nerve. The Hybrid Sensitive Motion Detector (HSMD) algorithm proposed in this work enhances the GSOC dynamic background subtraction (DBS) algorithm with a customised 3-layer spiking neural network (SNN) that outputs spiking responses akin to the OMS-GC. The algorithm was compared against existing background subtraction (BS) approaches, available on the OpenCV library, specifically on the 2012 change detection (CDnet2012) and the 2014 change detection (CDnet2014) benchmark datasets. The results show that the HSMD was ranked overall first among the competing approaches and has performed better than all the other algorithms on four of the categories across all the eight test metrics. Furthermore, the HSMD proposed in this paper is the first to use an SNN to enhance an existing state of the art DBS (GSOC) algorithm and the results demonstrate that the SNN provides near real-time performance in realistic applications.
△ Less
Submitted 9 September, 2021;
originally announced September 2021.
-
The Challenges and Opportunities of Human-Centered AI for Trustworthy Robots and Autonomous Systems
Authors:
Hongmei He,
John Gray,
Angelo Cangelosi,
Qinggang Meng,
T. Martin McGinnity,
Jörn Mehnen
Abstract:
The trustworthiness of Robots and Autonomous Systems (RAS) has gained a prominent position on many research agendas towards fully autonomous systems. This research systematically explores, for the first time, the key facets of human-centered AI (HAI) for trustworthy RAS. In this article, five key properties of a trustworthy RAS initially have been identified. RAS must be (i) safe in any uncertain…
▽ More
The trustworthiness of Robots and Autonomous Systems (RAS) has gained a prominent position on many research agendas towards fully autonomous systems. This research systematically explores, for the first time, the key facets of human-centered AI (HAI) for trustworthy RAS. In this article, five key properties of a trustworthy RAS initially have been identified. RAS must be (i) safe in any uncertain and dynamic surrounding environments; (ii) secure, thus protecting itself from any cyber-threats; (iii) healthy with fault tolerance; (iv) trusted and easy to use to allow effective human-machine interaction (HMI), and (v) compliant with the law and ethical expectations. Then, the challenges in implementing trustworthy autonomous system are analytically reviewed, in respects of the five key properties, and the roles of AI technologies have been explored to ensure the trustiness of RAS with respects to safety, security, health and HMI, while reflecting the requirements of ethics in the design of RAS. While applications of RAS have mainly focused on performance and productivity, the risks posed by advanced AI in RAS have not received sufficient scientific attention. Hence, a new acceptance model of RAS is provided, as a framework for requirements to human-centered AI and for implementing trustworthy RAS by design. This approach promotes human-level intelligence to augment human's capacity. while focusing on contributions to humanity.
△ Less
Submitted 7 May, 2021;
originally announced May 2021.
-
Strawberry Detection Using a Heterogeneous Multi-Processor Platform
Authors:
Samuel Brandenburg,
Pedro Machado,
Nikesh Lama,
T. M. McGinnity
Abstract:
Over the last few years, the number of precision farming projects has increased specifically in harvesting robots and many of which have made continued progress from identifying crops to grasping the desired fruit or vegetable. One of the most common issues found in precision farming projects is that successful application is heavily dependent not just on identifying the fruit but also on ensuring…
▽ More
Over the last few years, the number of precision farming projects has increased specifically in harvesting robots and many of which have made continued progress from identifying crops to grasping the desired fruit or vegetable. One of the most common issues found in precision farming projects is that successful application is heavily dependent not just on identifying the fruit but also on ensuring that localisation allows for accurate navigation. These issues become significant factors when the robot is not operating in a prearranged environment, or when vegetation becomes too thick, thus covering crop. Moreover, running a state-of-the-art deep learning algorithm on an embedded platform is also very challenging, resulting most of the times in low frame rates. This paper proposes using the You Only Look Once version 3 (YOLOv3) Convolutional Neural Network (CNN) in combination with utilising image processing techniques for the application of precision farming robots targeting strawberry detection, accelerated on a heterogeneous multiprocessor platform. The results show a performance acceleration by five times when implemented on a Field-Programmable Gate Array (FPGA) when compared with the same algorithm running on the processor side with an accuracy of 78.3\% over the test set comprised of 146 images.
△ Less
Submitted 6 November, 2020;
originally announced November 2020.
-
On-Device Transfer Learning for Personalising Psychological Stress Modelling using a Convolutional Neural Network
Authors:
Kieran Woodward,
Eiman Kanjo,
David J. Brown,
T. M. McGinnity
Abstract:
Stress is a growing concern in modern society adversely impacting the wider population more than ever before. The accurate inference of stress may result in the possibility for personalised interventions. However, individual differences between people limits the generalisability of machine learning models to infer emotions as people's physiology when experiencing the same emotions widely varies. I…
▽ More
Stress is a growing concern in modern society adversely impacting the wider population more than ever before. The accurate inference of stress may result in the possibility for personalised interventions. However, individual differences between people limits the generalisability of machine learning models to infer emotions as people's physiology when experiencing the same emotions widely varies. In addition, it is time consuming and extremely challenging to collect large datasets of individuals' emotions as it relies on users labelling sensor data in real-time for extended periods. We propose the development of a personalised, cross-domain 1D CNN by utilising transfer learning from an initial base model trained using data from 20 participants completing a controlled stressor experiment. By utilising physiological sensors (HR, HRV EDA) embedded within edge computing interfaces that additionally contain a labelling technique, it is possible to collect a small real-world personal dataset that can be used for on-device transfer learning to improve model personalisation and cross-domain performance.
△ Less
Submitted 3 April, 2020;
originally announced April 2020.
-
NatCSNN: A Convolutional Spiking Neural Network for recognition of objects extracted from natural images
Authors:
Pedro Machado,
Georgina Cosma,
T. M McGinnity
Abstract:
Biological image processing is performed by complex neural networks composed of thousands of neurons interconnected via thousands of synapses, some of which are excitatory and others inhibitory. Spiking neural models are distinguished from classical neurons by being biological plausible and exhibiting the same dynamics as those observed in biological neurons. This paper proposes a Natural Convolut…
▽ More
Biological image processing is performed by complex neural networks composed of thousands of neurons interconnected via thousands of synapses, some of which are excitatory and others inhibitory. Spiking neural models are distinguished from classical neurons by being biological plausible and exhibiting the same dynamics as those observed in biological neurons. This paper proposes a Natural Convolutional Neural Network (NatCSNN) which is a 3-layer bio-inspired Convolutional Spiking Neural Network (CSNN), for classifying objects extracted from natural images. A two-stage training algorithm is proposed using unsupervised Spike Timing Dependent Plasticity (STDP) learning (phase 1) and ReSuMe supervised learning (phase 2). The NatCSNN was trained and tested on the CIFAR-10 dataset and achieved an average testing accuracy of 84.7% which is an improvement over the 2-layer neural networks previously applied to this dataset.
△ Less
Submitted 18 September, 2019;
originally announced September 2019.
-
Beyond Mobile Apps: A Survey of Technologies for Mental Well-being
Authors:
Kieran Woodward,
Eiman Kanjo,
David Brown,
T. M. McGinnity,
Becky Inkster,
Donald J Macintyre,
Athanasios Tsanas
Abstract:
Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties,…
▽ More
Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time.
△ Less
Submitted 23 July, 2020; v1 submitted 1 May, 2019;
originally announced May 2019.
-
PRED18: Dataset and Further Experiments with DAVIS Event Camera in Predator-Prey Robot Chasing
Authors:
Diederik Paul Moeys,
Daniel Neil,
Federico Corradi,
Emmett Kerr,
Philip Vance,
Gautham Das,
Sonya A. Coleman,
Thomas M. McGinnity,
Dermot Kerr,
Tobi Delbruck
Abstract:
Machine vision systems using convolutional neural networks (CNNs) for robotic applications are increasingly being developed. Conventional vision CNNs are driven by camera frames at constant sample rate, thus achieving a fixed latency and power consumption tradeoff. This paper describes further work on the first experiments of a closed-loop robotic system integrating a CNN together with a Dynamic a…
▽ More
Machine vision systems using convolutional neural networks (CNNs) for robotic applications are increasingly being developed. Conventional vision CNNs are driven by camera frames at constant sample rate, thus achieving a fixed latency and power consumption tradeoff. This paper describes further work on the first experiments of a closed-loop robotic system integrating a CNN together with a Dynamic and Active Pixel Vision Sensor (DAVIS) in a predator/prey scenario. The DAVIS, mounted on the predator Summit XL robot, produces frames at a fixed 15 Hz frame-rate and Dynamic Vision Sensor (DVS) histograms containing 5k ON and OFF events at a variable frame-rate ranging from 15-500 Hz depending on the robot speeds. In contrast to conventional frame-based systems, the latency and processing cost depends on the rate of change of the image. The CNN is trained offline on the 1.25h labeled dataset to recognize the position and size of the prey robot, in the field of view of the predator. During inference, combining the ten output classes of the CNN allows extracting the analog position vector of the prey relative to the predator with a mean 8.7% error in angular estimation. The system is compatible with conventional deep learning technology, but achieves a variable latency-power tradeoff that adapts automatically to the dynamics. Finally, investigations on the robustness of the algorithm, a human performance comparison and a deconvolution analysis are also explored.
△ Less
Submitted 2 July, 2018;
originally announced July 2018.
