-
Employing Multimodal Machine Learning for Stress Detection
Authors:
Rahee Walambe,
Pranav Nayak,
Ashmit Bhardwaj,
Ketan Kotecha
Abstract:
In the current age, human lifestyle has become more knowledge oriented leading to generation of sedentary employment. This has given rise to a number of health and mental disorders. Mental wellness is one of the most neglected but crucial aspects of today's world. Mental health issues can, both directly and indirectly, affect other sections of human physiology and impede an individual's day-to-day…
▽ More
In the current age, human lifestyle has become more knowledge oriented leading to generation of sedentary employment. This has given rise to a number of health and mental disorders. Mental wellness is one of the most neglected but crucial aspects of today's world. Mental health issues can, both directly and indirectly, affect other sections of human physiology and impede an individual's day-to-day activities and performance. However, identifying the stress and finding the stress trend for an individual leading to serious mental ailments is challenging and involves multiple factors. Such identification can be achieved accurately by fusing these multiple modalities (due to various factors) arising from behavioral patterns. Certain techniques are identified in the literature for this purpose; however, very few machine learning-based methods are proposed for such multimodal fusion tasks. In this work, a multimodal AI-based framework is proposed to monitor a person's working behavior and stress levels. We propose a methodology for efficiently detecting stress due to workload by concatenating heterogeneous raw sensor data streams (e.g., face expressions, posture, heart rate, computer interaction). This data can be securely stored and analyzed to understand and discover personalized unique behavioral patterns leading to mental strain and fatigue. The contribution of this work is twofold; proposing a multimodal AI-based strategy for fusion to detect stress and its level and secondly identify a stress pattern over a period of time. We were able to achieve 96.09% accuracy on the test set in stress detection and classification. Further, we reduce the stress scale prediction model loss to 0.036 using these modalities. This work can prove important for the community at large, specifically those working sedentary jobs to monitor and identify stress levels, especially in current times of COVID-19.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
-
Transfer Learning for Real-time Deployment of a Screening Tool for Depression Detection Using Actigraphy
Authors:
Rajanikant Ghate,
Nayan Kalnad,
Rahee Walambe,
Ketan Kotecha
Abstract:
Automated depression screening and diagnosis is a highly relevant problem today. There are a number of limitations of the traditional depression detection methods, namely, high dependence on clinicians and biased self-reporting. In recent years, research has suggested strong potential in machine learning (ML) based methods that make use of the user's passive data collected via wearable devices. Ho…
▽ More
Automated depression screening and diagnosis is a highly relevant problem today. There are a number of limitations of the traditional depression detection methods, namely, high dependence on clinicians and biased self-reporting. In recent years, research has suggested strong potential in machine learning (ML) based methods that make use of the user's passive data collected via wearable devices. However, ML is data hungry. Especially in the healthcare domain primary data collection is challenging. In this work, we present an approach based on transfer learning, from a model trained on a secondary dataset, for the real time deployment of the depression screening tool based on the actigraphy data of users. This approach enables machine learning modelling even with limited primary data samples. A modified version of leave one out cross validation approach performed on the primary set resulted in mean accuracy of 0.96, where in each iteration one subject's data from the primary set was set aside for testing.
△ Less
Submitted 14 March, 2023;
originally announced March 2023.
-
RestoreX-AI: A Contrastive Approach towards Guiding Image Restoration via Explainable AI Systems
Authors:
Aboli Marathe,
Pushkar Jain,
Rahee Walambe,
Ketan Kotecha
Abstract:
Modern applications such as self-driving cars and drones rely heavily upon robust object detection techniques. However, weather corruptions can hinder the object detectability and pose a serious threat to their navigation and reliability. Thus, there is a need for efficient denoising, deraining, and restoration techniques. Generative adversarial networks and transformers have been widely adopted f…
▽ More
Modern applications such as self-driving cars and drones rely heavily upon robust object detection techniques. However, weather corruptions can hinder the object detectability and pose a serious threat to their navigation and reliability. Thus, there is a need for efficient denoising, deraining, and restoration techniques. Generative adversarial networks and transformers have been widely adopted for image restoration. However, the training of these methods is often unstable and time-consuming. Furthermore, when used for object detection (OD), the output images generated by these methods may provide unsatisfactory results despite image clarity. In this work, we propose a contrastive approach towards mitigating this problem, by evaluating images generated by restoration models during and post training. This approach leverages OD scores combined with attention maps for predicting the usefulness of restored images for the OD task. We conduct experiments using two novel use-cases of conditional GANs and two transformer methods that probe the robustness of the proposed approach on multi-weather corruptions in the OD task. Our approach achieves an averaged 178 percent increase in mAP between the input and restored images under adverse weather conditions like dust tornadoes and snowfall. We report unique cases where greater denoising does not improve OD performance and conversely where noisy generated images demonstrate good results. We conclude the need for explainability frameworks to bridge the gap between human and machine perception, especially in the context of robust object detection for autonomous vehicles.
△ Less
Submitted 3 April, 2022;
originally announced April 2022.
-
Incremental learning of LSTM framework for sensor fusion in attitude estimation
Authors:
Parag Narkhede,
Rahee Walambe,
Shashi Poddar,
Ketan Kotecha
Abstract:
This paper presents a novel method for attitude estimation of an object in 3D space by incremental learning of the Long-Short Term Memory (LSTM) network. Gyroscope, accelerometer, and magnetometer are few widely used sensors in attitude estimation applications. Traditionally, multi-sensor fusion methods such as the Extended Kalman Filter and Complementary Filter are employed to fuse the measuremen…
▽ More
This paper presents a novel method for attitude estimation of an object in 3D space by incremental learning of the Long-Short Term Memory (LSTM) network. Gyroscope, accelerometer, and magnetometer are few widely used sensors in attitude estimation applications. Traditionally, multi-sensor fusion methods such as the Extended Kalman Filter and Complementary Filter are employed to fuse the measurements from these sensors. However, these methods exhibit limitations in accounting for the uncertainty, unpredictability, and dynamic nature of the motion in real-world situations. In this paper, the inertial sensors data are fed to the LSTM network which are then updated incrementally to incorporate the dynamic changes in motion occurring in the run time. The robustness and efficiency of the proposed framework is demonstrated on the dataset collected from a commercially available inertial measurement unit. The proposed framework offers a significant improvement in the results compared to the traditional method, even in the case of a highly dynamic environment. The LSTM framework-based attitude estimation approach can be deployed on a standard AI-supported processing module for real-time applications.
△ Less
Submitted 4 August, 2021;
originally announced August 2021.
-
Cascaded Complementary Filter Architecture for Sensor Fusion in Attitude Estimation
Authors:
Parag Narkhede,
Shashi Poddar,
Rahee Walambe,
George Ghinea,
Ketan Kotecha
Abstract:
Attitude estimation is the process of computing the orientation angles of an object with respect to a fixed frame of reference. Gyroscope, accelerometer, and magnetometer are some of the fundamental sensors used in attitude estimation. The orientation angles computed from these sensors are combined using the sensor fusion methodologies to obtain accurate estimates. The complementary filter is one…
▽ More
Attitude estimation is the process of computing the orientation angles of an object with respect to a fixed frame of reference. Gyroscope, accelerometer, and magnetometer are some of the fundamental sensors used in attitude estimation. The orientation angles computed from these sensors are combined using the sensor fusion methodologies to obtain accurate estimates. The complementary filter is one of the widely adopted techniques whose performance is highly dependent on the appropriate selection of its gain parameters. This paper presents a novel cascaded architecture of the complementary filter that employs a nonlinear and linear version of the complementary filter within one framework. The nonlinear version is used to correct the gyroscope bias, while the linear version estimates the attitude angle. The significant advantage of the proposed architecture is its independence of the filter parameters, thereby avoiding tuning the filters gain parameters. The proposed architecture does not require any mathematical modeling of the system and is computationally inexpensive. The proposed methodology is applied to the real-world datasets, and the estimation results were found to be promising compared to the other state-of-the-art algorithms.
△ Less
Submitted 8 July, 2021;
originally announced July 2021.
-
Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion
Authors:
Parag Narkhede,
Rahee Walambe,
Shruti Mandaokar,
Pulkit Chandel,
Ketan Kotecha,
George Ghinea
Abstract:
With the rapid industrialization and technological advancements, innovative engineering technologies which are cost effective, faster and easier to implement are essential. One such area of concern is the rising number of accidents happening due to gas leaks at coal mines, chemical industries, home appliances etc. In this paper we propose a novel approach to detect and identify the gaseous emissio…
▽ More
With the rapid industrialization and technological advancements, innovative engineering technologies which are cost effective, faster and easier to implement are essential. One such area of concern is the rising number of accidents happening due to gas leaks at coal mines, chemical industries, home appliances etc. In this paper we propose a novel approach to detect and identify the gaseous emissions using the multimodal AI fusion techniques. Most of the gases and their fumes are colorless, odorless, and tasteless, thereby challenging our normal human senses. Sensing based on a single sensor may not be accurate, and sensor fusion is essential for robust and reliable detection in several real-world applications. We manually collected 6400 gas samples (1600 samples per class for four classes) using two specific sensors: the 7-semiconductor gas sensors array, and a thermal camera. The early fusion method of multimodal AI, is applied The network architecture consists of a feature extraction module for individual modality, which is then fused using a merged layer followed by a dense layer, which provides a single output for identifying the gas. We obtained the testing accuracy of 96% (for fused model) as opposed to individual model accuracies of 82% (based on Gas Sensor data using LSTM) and 93% (based on thermal images data using CNN model). Results demonstrate that the fusion of multiple sensors and modalities outperforms the outcome of a single sensor.
△ Less
Submitted 11 June, 2021;
originally announced June 2021.
