-
EEG-BBNet: a Hybrid Framework for Brain Biometric using Graph Connectivity
Authors:
Payongkit Lakhan,
Nannapas Banluesombatkul,
Natchaya Sricom,
Korn Surapat,
Ratha Rotruchiphong,
Phattarapong Sawangjai,
Tohru Yagi,
Tulaya Limpiti,
Theerawit Wilaiprasitporn
Abstract:
Brain biometrics based on electroencephalography (EEG) have been used increasingly for personal identification. Traditional machine learning techniques as well as modern day deep learning methods have been applied with promising results. In this paper we present EEG-BBNet, a hybrid network which integrates convolutional neural networks (CNN) with graph convolutional neural networks (GCNN). The ben…
▽ More
Brain biometrics based on electroencephalography (EEG) have been used increasingly for personal identification. Traditional machine learning techniques as well as modern day deep learning methods have been applied with promising results. In this paper we present EEG-BBNet, a hybrid network which integrates convolutional neural networks (CNN) with graph convolutional neural networks (GCNN). The benefit of the CNN in automatic feature extraction and the capability of GCNN in learning connectivity between EEG electrodes through graph representation are jointly exploited. We examine various connectivity measures, namely the Euclidean distance, Pearson's correlation coefficient, phase-locked value, phase-lag index, and Rho index. The performance of the proposed method is assessed on a benchmark dataset consisting of various brain-computer interface (BCI) tasks and compared to other state-of-the-art approaches. We found that our models outperform all baselines in the event-related potential (ERP) task with an average correct recognition rates up to 99.26% using intra-session data. EEG-BBNet with Pearson's correlation and RHO index provide the best classification results. In addition, our model demonstrates greater adaptability using inter-session and inter-task data. We also investigate the practicality of our proposed model with smaller number of electrodes. Electrode placements over the frontal lobe region appears to be most appropriate with minimal lost in performance.
△ Less
Submitted 17 August, 2022;
originally announced August 2022.
-
MetaSleepLearner: A Pilot Study on Fast Adaptation of Bio-signals-Based Sleep Stage Classifier to New Individual Subject Using Meta-Learning
Authors:
Nannapas Banluesombatkul,
Pichayoot Ouppaphan,
Pitshaporn Leelaarporn,
Payongkit Lakhan,
Busarakum Chaitusaney,
Nattapong Jaimchariyatam,
Ekapol Chuangsuwanich,
Wei Chen,
Huy Phan,
Nat Dilokthanakul,
Theerawit Wilaiprasitporn
Abstract:
Identifying bio-signals based-sleep stages requires time-consuming and tedious labor of skilled clinicians. Deep learning approaches have been introduced in order to challenge the automatic sleep stage classification conundrum. However, the difficulties can be posed in replacing the clinicians with the automatic system due to the differences in many aspects found in individual bio-signals, causing…
▽ More
Identifying bio-signals based-sleep stages requires time-consuming and tedious labor of skilled clinicians. Deep learning approaches have been introduced in order to challenge the automatic sleep stage classification conundrum. However, the difficulties can be posed in replacing the clinicians with the automatic system due to the differences in many aspects found in individual bio-signals, causing the inconsistency in the performance of the model on every incoming individual. Thus, we aim to explore the feasibility of using a novel approach, capable of assisting the clinicians and lessening the workload. We propose the transfer learning framework, entitled MetaSleepLearner, based on Model Agnostic Meta-Learning (MAML), in order to transfer the acquired sleep staging knowledge from a large dataset to new individual subjects. The framework was demonstrated to require the labelling of only a few sleep epochs by the clinicians and allow the remainder to be handled by the system. Layer-wise Relevance Propagation (LRP) was also applied to understand the learning course of our approach. In all acquired datasets, in comparison to the conventional approach, MetaSleepLearner achieved a range of 5.4\% to 17.7\% improvement with statistical difference in the mean of both approaches. The illustration of the model interpretation after the adaptation to each subject also confirmed that the performance was directed towards reasonable learning. MetaSleepLearner outperformed the conventional approaches as a result from the fine-tuning using the recordings of both healthy subjects and patients. This is the first work that investigated a non-conventional pre-training method, MAML, resulting in a possibility for human-machine collaboration in sleep stage classification and easing the burden of the clinicians in labelling the sleep stages through only several epochs rather than an entire recording.
△ Less
Submitted 10 November, 2020; v1 submitted 8 April, 2020;
originally announced April 2020.
-
Consumer Grade Brain Sensing for Emotion Recognition
Authors:
Payongkit Lakhan,
Nannapas Banluesombatkul,
Vongsagon Changniam,
Ratwade Dhithijaiyratn,
Pitshaporn Leelaarporn,
Ekkarat Boonchieng,
Supanida Hompoonsup,
Theerawit Wilaiprasitporn
Abstract:
For several decades, electroencephalography (EEG) has featured as one of the most commonly used tools in emotional state recognition via monitoring of distinctive brain activities. An array of datasets have been generated with the use of diverse emotion-eliciting stimuli and the resulting brainwave responses conventionally captured with high-end EEG devices. However, the applicability of these dev…
▽ More
For several decades, electroencephalography (EEG) has featured as one of the most commonly used tools in emotional state recognition via monitoring of distinctive brain activities. An array of datasets have been generated with the use of diverse emotion-eliciting stimuli and the resulting brainwave responses conventionally captured with high-end EEG devices. However, the applicability of these devices is to some extent limited by practical constraints and may prove difficult to be deployed in highly mobile context omnipresent in everyday happenings. In this study, we evaluate the potential of OpenBCI to bridge this gap by first comparing its performance to research grade EEG system, employing the same algorithms that were applied on benchmark datasets. Moreover, for the purpose of emotion classification, we propose a novel method to facilitate the selection of audio-visual stimuli of high/low valence and arousal. Our setup entailed recruiting 200 healthy volunteers of varying years of age to identify the top 60 affective video clips from a total of 120 candidates through standardized self assessment, genre tags, and unsupervised machine learning. Additional 43 participants were enrolled to watch the pre-selected clips during which emotional EEG brainwaves and peripheral physiological signals were collected. These recordings were analyzed and extracted features fed into a classification model to predict whether the elicited signals were associated with a high or low level of valence and arousal. As it turned out, our prediction accuracies were decidedly comparable to those of previous studies that utilized more costly EEG amplifiers for data acquisition.
△ Less
Submitted 9 August, 2019; v1 submitted 10 October, 2018;
originally announced October 2018.
-
Deep Neural Networks with Weighted Averaged Overnight Airflow Features for Sleep Apnea-Hypopnea Severity Classification
Authors:
Payongkit Lakhan,
Apiwat Ditthapron,
Nannapas Banluesombatkul,
Theerawit Wilaiprasitporn
Abstract:
Dramatic raising of Deep Learning (DL) approach and its capability in biomedical applications lead us to explore the advantages of using DL for sleep Apnea-Hypopnea severity classification. To reduce the complexity of clinical diagnosis using Polysomnography (PSG), which is multiple sensing platform, we incorporates our proposed DL scheme into one single Airflow (AF) sensing signal (subset of PSG)…
▽ More
Dramatic raising of Deep Learning (DL) approach and its capability in biomedical applications lead us to explore the advantages of using DL for sleep Apnea-Hypopnea severity classification. To reduce the complexity of clinical diagnosis using Polysomnography (PSG), which is multiple sensing platform, we incorporates our proposed DL scheme into one single Airflow (AF) sensing signal (subset of PSG). Seventeen features have been extracted from AF and then fed into Deep Neural Networks to classify in two studies. First, we proposed a binary classifications which use the cutoff indices at AHI = 5, 15 and 30 events/hour. Second, the multiple Sleep Apnea-Hypopnea Syndrome (SAHS) severity classification was proposed to classify patients into 4 groups including no SAHS, mild SAHS, moderate SAHS, and severe SAHS. For methods evaluation, we used a higher number of patients than related works to accommodate more diversity which includes 520 AF records obtained from the MrOS sleep study (Visit 2) database. We then applied the 10-fold cross-validation technique to get the accuracy, sensitivity and specificity. Moreover, we compared the results from our main classifier with other two approaches which were used in previous researches including the Support Vector Machine (SVM) and the Adaboost-Classification and Regression Trees (AB-CART). From the binary classification, our proposed method provides significantly higher performance than other two approaches with the accuracy of 83.46 %, 85.39 % and 92.69 % in each cutoff, respectively. For the multiclass classification, it also returns a highest accuracy of all approaches with 63.70 %.
△ Less
Submitted 31 August, 2018;
originally announced August 2018.
-
Single Channel ECG for Obstructive Sleep Apnea Severity Detection using a Deep Learning Approach
Authors:
Nannapas Banluesombatkul,
Thanawin Rakthanmanon,
Theerawit Wilaiprasitporn
Abstract:
Obstructive sleep apnea (OSA) is a common sleep disorder caused by abnormal breathing. The severity of OSA can lead to many symptoms such as sudden cardiac death (SCD). Polysomnography (PSG) is a gold standard for OSA diagnosis. It records many signals from the patient's body for at least one whole night and calculates the Apnea-Hypopnea Index (AHI) which is the number of apnea or hypopnea inciden…
▽ More
Obstructive sleep apnea (OSA) is a common sleep disorder caused by abnormal breathing. The severity of OSA can lead to many symptoms such as sudden cardiac death (SCD). Polysomnography (PSG) is a gold standard for OSA diagnosis. It records many signals from the patient's body for at least one whole night and calculates the Apnea-Hypopnea Index (AHI) which is the number of apnea or hypopnea incidences per hour. This value is then used to classify patients into OSA severity levels. However, it has many disadvantages and limitations. Consequently, we proposed a novel methodology of OSA severity classification using a Deep Learning approach. We focused on the classification between normal subjects (AHI $<$ 5) and severe OSA patients (AHI $>$ 30). The 15-second raw ECG records with apnea or hypopnea events were used with a series of deep learning models. The main advantages of our proposed method include easier data acquisition, instantaneous OSA severity detection, and effective feature extraction without domain knowledge from expertise. To evaluate our proposed method, 545 subjects of which 364 were normal and 181 were severe OSA patients obtained from the MrOS sleep study (Visit 1) database were used with the k-fold cross-validation technique. The accuracy of 79.45\% for OSA severity classification with sensitivity, specificity, and F-score was achieved. This is significantly higher than the results from the SVM classifier with RR Intervals and ECG derived respiration (EDR) signal feature extraction. The promising result shows that this proposed method is a good start for the detection of OSA severity from a single channel ECG which can be obtained from wearable devices at home and can also be applied to near real-time alerting systems such as before SCD occurs.
△ Less
Submitted 31 August, 2018;
originally announced August 2018.
-
Universal Joint Feature Extraction for P300 EEG Classification using Multi-task Autoencoder
Authors:
Apiwat Ditthapron,
Nannapas Banluesombatkul,
Sombat Ketrat,
Ekapol Chuangsuwanich,
Theerawit Wilaiprasitporn
Abstract:
The process of recording Electroencephalography (EEG) signals is onerous and requires massive storage to store signals at an applicable frequency rate. In this work, we propose the EventRelated Potential Encoder Network (ERPENet); a multi-task autoencoder-based model, that can be applied to any ERP-related tasks. The strength of ERPENet lies in its capability to handle various kinds of ERP dataset…
▽ More
The process of recording Electroencephalography (EEG) signals is onerous and requires massive storage to store signals at an applicable frequency rate. In this work, we propose the EventRelated Potential Encoder Network (ERPENet); a multi-task autoencoder-based model, that can be applied to any ERP-related tasks. The strength of ERPENet lies in its capability to handle various kinds of ERP datasets and its robustness across multiple recording setups, enabling joint training across datasets. ERPENet incorporates Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM), in an autoencoder setup, which tries to simultaneously compress the input EEG signal and extract related P300 features into a latent vector. Here, we can infer the process for generating the latent vector as universal joint feature extraction. The network also includes a classification part for attended and unattended events classification as an auxiliary task. We experimented on six different P300 datasets. The results show that the latent vector exhibits better compression capability than the previous state-of-the-art semi-supervised autoencoder model. For attended and unattended events classification, pre-trained weights are adopted as initial weights and tested on unseen P300 datasets to evaluate the adaptability of the model, which shortens the training process as compared to using random Xavier weight initialization. At the compression rate of 6.84, the classification accuracy outperforms conventional P300 classification models: XdawnLDA, DeepConvNet, and EEGNet achieving 79.37% - 88.52% classification accuracy depending on the dataset.
△ Less
Submitted 30 April, 2019; v1 submitted 30 July, 2018;
originally announced August 2018.
-
Affective EEG-Based Person Identification Using the Deep Learning Approach
Authors:
Theerawit Wilaiprasitporn,
Apiwat Ditthapron,
Karis Matchaparn,
Tanaboon Tongbuasirilai,
Nannapas Banluesombatkul,
Ekapol Chuangsuwanich
Abstract:
Electroencephalography (EEG) is another mode for performing Person Identification (PI). Due to the nature of the EEG signals, EEG-based PI is typically done while the person is performing some kind of mental task, such as motor control. However, few works have considered EEG-based PI while the person is in different mental states (affective EEG). The aim of this paper is to improve the performance…
▽ More
Electroencephalography (EEG) is another mode for performing Person Identification (PI). Due to the nature of the EEG signals, EEG-based PI is typically done while the person is performing some kind of mental task, such as motor control. However, few works have considered EEG-based PI while the person is in different mental states (affective EEG). The aim of this paper is to improve the performance of affective EEG-based PI using a deep learning approach. \textcolor{red}{We proposed a cascade of deep learning using a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs)}. CNNs are used to handle the spatial information from the EEG while RNNs extract the temporal information. \textcolor{red}{We evaluated two types of RNNs, namely, Long Short-Term Memory (CNN-LSTM) and Gated Recurrent Unit (CNN-GRU). } The proposed method is evaluated on the state-of-the-art affective dataset DEAP. The results indicate that CNN-GRU and CNN-LSTM can perform PI from different affective states and reach up to 99.90--100\% mean Correct Recognition Rate (CRR), significantly outperforming a support vector machine (SVM) baseline system that uses power spectral density (PSD) features. Notably, the 100\% mean \emph{CRR} comes from only 40 subjects in DEAP dataset. To reduce the number of EEG electrodes from thirty-two to five for more practical applications, the frontal region gives the best results reaching up to 99.17\% CRR (from CNN-GRU). Amongst the two deep learning models, we find CNN-GRU to slightly outperform CNN-LSTM, while having faster training time. \textcolor{red}{Furthermore, CNN-GRU overcomes the influence of affective states in EEG-Based PI reported in the previous works.
△ Less
Submitted 29 April, 2019; v1 submitted 5 July, 2018;
originally announced July 2018.
