-
AttentioNet: Monitoring Student Attention Type in Learning with EEG-Based Measurement System
Authors:
Dhruv Verma,
Sejal Bhalla,
S. V. Sai Santosh,
Saumya Yadav,
Aman Parnami,
Jainendra Shukla
Abstract:
Student attention is an indispensable input for uncovering their goals, intentions, and interests, which prove to be invaluable for a multitude of research areas, ranging from psychology to interactive systems. However, most existing methods to classify attention fail to model its complex nature. To bridge this gap, we propose AttentioNet, a novel Convolutional Neural Network-based approach that u…
▽ More
Student attention is an indispensable input for uncovering their goals, intentions, and interests, which prove to be invaluable for a multitude of research areas, ranging from psychology to interactive systems. However, most existing methods to classify attention fail to model its complex nature. To bridge this gap, we propose AttentioNet, a novel Convolutional Neural Network-based approach that utilizes Electroencephalography (EEG) data to classify attention into five states: Selective, Sustained, Divided, Alternating, and relaxed state. We collected a dataset of 20 subjects through standard neuropsychological tasks to elicit different attentional states. The average across-student accuracy of our proposed model at this configuration is 92.3% (SD=3.04), which is well-suited for end-user applications. Our transfer learning-based approach for personalizing the model to individual subjects effectively addresses the issue of individual variability in EEG signals, resulting in improved performance and adaptability of the model for real-world applications. This represents a significant advancement in the field of EEG-based classification. Experimental results demonstrate that AttentioNet outperforms a popular EEGnet baseline (p-value < 0.05) in both subject-independent and subject-dependent settings, confirming the effectiveness of our proposed approach despite the limitations of our dataset. These results highlight the promising potential of AttentioNet for attention classification using EEG data.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
An Analysis of Physiological and Psychological Responses in Virtual Reality and Flat Screen Gaming
Authors:
Ritik Vatsal,
Shrivatsa Mishra,
Rushil Thareja,
Mrinmoy Chakrabarty,
Ojaswa Sharma,
Jainendra Shukla
Abstract:
Recent research has focused on the effectiveness of Virtual Reality (VR) in games as a more immersive method of interaction. However, there is a lack of robust analysis of the physiological effects between VR and flatscreen (FS) gaming. This paper introduces the first systematic comparison and analysis of emotional and physiological responses to commercially available games in VR and FS environmen…
▽ More
Recent research has focused on the effectiveness of Virtual Reality (VR) in games as a more immersive method of interaction. However, there is a lack of robust analysis of the physiological effects between VR and flatscreen (FS) gaming. This paper introduces the first systematic comparison and analysis of emotional and physiological responses to commercially available games in VR and FS environments. To elicit these responses, we first selected four games through a pilot study of 6 participants to cover all four quadrants of the valence-arousal space. Using these games, we recorded the physiological activity, including Blood Volume Pulse and Electrodermal Activity, and self-reported emotions of 33 participants in a user study. Our data analysis revealed that VR gaming elicited more pronounced emotions, higher arousal, increased cognitive load and stress, and lower dominance than FS gaming. The Virtual Reality and Flat Screen (VRFS) dataset, containing over 15 hours of multimodal data comparing FS and VR gaming across different games, is also made publicly available for research purposes. Our analysis provides valuable insights for further investigations into the physiological and emotional effects of VR and FS gaming.
△ Less
Submitted 20 February, 2024; v1 submitted 16 June, 2023;
originally announced June 2023.
-
Exploring Semi-Supervised Learning for Predicting Listener Backchannels
Authors:
Vidit Jain,
Maitree Leekha,
Rajiv Ratn Shah,
Jainendra Shukla
Abstract:
Developing human-like conversational agents is a prime area in HCI research and subsumes many tasks. Predicting listener backchannels is one such actively-researched task. While many studies have used different approaches for backchannel prediction, they all have depended on manual annotations for a large dataset. This is a bottleneck impacting the scalability of development. To this end, we propo…
▽ More
Developing human-like conversational agents is a prime area in HCI research and subsumes many tasks. Predicting listener backchannels is one such actively-researched task. While many studies have used different approaches for backchannel prediction, they all have depended on manual annotations for a large dataset. This is a bottleneck impacting the scalability of development. To this end, we propose using semi-supervised techniques to automate the process of identifying backchannels, thereby easing the annotation process. To analyze our identification module's feasibility, we compared the backchannel prediction models trained on (a) manually-annotated and (b) semi-supervised labels. Quantitative analysis revealed that the proposed semi-supervised approach could attain 95% of the former's performance. Our user-study findings revealed that almost 60% of the participants found the backchannel responses predicted by the proposed model more natural. Finally, we also analyzed the impact of personality on the type of backchannel signals and validated our findings in the user-study.
△ Less
Submitted 6 January, 2021;
originally announced January 2021.
-
Vyaktitv: A Multimodal Peer-to-Peer Hindi Conversations based Dataset for Personality Assessment
Authors:
Shahid Nawaz Khan,
Maitree Leekha,
Jainendra Shukla,
Rajiv Ratn Shah
Abstract:
Automatically detecting personality traits can aid several applications, such as mental health recognition and human resource management. Most datasets introduced for personality detection so far have analyzed these traits for each individual in isolation. However, personality is intimately linked to our social behavior. Furthermore, surprisingly little research has focused on personality analysis…
▽ More
Automatically detecting personality traits can aid several applications, such as mental health recognition and human resource management. Most datasets introduced for personality detection so far have analyzed these traits for each individual in isolation. However, personality is intimately linked to our social behavior. Furthermore, surprisingly little research has focused on personality analysis using low resource languages. To this end, we present a novel peer-to-peer Hindi conversation dataset- Vyaktitv. It consists of high-quality audio and video recordings of the participants, with Hinglish textual transcriptions for each conversation. The dataset also contains a rich set of socio-demographic features, like income, cultural orientation, amongst several others, for all the participants. We release the dataset for public use, as well as perform preliminary statistical analysis along the different dimensions. Finally, we also discuss various other applications and tasks for which the dataset can be employed.
△ Less
Submitted 31 August, 2020;
originally announced August 2020.
-
It's LeVAsa not LevioSA! Latent Encodings for Valence-Arousal Structure Alignment
Authors:
Surabhi S. Nath,
Vishaal Udandarao,
Jainendra Shukla
Abstract:
In recent years, great strides have been made in the field of affective computing. Several models have been developed to represent and quantify emotions. Two popular ones include (i) categorical models which represent emotions as discrete labels, and (ii) dimensional models which represent emotions in a Valence-Arousal (VA) circumplex domain. However, there is no standard for annotation mapping be…
▽ More
In recent years, great strides have been made in the field of affective computing. Several models have been developed to represent and quantify emotions. Two popular ones include (i) categorical models which represent emotions as discrete labels, and (ii) dimensional models which represent emotions in a Valence-Arousal (VA) circumplex domain. However, there is no standard for annotation mapping between the two labelling methods. We build a novel algorithm for mapping categorical and dimensional model labels using annotation transfer across affective facial image datasets. Further, we utilize the transferred annotations to learn rich and interpretable data representations using a variational autoencoder (VAE). We present "LeVAsa", a VAE model that learns implicit structure by aligning the latent space with the VA space. We evaluate the efficacy of LeVAsa by comparing performance with the Vanilla VAE using quantitative and qualitative analysis on two benchmark affective image datasets. Our results reveal that LeVAsa achieves high latent-circumplex alignment which leads to improved downstream categorical emotion prediction. The work also demonstrates the trade-off between degree of alignment and quality of reconstructions.
△ Less
Submitted 30 November, 2020; v1 submitted 20 July, 2020;
originally announced July 2020.
-
Proceedings of the Workshop on Social Robots in Therapy: Focusing on Autonomy and Ethical Challenges
Authors:
Pablo G. Esteban,
Daniel Hernández García,
Hee Rin Lee,
Pauline Chevalier,
Paul Baxter,
Cindy L. Bethel,
Jainendra Shukla,
Joan Oliver,
Domènec Puig,
Jason R. Wilson,
Linda Tickle-Degnen,
Madeleine Bartlett,
Tony Belpaeme,
Serge Thill,
Kim Baraka,
Francisco S. Melo,
Manuela Veloso,
David Becerra,
Maja Matarić,
Eduard Fosch-Villaronga,
Jordi Albo-Canals,
Gloria Beraldo,
Emanuele Menegatti,
Valentina De Tommasi,
Roberto Mancin
, et al. (13 additional authors not shown)
Abstract:
Robot-Assisted Therapy (RAT) has successfully been used in HRI research by including social robots in health-care interventions by virtue of their ability to engage human users both social and emotional dimensions. Research projects on this topic exist all over the globe in the USA, Europe, and Asia. All of these projects have the overall ambitious goal to increase the well-being of a vulnerable p…
▽ More
Robot-Assisted Therapy (RAT) has successfully been used in HRI research by including social robots in health-care interventions by virtue of their ability to engage human users both social and emotional dimensions. Research projects on this topic exist all over the globe in the USA, Europe, and Asia. All of these projects have the overall ambitious goal to increase the well-being of a vulnerable population. Typical work in RAT is performed using remote controlled robots; a technique called Wizard-of-Oz (WoZ). The robot is usually controlled, unbeknownst to the patient, by a human operator. However, WoZ has been demonstrated to not be a sustainable technique in the long-term. Providing the robots with autonomy (while remaining under the supervision of the therapist) has the potential to lighten the therapists burden, not only in the therapeutic session itself but also in longer-term diagnostic tasks. Therefore, there is a need for exploring several degrees of autonomy in social robots used in therapy. Increasing the autonomy of robots might also bring about a new set of challenges. In particular, there will be a need to answer new ethical questions regarding the use of robots with a vulnerable population, as well as a need to ensure ethically-compliant robot behaviours. Therefore, in this workshop we want to gather findings and explore which degree of autonomy might help to improve health-care interventions and how we can overcome the ethical challenges inherent to it.
△ Less
Submitted 18 December, 2018;
originally announced December 2018.
-
A Survey for New Members of the Taurus Star-Forming Region with the Sloan Digital Sky Survey
Authors:
K. L. Luhman,
E. E. Mamajek,
S. J. Shukla,
N. P. Loutrel
Abstract:
Previous studies have found that ~1 deg2 fields surrounding the stellar aggregates in the Taurus star-forming region exhibit a surplus of solar-mass stars relative to denser clusters like IC~348 and the Orion Nebula Cluster. To test whether this difference reflects mass segregation in Taurus or a variation in the IMF, we have performed a survey for members of Taurus across a large field (~40 deg2)…
▽ More
Previous studies have found that ~1 deg2 fields surrounding the stellar aggregates in the Taurus star-forming region exhibit a surplus of solar-mass stars relative to denser clusters like IC~348 and the Orion Nebula Cluster. To test whether this difference reflects mass segregation in Taurus or a variation in the IMF, we have performed a survey for members of Taurus across a large field (~40 deg2) that was imaged by the Sloan Digital Sky Survey (SDSS). We obtained optical and near-infrared spectra of candidate members identified with those images and the Two Micron All Sky Survey, as well as miscellaneous candidates that were selected with several other diagnostics of membership. We have classified 22 of the candidates as new members of Taurus, which includes one of the coolest known members (M9.75). Our updated census of members within the SDSS field shows a surplus of solar-mass stars relative to clusters, although it is less pronounced than in the smaller fields towards the stellar aggregates that were surveyed for previously measured mass functions in Taurus. In addition to spectra of our new members, we include in our study near-IR spectra of roughly half of the known members of Taurus, which are used to refine their spectral types and extinctions. We also present an updated set of near-IR standard spectra for classifying young stars and brown dwarfs at M and L types.
△ Less
Submitted 3 November, 2016; v1 submitted 28 October, 2016;
originally announced October 2016.
-
Spatially-resolved Chandra Imaging Spectroscopy of the classical/weak-lined T Tauri system V710 Tau
Authors:
Sonali J. Shukla,
David A. Weintraub,
Joel H. Kastner
Abstract:
We present spatially-resolved X-ray observations of the binary T Tauri star system V710 Tau. Using Chandra's Advanced CCD Imaging Spectrometer (ACIS), we imaged this 3.2'' separation binary system, consisting of a classical T Tauri star, V710 Tau N, and a weak-lined T Tauri star, V710 Tau S. The Chandra ACIS-S3 images -- obtained in two 9 ks exposures separated by about three months (2004 Decemb…
▽ More
We present spatially-resolved X-ray observations of the binary T Tauri star system V710 Tau. Using Chandra's Advanced CCD Imaging Spectrometer (ACIS), we imaged this 3.2'' separation binary system, consisting of a classical T Tauri star, V710 Tau N, and a weak-lined T Tauri star, V710 Tau S. The Chandra ACIS-S3 images -- obtained in two 9 ks exposures separated by about three months (2004 December and 2005 April) -- cleanly resolve the V710 Tau binary, demonstrating that both stars emit X-rays and thereby enabling the first spectral/temporal study of the individual components of this mixed (classical and weak-lined) T Tauri star binary system. The northern component, V710 Tau N, appears to have been in a flaring state during the first (2004 December) exposure. During this flare event, the X-ray flux of the classical T Tauri star hardened significantly. Single-component plasma models with plasma temperatures in the range kT ~ 0.7-1.1 keV are adequate to fit the observed X-ray spectra of V710 Tau S in 2004 December and both stars in 2005 April. The 2004 December flare-state observation of V710 Tau N requires a higher-temperature plasma component (kT ~ 2.5 - 3.0 keV) in addition to the soft component (kT ~ 0.5 keV) and is better fit by a model that includes a slightly enhanced Ne/Fe abundance ratio. These results are generally consistent with statistical contrasts between the X-ray emission properties of classical (rapidly accreting) vs. weak-lined (weakly accreting or non-accreting) T Tauri stars.
△ Less
Submitted 8 May, 2008;
originally announced May 2008.
-
Quiescent H2 Emission From Pre-Main Sequence Stars in Chamaeleon I
Authors:
Jeffrey S. Bary,
David A. Weintraub,
Sonali J. Shukla,
Jarron M. Leisenring,
Joel H. Kastner
Abstract:
We report the discovery of quiescent emission from molecular hydrogen gas located in the circumstellar disks of six pre-main sequence stars, including two weak-line T Tauri stars (TTS), and one Herbig AeBe star, in the Chamaeleon I star forming region. For two of these stars, we also place upper limits on the 2->1 S(1)/1->0 S(1) line ratios of 0.4 and 0.5. Of the 11 pre-main sequence sources now…
▽ More
We report the discovery of quiescent emission from molecular hydrogen gas located in the circumstellar disks of six pre-main sequence stars, including two weak-line T Tauri stars (TTS), and one Herbig AeBe star, in the Chamaeleon I star forming region. For two of these stars, we also place upper limits on the 2->1 S(1)/1->0 S(1) line ratios of 0.4 and 0.5. Of the 11 pre-main sequence sources now known to be sources of quiescent near-infrared hydrogen emission, four possess transitional disks, which suggests that detectable levels of H$_2$ emission and the presence of inner disk holes are correlated. These H$_2$ detections demonstrate that these inner holes are not completely devoid of gas, in agreement with the presence of observable accretion signatures for all four of these stars and the recent detections of [Ne II] emission from three of them. The overlap in [Ne II] and H$_2$ detections hints at a possible correlation between these two features and suggests a shared excitation mechanism of high energy photons. Our models, combined with the kinematic information from the H$_2$ lines, locate the bulk of the emitting gas at a few tens of AU from the stars. We also find a correlation between H$_2$ detections and those targets which possess the largest H$α$ equivalent widths, suggesting a link between accretion activity and quiescent H$_2$ emission. We conclude that quiescent H$_2$ emission from relatively hot gas within the disks of TTS is most likely related to on-going accretion activity, the production of UV photons and/or X-rays, and the evolutionary status of the dust grain populations in the inner disks.
△ Less
Submitted 17 January, 2008;
originally announced January 2008.
