Showing 1–8 of 8 results for author: Foti, D

Cognitive Load-based Affective Workload Allocation for Multi-human Multi-robot Teams

Authors: Wonse Jo, Ruiqi Wang, Baijian Yang, Dan Foti, Mo Rastgaar, Byung-Cheol Min

Abstract: The interaction and collaboration between humans and multiple robots represent a novel field of research known as human multi-robot systems. Adequately designed systems within this field allow teams composed of both humans and robots to work together effectively on tasks such as monitoring, exploration, and search and rescue operations. This paper presents a deep reinforcement learning-based affec… ▽ More The interaction and collaboration between humans and multiple robots represent a novel field of research known as human multi-robot systems. Adequately designed systems within this field allow teams composed of both humans and robots to work together effectively on tasks such as monitoring, exploration, and search and rescue operations. This paper presents a deep reinforcement learning-based affective workload allocation controller specifically for multi-human multi-robot teams. The proposed controller can dynamically reallocate workloads based on the performance of the operators during collaborative missions with multi-robot systems. The operators' performances are evaluated through the scores of a self-reported questionnaire (i.e., subjective measurement) and the results of a deep learning-based cognitive workload prediction algorithm that uses physiological and behavioral data (i.e., objective measurement). To evaluate the effectiveness of the proposed controller, we use a multi-human multi-robot CCTV monitoring task as an example and carry out comprehensive real-world experiments with 32 human subjects for both quantitative measurement and qualitative analysis. Our results demonstrate the performance and effectiveness of the proposed controller and highlight the importance of incorporating both subjective and objective measurements of the operators' cognitive workload as well as seeking consent for workload transitions, to enhance the performance of multi-human multi-robot teams. △ Less

Submitted 30 October, 2024; v1 submitted 18 March, 2023; originally announced March 2023.

Comments: This paper is submitted and accepted to IEEE Transactions on Human-Machine Systems

MOCAS: A Multimodal Dataset for Objective Cognitive Workload Assessment on Simultaneous Tasks

Authors: Wonse Jo, Ruiqi Wang, Su Sun, Revanth Krishna Senthilkumaran, Daniel Foti, Byung-Cheol Min

Abstract: This paper presents MOCAS, a multimodal dataset dedicated for human cognitive workload (CWL) assessment. In contrast to existing datasets based on virtual game stimuli, the data in MOCAS was collected from realistic closed-circuit television (CCTV) monitoring tasks, increasing its applicability for real-world scenarios. To build MOCAS, two off-the-shelf wearable sensors and one webcam were utilize… ▽ More This paper presents MOCAS, a multimodal dataset dedicated for human cognitive workload (CWL) assessment. In contrast to existing datasets based on virtual game stimuli, the data in MOCAS was collected from realistic closed-circuit television (CCTV) monitoring tasks, increasing its applicability for real-world scenarios. To build MOCAS, two off-the-shelf wearable sensors and one webcam were utilized to collect physiological signals and behavioral features from 21 human subjects. After each task, participants reported their CWL by completing the NASA-Task Load Index (NASA-TLX) and Instantaneous Self-Assessment (ISA). Personal background (e.g., personality and prior experience) was surveyed using demographic and Big Five Factor personality questionnaires, and two domains of subjective emotion information (i.e., arousal and valence) were obtained from the Self-Assessment Manikin (SAM), which could serve as potential indicators for improving CWL recognition performance. Technical validation was conducted to demonstrate that target CWL levels were elicited during simultaneous CCTV monitoring tasks; its results support the high quality of the collected multimodal signals. △ Less

Submitted 10 June, 2024; v1 submitted 6 October, 2022; originally announced October 2022.

Comments: 18 pages, 10 figures, 9 tables, Accepted IEEE transaction Affective Computing;

Comparison of aerosol emissions during specific speech tasks

Authors: Miriam van Mersbergen, Jeffrey Marchetta, Daniel Foti, Eric Pillow, Apartim Dasgupta, Chandler Cain, Stephen Morvant

Abstract: The study of aerosols and droplets emitted from the oral cavity has become increasingly important throughout the COVID-19 pandemic. Studies show particulates emitted while speaking were generally much smaller compared to coughing or sneezing. However, recent investigations revealed that they are large enough to carry respiratory contagions. Although studies have shown that particulate emissions do… ▽ More The study of aerosols and droplets emitted from the oral cavity has become increasingly important throughout the COVID-19 pandemic. Studies show particulates emitted while speaking were generally much smaller compared to coughing or sneezing. However, recent investigations revealed that they are large enough to carry respiratory contagions. Although studies have shown that particulate emissions do indeed occur during speech, to date, there is little information about the relative contribution of different speech sounds in producing particle emissions. This study compares airborne aerosol generation in participants producing isolated speech sounds: fricative consonants, plosive consonants, and vowel sounds. While participants produced isolated speech tasks, a planar beam of laser light, a high-speed camera, and image software calculated the number of particulates detected overtime. This study compares airborne aerosols emitted by human participants at a distance of 2.54 cm between the laser sheet and the mouth and reveals statistically significant increases in particulate counts over ambient dust distribution for all speech sounds. Vowel sounds were statistically greater than consonants, suggesting that mouth opening, as opposed to place of vocal tract constriction or manner of sound production, might be the primary influence in the degree to which particulates become aerosolized during speech. Results of this research will inform boundary conditions for computation models of aerosolized particulates during speech. △ Less

Submitted 18 May, 2022; originally announced June 2022.

Comments: 22 double spaces pages, 6 figures, 2 tables

A methodology to characterize the energy transfer and inter-scale transport of coherent structures using mode decomposition

Authors: Daniel Foti

Abstract: A methodology is developed to quantify the transfer and transport of kinetic energy of specific scales associated with coherent structures. Coherent motions are characterized by the triple decomposition of a multi-scale flow and used to define mean, coherent, and random kinetic energy. Specific scales of individual coherent structures are identified through mode decomposition, whereby the total co… ▽ More A methodology is developed to quantify the transfer and transport of kinetic energy of specific scales associated with coherent structures. Coherent motions are characterized by the triple decomposition of a multi-scale flow and used to define mean, coherent, and random kinetic energy. Specific scales of individual coherent structures are identified through mode decomposition, whereby the total coherent velocity is separated into a set of velocities classified by the scale of the mode based on frequency to embed spectral characteristics. The set of scale-specific coherent velocities are used to identify scale-specific coherent kinetic energy, which quantifies the kinetic energy of a specific scale, and an equation for the balance of scale-specific coherent kinetic energy. Each equation includes terms with velocity triads, which represent the inter-scale transfer. The methodology is assessed in the wake behind a square cylinder, where scales of the flow are related to the vortex shedding. The scale-specific inter-scale transfer identifies that kinetic energy initially in the near wake is transferred from vortex shedding to scales associated with higher harmonics of the vortex shedding. In the far wake, inter-scale transfer is homogeneous and transfer occurs over many frequencies. While the inter-scale transfer is identified over a range of scales, the scale-specific transport, convection, and dissipation of coherent kinetic energy are associated with only the largest, dominant scales. The wake flow is subjected to unsteady inflow at various frequencies to identify the changes to inter-scale transfer and coherent kinetic energy induced by other frequencies. △ Less

Submitted 15 December, 2021; originally announced December 2021.

Sub-grid scale characterization and asymptotic behavior of multi-dimensional upwind schemes for the vorticity transport equations

Authors: Daniel Foti, Karthik Duraisamy

Abstract: We study the sub-grid scale characteristics of a vorticity-transport-based approach for large-eddy simulations. In particular, we consider a multi-dimensional upwind scheme for the vorticity transport equations and establish its properties in the under-resolved regime. The asymptotic behavior of key turbulence statistics of velocity gradients, vorticity, and invariants is studied in detail. Modifi… ▽ More We study the sub-grid scale characteristics of a vorticity-transport-based approach for large-eddy simulations. In particular, we consider a multi-dimensional upwind scheme for the vorticity transport equations and establish its properties in the under-resolved regime. The asymptotic behavior of key turbulence statistics of velocity gradients, vorticity, and invariants is studied in detail. Modified equation analysis indicates that dissipation can be controlled locally via non-linear limiting of the gradient employed for the vorticity reconstruction on the cell face such that low numerical diffusion is obtained in well-resolved regimes and high numerical diffusion is realized in under-resolved regions. The enstrophy budget highlights the remarkable ability of the truncation terms to mimic the true sub-grid scale dissipation and diffusion. The modified equation also reveals diffusive terms that are similar to several commonly employed sub-grid scale models including tensor-gradient and hyper-viscosity models. Investigations on several canonical turbulence flow cases show that large-scale features are adequately represented and remain consistent in terms of spectral energy over a range of grid resolutions. Numerical dissipation in under-resolved simulations is consistent and can be characterized by diffusion terms discovered in the modified equation analysis. A minimum state of scale separation necessary to obtain asymptotic behavior is characterized using metrics such as effective Reynolds number and effective grid spacing. Temporally-evolving jet simulations, characterized by large-scale vortical structures, demonstrate that high Reynolds number vortex-dominated flows are captured when criteria is met and necessitate diffusive non-linear limiting of vorticity reconstruction be employed to realize accuracy in under-resolved simulations. △ Less

Submitted 3 February, 2021; originally announced February 2021.

Electrophysiological indicators of gesture perception

Authors: Maria E. Cabrera, Keisha Novak, Dan Foti, Richard Voyles, Juan P. Wachs

Abstract: Background: While there has been abundant research concerning neurological responses to gesture generation, the time course of gesture processing is not well understood. Specifically, it is not clear if or how particular characteristics within the kinematic execution of gestures capture attention and aid in the classification of gestures with communicative intent. If indeed key features of gesture… ▽ More Background: While there has been abundant research concerning neurological responses to gesture generation, the time course of gesture processing is not well understood. Specifically, it is not clear if or how particular characteristics within the kinematic execution of gestures capture attention and aid in the classification of gestures with communicative intent. If indeed key features of gestures with perceptual saliency exist, such features could help form the basis of a compact representation of the gestures in memory. Methods: This study used a set of available gesture videos as stimuli. The timing for salient features of performed gestures was determined by isolating inflection points in the hands' motion trajectories. Participants passively viewed the gesture videos while continuous EEG data was collected. We focused on mu oscillations (10 Hz) and used linear regression to test for associations between the timing of mu oscillations and inflection points in motion trajectories. Results: Peaks in the EEG signals at central and occipital electrodes were used to isolate the salient events within each gesture. EEG power oscillations were detected 343 and 400ms on average after inflection points at occipital and central electrodes, respectively. A regression model showed that inflection points in the motion trajectories strongly predicted subsequent mu oscillations (R^2=0.961, p<.01). Conclusion: The results suggest that coordinated activity in the visual and motor cortices are highly correlated with key motion components within gesture trajectories. These points may be associated with neural signatures used to encode gestures in memory for later identification and even recognition. △ Less

Submitted 12 November, 2018; originally announced November 2018.

Comments: 29 pages, 8 figures, 1 table

On the wake meandering of a model wind turbine operating in two different regimes

Authors: Daniel Foti, Xiaolei Yang, Filippo Campagnolo, David Maniaci, Fotis Sotiropoulos

Abstract: The flows behind a model wind turbine under two turbine operating regimes are investigated using wind tunnel experiments and large-eddy simulations. Measurements from the model wind turbine experiment reveal that the power coefficient and turbine wake are affected by the operating regime. Simulations employing a new class of actuator surface methods which parameterize both the turbine blades and n… ▽ More The flows behind a model wind turbine under two turbine operating regimes are investigated using wind tunnel experiments and large-eddy simulations. Measurements from the model wind turbine experiment reveal that the power coefficient and turbine wake are affected by the operating regime. Simulations employing a new class of actuator surface methods which parameterize both the turbine blades and nacelle with and without a nacelle model are carried out for each operating condition to study the influence of the operating regime and nacelle. For simulations with a nacelle model, the mean flow field is composed of an outer wake, caused by energy extraction from the incoming wind by the turbine blades, and an inner wake directly behind the nacelle, while for the simulations without a nacelle model, the central region of the wake is occupied by a jet. The simulations with the nacelle model reveal an unstable helical hub vortex expanding outwards towards the outer wake; while the simulations without a nacelle model show a stable and columnar hub vortex. The hub vortex for the turbine operating in Region 3 remains in a tight helical spiral and intercepts the outer wake a few diameters further downstream than for the turbine operating in Region 2. Wake meandering commences in a region of high turbulence intensity for all simulations indicating that neither a nacelle model nor an unstable hub vortex is a necessary requirement for wake meandering. However, further analysis of the wake meandering using a filtering technique and dynamic mode decomposition show that the unstable hub vortex energizes the wake meandering. The turbine operating regime affects the shape and expansion of the hub vortex altering the location of the onset of the wake meandering and its oscillating intensity. The unstable hub vortex promotes a energetic meandering which cannot be predicted without a nacelle model. △ Less

Submitted 11 February, 2018; originally announced February 2018.

What makes a gesture a gesture? Neural signatures involved in gesture recognition

Authors: Maria Cabrera, Keisha Novak, Daniel Foti, Richard Voyles, Juan Wachs

Abstract: Previous work in the area of gesture production, has made the assumption that machines can replicate "human-like" gestures by connecting a bounded set of salient points in the motion trajectory. Those inflection points were hypothesized to also display cognitive saliency. The purpose of this paper is to validate that claim using electroencephalography (EEG). That is, this paper attempts to find ne… ▽ More Previous work in the area of gesture production, has made the assumption that machines can replicate "human-like" gestures by connecting a bounded set of salient points in the motion trajectory. Those inflection points were hypothesized to also display cognitive saliency. The purpose of this paper is to validate that claim using electroencephalography (EEG). That is, this paper attempts to find neural signatures of gestures (also referred as placeholders) in human cognition, which facilitate the understanding, learning and repetition of gestures. Further, it is discussed whether there is a direct mapping between the placeholders and kinematic salient points in the gesture trajectories. These are expressed as relationships between inflection points in the gestures' trajectories with oscillatory mu rhythms (8-12 Hz) in the EEG. This is achieved by correlating fluctuations in mu power during gesture observation with salient motion points found for each gesture. Peaks in the EEG signal at central electrodes (motor cortex) and occipital electrodes (visual cortex) were used to isolate the salient events within each gesture. We found that a linear model predicting mu peaks from motion inflections fits the data well. Increases in EEG power were detected 380 and 500ms after inflection points at occipital and central electrodes, respectively. These results suggest that coordinated activity in visual and motor cortices is sensitive to motion trajectories during gesture observation, and it is consistent with the proposal that inflection points operate as placeholders in gesture recognition. △ Less

Submitted 20 January, 2017; originally announced January 2017.

Comments: This work has been submitted to a IEEE conference and is awaiting for a decision