-
PainNet: Statistical Relation Network with Episode-Based Training for Pain Estimation
Authors:
Mina Bishay,
Graham Page,
Mohammad Mavadati
Abstract:
Despite the span in estimating pain from facial expressions, limited works have focused on estimating the sequence-level pain, which is reported by patients and used commonly in clinics. In this paper, we introduce a novel Statistical Relation Network, referred to as PainNet, designed for the estimation of the sequence-level pain. PainNet employs two key modules, the embedding and the relation mod…
▽ More
Despite the span in estimating pain from facial expressions, limited works have focused on estimating the sequence-level pain, which is reported by patients and used commonly in clinics. In this paper, we introduce a novel Statistical Relation Network, referred to as PainNet, designed for the estimation of the sequence-level pain. PainNet employs two key modules, the embedding and the relation modules, for comparing pairs of pain videos, and producing relation scores indicating if each pair belongs to the same pain category or not. At the core of the embedding module is a statistical layer mounted on the top of a RNN for extracting compact video-level features. The statistical layer is implemented as part of the deep architecture. Doing so, allows combining multiple training stages used in previous research, into a single end-to-end training stage. PainNet is trained using the episode-based training scheme, which involves comparing a query video with a set of videos representing the different pain categories. Experimental results show the benefit of using the statistical layer and the episode-based training in the proposed model. Furthermore, PainNet outperforms the state-of-the-art results on self-reported pain estimation.
△ Less
Submitted 8 April, 2025;
originally announced April 2025.
-
Monitoring Viewer Attention During Online Ads
Authors:
Mina Bishay,
Graham Page,
Waleed Emad,
Mohammad Mavadati
Abstract:
Nowadays, video ads spread through numerous online platforms, and are being watched by millions of viewers worldwide. Big brands gauge the liking and purchase intent of their new ads, by analyzing the facial responses of viewers recruited online to watch the ads from home or work. Although this approach captures naturalistic responses, it is susceptible to distractions inherent in the participants…
▽ More
Nowadays, video ads spread through numerous online platforms, and are being watched by millions of viewers worldwide. Big brands gauge the liking and purchase intent of their new ads, by analyzing the facial responses of viewers recruited online to watch the ads from home or work. Although this approach captures naturalistic responses, it is susceptible to distractions inherent in the participants' environments, such as a movie playing on TV, a colleague speaking, or mobile notifications. Inattentive participants should get flagged and eliminated to avoid skewing the ad-testing process. In this paper we introduce an architecture for monitoring viewer attention during online ads. Leveraging two behavior analysis toolkits; AFFDEX 2.0 and SmartEye SDK, we extract low-level facial features encompassing facial expressions, head pose, and gaze direction. These features are then combined to extract high-level features that include estimated gaze on the screen plane, yawning, speaking, etc -- this enables the identification of four primary distractors; off-screen gaze, drowsiness, speaking, and unattended screen. Our architecture tailors the gaze settings according to the device type (desktop or mobile). We validate our architecture first on datasets annotated for specific distractors, and then on a real-world ad testing dataset with various distractors. The proposed architecture shows promising results in detecting distraction across both desktop and mobile devices.
△ Less
Submitted 8 April, 2025;
originally announced April 2025.
-
WindsorML: High-Fidelity Computational Fluid Dynamics Dataset For Automotive Aerodynamics
Authors:
Neil Ashton,
Jordan B. Angel,
Aditya S. Ghate,
Gaetan K. W. Kenway,
Man Long Wong,
Cetin Kiris,
Astrid Walle,
Danielle C. Maddix,
Gary Page
Abstract:
This paper presents a new open-source high-fidelity dataset for Machine Learning (ML) containing 355 geometric variants of the Windsor body, to help the development and testing of ML surrogate models for external automotive aerodynamics. Each Computational Fluid Dynamics (CFD) simulation was run with a GPU-native high-fidelity Wall-Modeled Large-Eddy Simulations (WMLES) using a Cartesian immersed-…
▽ More
This paper presents a new open-source high-fidelity dataset for Machine Learning (ML) containing 355 geometric variants of the Windsor body, to help the development and testing of ML surrogate models for external automotive aerodynamics. Each Computational Fluid Dynamics (CFD) simulation was run with a GPU-native high-fidelity Wall-Modeled Large-Eddy Simulations (WMLES) using a Cartesian immersed-boundary method using more than 280M cells to ensure the greatest possible accuracy. The dataset contains geometry variants that exhibits a wide range of flow characteristics that are representative of those observed on road-cars. The dataset itself contains the 3D time-averaged volume & boundary data as well as the geometry and force & moment coefficients. This paper discusses the validation of the underlying CFD methods as well as contents and structure of the dataset. To the authors knowledge, this represents the first, large-scale high-fidelity CFD dataset for the Windsor body with a permissive open-source license (CC-BY-SA).
△ Less
Submitted 16 January, 2025; v1 submitted 27 July, 2024;
originally announced July 2024.
-
"It's Sink or Swim'': Exploring Patients' Challenges and Tool Needs for Self-Management of Postoperative Acute Pain
Authors:
Souleima Zghab,
Gabrielle Pagé,
Mélanie Lussier,
Sylvain Bédard,
Jinghui Cheng
Abstract:
Poorly managed postoperative acute pain can have long-lasting negative impacts and pose a major healthcare issue. There is limited investigation to understand and address the unique needs of patients experiencing acute pain. In this paper, we tackle this gap through an interview study with 14 patients who recently underwent postoperative acute pain to understand their challenges in pain self-manag…
▽ More
Poorly managed postoperative acute pain can have long-lasting negative impacts and pose a major healthcare issue. There is limited investigation to understand and address the unique needs of patients experiencing acute pain. In this paper, we tackle this gap through an interview study with 14 patients who recently underwent postoperative acute pain to understand their challenges in pain self-management and their need for supportive tools. Our analysis identified various factors associated with the major aspects of acute pain self-management. Together, our findings indicated that tools for supporting these patients need to carefully consider information and support delivery to adapt to rapid changes in pain experiences, offer personalized and dynamic assistance that adapts to individual situations in context, and monitor emotion when promoting motivation. Overall, our work provided valuable knowledge to address the less-investigated but highly-needed problem of designing technology for the self-management of acute pain and similar health conditions.
△ Less
Submitted 26 January, 2024;
originally announced January 2024.
-
Automatic Detection of Sentimentality from Facial Expressions
Authors:
Mina Bishay,
Jay Turcot,
Graham Page,
Mohammad Mavadati
Abstract:
Emotion recognition has received considerable attention from the Computer Vision community in the last 20 years. However, most of the research focused on analyzing the six basic emotions (e.g. joy, anger, surprise), with a limited work directed to other affective states. In this paper, we tackle sentimentality (strong feeling of heartwarming or nostalgia), a new emotional state that has few works…
▽ More
Emotion recognition has received considerable attention from the Computer Vision community in the last 20 years. However, most of the research focused on analyzing the six basic emotions (e.g. joy, anger, surprise), with a limited work directed to other affective states. In this paper, we tackle sentimentality (strong feeling of heartwarming or nostalgia), a new emotional state that has few works in the literature, and no guideline defining its facial markers. To this end, we first collect a dataset of 4.9K videos of participants watching some sentimental and non-sentimental ads, and then we label the moments evoking sentimentality in the ads. Second, we use the ad-level labels and the facial Action Units (AUs) activation across different frames for defining some weak frame-level sentimentality labels. Third, we train a Multilayer Perceptron (MLP) using the AUs activation for sentimentality detection. Finally, we define two new ad-level metrics for evaluating our model performance. Quantitative and qualitative results show promising results for sentimentality detection. To the best of our knowledge this is the first work to address the problem of sentimentality detection.
△ Less
Submitted 11 September, 2022;
originally announced September 2022.
-
AFFDEX 2.0: A Real-Time Facial Expression Analysis Toolkit
Authors:
Mina Bishay,
Kenneth Preston,
Matthew Strafuss,
Graham Page,
Jay Turcot,
Mohammad Mavadati
Abstract:
In this paper we introduce AFFDEX 2.0 - a toolkit for analyzing facial expressions in the wild, that is, it is intended for users aiming to; a) estimate the 3D head pose, b) detect facial Action Units (AUs), c) recognize basic emotions and 2 new emotional states (sentimentality and confusion), and d) detect high-level expressive metrics like blink and attention. AFFDEX 2.0 models are mainly based…
▽ More
In this paper we introduce AFFDEX 2.0 - a toolkit for analyzing facial expressions in the wild, that is, it is intended for users aiming to; a) estimate the 3D head pose, b) detect facial Action Units (AUs), c) recognize basic emotions and 2 new emotional states (sentimentality and confusion), and d) detect high-level expressive metrics like blink and attention. AFFDEX 2.0 models are mainly based on Deep Learning, and are trained using a large-scale naturalistic dataset consisting of thousands of participants from different demographic groups. AFFDEX 2.0 is an enhanced version of our previous toolkit [1], that is capable of tracking efficiently faces at more challenging conditions, detecting more accurately facial expressions, and recognizing new emotional states (sentimentality and confusion). AFFDEX 2.0 can process multiple faces in real time, and is working across the Windows and Linux platforms.
△ Less
Submitted 2 November, 2022; v1 submitted 24 February, 2022;
originally announced February 2022.
