-
FIESTA: Fisher Information-based Efficient Selective Test-time Adaptation
Authors:
Mohammadmahdi Honarmand,
Onur Cezmi Mutlu,
Parnian Azizian,
Saimourya Surabhi,
Dennis P. Wall
Abstract:
Robust facial expression recognition in unconstrained, "in-the-wild" environments remains challenging due to significant domain shifts between training and testing distributions. Test-time adaptation (TTA) offers a promising solution by adapting pre-trained models during inference without requiring labeled test data. However, existing TTA approaches typically rely on manually selecting which param…
▽ More
Robust facial expression recognition in unconstrained, "in-the-wild" environments remains challenging due to significant domain shifts between training and testing distributions. Test-time adaptation (TTA) offers a promising solution by adapting pre-trained models during inference without requiring labeled test data. However, existing TTA approaches typically rely on manually selecting which parameters to update, potentially leading to suboptimal adaptation and high computational costs. This paper introduces a novel Fisher-driven selective adaptation framework that dynamically identifies and updates only the most critical model parameters based on their importance as quantified by Fisher information. By integrating this principled parameter selection approach with temporal consistency constraints, our method enables efficient and effective adaptation specifically tailored for video-based facial expression recognition. Experiments on the challenging AffWild2 benchmark demonstrate that our approach significantly outperforms existing TTA methods, achieving a 7.7% improvement in F1 score over the base model while adapting only 22,000 parameters-more than 20 times fewer than comparable methods. Our ablation studies further reveal that parameter importance can be effectively estimated from minimal data, with sampling just 1-3 frames sufficient for substantial performance gains. The proposed approach not only enhances recognition accuracy but also dramatically reduces computational overhead, making test-time adaptation more practical for real-world affective computing applications.
△ Less
Submitted 29 March, 2025;
originally announced March 2025.
-
Challenges in the Differential Classification of Individual Diagnoses from Co-Occurring Autism and ADHD Using Survey Data
Authors:
Aditi Jaiswal,
Dennis P. Wall,
Peter Washington
Abstract:
Autism and Attention-Deficit Hyperactivity Disorder (ADHD) are two of the most commonly observed neurodevelopmental conditions in childhood. Providing a specific computational assessment to distinguish between the two can prove difficult and time intensive. Given the high prevalence of their co-occurrence, there is a need for scalable and accessible methods for distinguishing the co-occurrence of…
▽ More
Autism and Attention-Deficit Hyperactivity Disorder (ADHD) are two of the most commonly observed neurodevelopmental conditions in childhood. Providing a specific computational assessment to distinguish between the two can prove difficult and time intensive. Given the high prevalence of their co-occurrence, there is a need for scalable and accessible methods for distinguishing the co-occurrence of autism and ADHD from individual diagnoses. The first step is to identify a core set of features that can serve as the basis for behavioral feature extraction. We trained machine learning models on data from the National Survey of Children's Health to identify behaviors to target as features in automated clinical decision support systems. A model trained on the binary task of distinguishing either developmental delay (autism or ADHD) vs. neither achieved sensitivity >92% and specificity >94%, while a model trained on the 4-way classification task of autism vs. ADHD vs. both vs. none demonstrated >65% sensitivity and >66% specificity. While the performance of the binary model was respectable, the relatively low performance in the differential classification of autism and ADHD highlights the challenges that persist in achieving specificity within clinical decision support tools for developmental delays. Nevertheless, this study demonstrates the potential of applying behavioral questionnaires not traditionally used for clinical purposes towards supporting digital screening assessments for pediatric developmental delays.
△ Less
Submitted 12 November, 2024;
originally announced November 2024.
-
Ensemble Modeling of Multiple Physical Indicators to Dynamically Phenotype Autism Spectrum Disorder
Authors:
Marie Huynh,
Aaron Kline,
Saimourya Surabhi,
Kaitlyn Dunlap,
Onur Cezmi Mutlu,
Mohammadmahdi Honarmand,
Parnian Azizian,
Peter Washington,
Dennis P. Wall
Abstract:
Early detection of autism, a neurodevelopmental disorder marked by social communication challenges, is crucial for timely intervention. Recent advancements have utilized naturalistic home videos captured via the mobile application GuessWhat. Through interactive games played between children and their guardians, GuessWhat has amassed over 3,000 structured videos from 382 children, both diagnosed wi…
▽ More
Early detection of autism, a neurodevelopmental disorder marked by social communication challenges, is crucial for timely intervention. Recent advancements have utilized naturalistic home videos captured via the mobile application GuessWhat. Through interactive games played between children and their guardians, GuessWhat has amassed over 3,000 structured videos from 382 children, both diagnosed with and without Autism Spectrum Disorder (ASD). This collection provides a robust dataset for training computer vision models to detect ASD-related phenotypic markers, including variations in emotional expression, eye contact, and head movements. We have developed a protocol to curate high-quality videos from this dataset, forming a comprehensive training set. Utilizing this set, we trained individual LSTM-based models using eye gaze, head positions, and facial landmarks as input features, achieving test AUCs of 86%, 67%, and 78%, respectively. To boost diagnostic accuracy, we applied late fusion techniques to create ensemble models, improving the overall AUC to 90%. This approach also yielded more equitable results across different genders and age groups. Our methodology offers a significant step forward in the early detection of ASD by potentially reducing the reliance on subjective assessments and making early identification more accessibly and equitable.
△ Less
Submitted 23 August, 2024;
originally announced August 2024.
-
Machine and deep learning methods for predicting 3D genome organization
Authors:
Brydon P. G. Wall,
My Nguyen,
J. Chuck Harrell,
Mikhail G. Dozmorov
Abstract:
Three-Dimensional (3D) chromatin interactions, such as enhancer-promoter interactions (EPIs), loops, Topologically Associating Domains (TADs), and A/B compartments play critical roles in a wide range of cellular processes by regulating gene expression. Recent development of chromatin conformation capture technologies has enabled genome-wide profiling of various 3D structures, even with single cell…
▽ More
Three-Dimensional (3D) chromatin interactions, such as enhancer-promoter interactions (EPIs), loops, Topologically Associating Domains (TADs), and A/B compartments play critical roles in a wide range of cellular processes by regulating gene expression. Recent development of chromatin conformation capture technologies has enabled genome-wide profiling of various 3D structures, even with single cells. However, current catalogs of 3D structures remain incomplete and unreliable due to differences in technology, tools, and low data resolution. Machine learning methods have emerged as an alternative to obtain missing 3D interactions and/or improve resolution. Such methods frequently use genome annotation data (ChIP-seq, DNAse-seq, etc.), DNA sequencing information (k-mers, Transcription Factor Binding Site (TFBS) motifs), and other genomic properties to learn the associations between genomic features and chromatin interactions. In this review, we discuss computational tools for predicting three types of 3D interactions (EPIs, chromatin interactions, TAD boundaries) and analyze their pros and cons. We also point out obstacles of computational prediction of 3D interactions and suggest future research directions.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Advancing Human Action Recognition with Foundation Models trained on Unlabeled Public Videos
Authors:
Yang Qian,
Yinan Sun,
Ali Kargarandehkordi,
Parnian Azizian,
Onur Cezmi Mutlu,
Saimourya Surabhi,
Pingyi Chen,
Zain Jabbar,
Dennis Paul Wall,
Peter Washington
Abstract:
The increasing variety and quantity of tagged multimedia content on a variety of online platforms offer a unique opportunity to advance the field of human action recognition. In this study, we utilize 283,582 unique, unlabeled TikTok video clips, categorized into 386 hashtags, to train a domain-specific foundation model for action recognition. We employ VideoMAE V2, an advanced model integrating M…
▽ More
The increasing variety and quantity of tagged multimedia content on a variety of online platforms offer a unique opportunity to advance the field of human action recognition. In this study, we utilize 283,582 unique, unlabeled TikTok video clips, categorized into 386 hashtags, to train a domain-specific foundation model for action recognition. We employ VideoMAE V2, an advanced model integrating Masked Autoencoders (MAE) with Vision Transformers (ViT), pre-trained on this diverse collection of unstructured videos. Our model, fine-tuned on established action recognition benchmarks such as UCF101 and HMDB51, achieves state-of-the-art results: 99.05% on UCF101, 86.08% on HMDB51, 85.51% on Kinetics-400, and 74.27% on Something-Something V2 using the ViT-giant backbone. These results highlight the potential of using unstructured and unlabeled videos as a valuable source of diverse and dynamic content for training foundation models. Our investigation confirms that while initial increases in pre-training data volume significantly enhance model performance, the gains diminish as the dataset size continues to expand. Our findings emphasize two critical axioms in self-supervised learning for computer vision: (1) additional pre-training data can yield diminishing benefits for some datasets and (2) quality is more important than quantity in self-supervised learning, especially when building foundation models.
△ Less
Submitted 15 July, 2024; v1 submitted 13 February, 2024;
originally announced February 2024.
-
Computer Vision Estimation of Emotion Reaction Intensity in the Wild
Authors:
Yang Qian,
Ali Kargarandehkordi,
Onur Cezmi Mutlu,
Saimourya Surabhi,
Mohammadmahdi Honarmand,
Dennis Paul Wall,
Peter Washington
Abstract:
Emotions play an essential role in human communication. Developing computer vision models for automatic recognition of emotion expression can aid in a variety of domains, including robotics, digital behavioral healthcare, and media analytics. There are three types of emotional representations which are traditionally modeled in affective computing research: Action Units, Valence Arousal (VA), and C…
▽ More
Emotions play an essential role in human communication. Developing computer vision models for automatic recognition of emotion expression can aid in a variety of domains, including robotics, digital behavioral healthcare, and media analytics. There are three types of emotional representations which are traditionally modeled in affective computing research: Action Units, Valence Arousal (VA), and Categorical Emotions. As part of an effort to move beyond these representations towards more fine-grained labels, we describe our submission to the newly introduced Emotional Reaction Intensity (ERI) Estimation challenge in the 5th competition for Affective Behavior Analysis in-the-Wild (ABAW). We developed four deep neural networks trained in the visual domain and a multimodal model trained with both visual and audio features to predict emotion reaction intensity. Our best performing model on the Hume-Reaction dataset achieved an average Pearson correlation coefficient of 0.4080 on the test set using a pre-trained ResNet50 model. This work provides a first step towards the development of production-grade models which predict emotion reaction intensities rather than discrete emotion categories.
△ Less
Submitted 2 August, 2023; v1 submitted 19 March, 2023;
originally announced March 2023.
-
TempT: Temporal consistency for Test-time adaptation
Authors:
Onur Cezmi Mutlu,
Mohammadmahdi Honarmand,
Saimourya Surabhi,
Dennis P. Wall
Abstract:
We introduce Temporal consistency for Test-time adaptation (TempT) a novel method for test-time adaptation on videos through the use of temporal coherence of predictions across sequential frames as a self-supervision signal. TempT is an approach with broad potential applications in computer vision tasks including facial expression recognition (FER) in videos. We evaluate TempT performance on the A…
▽ More
We introduce Temporal consistency for Test-time adaptation (TempT) a novel method for test-time adaptation on videos through the use of temporal coherence of predictions across sequential frames as a self-supervision signal. TempT is an approach with broad potential applications in computer vision tasks including facial expression recognition (FER) in videos. We evaluate TempT performance on the AffWild2 dataset. Our approach focuses solely on the unimodal visual aspect of the data and utilizes a popular 2D CNN backbone in contrast to larger sequential or attention-based models used in other approaches. Our preliminary experimental results demonstrate that TempT has competitive performance compared to the previous years reported performances and its efficacy provides a compelling proof-of-concept for its use in various real-world applications.
△ Less
Submitted 18 April, 2023; v1 submitted 18 March, 2023;
originally announced March 2023.
-
A Review of and Roadmap for Data Science and Machine Learning for the Neuropsychiatric Phenotype of Autism
Authors:
Peter Washington,
Dennis P. Wall
Abstract:
Autism Spectrum Disorder (autism) is a neurodevelopmental delay which affects at least 1 in 44 children. Like many neurological disorder phenotypes, the diagnostic features are observable, can be tracked over time, and can be managed or even eliminated through proper therapy and treatments. Yet, there are major bottlenecks in the diagnostic, therapeutic, and longitudinal tracking pipelines for aut…
▽ More
Autism Spectrum Disorder (autism) is a neurodevelopmental delay which affects at least 1 in 44 children. Like many neurological disorder phenotypes, the diagnostic features are observable, can be tracked over time, and can be managed or even eliminated through proper therapy and treatments. Yet, there are major bottlenecks in the diagnostic, therapeutic, and longitudinal tracking pipelines for autism and related delays, creating an opportunity for novel data science solutions to augment and transform existing workflows and provide access to services for more affected families. Several prior efforts conducted by a multitude of research labs have spawned great progress towards improved digital diagnostics and digital therapies for children with autism. We review the literature of digital health methods for autism behavior quantification using data science. We describe both case-control studies and classification systems for digital phenotyping. We then discuss digital diagnostics and therapeutics which integrate machine learning models of autism-related behaviors, including the factors which must be addressed for translational use. Finally, we describe ongoing challenges and potent opportunities for the field of autism data science. Given the heterogeneous nature of autism and the complexities of the relevant behaviors, this review contains insights which are relevant to neurological behavior analysis and digital psychiatry more broadly.
△ Less
Submitted 6 March, 2023;
originally announced March 2023.
-
Mitigating Negative Transfer in Multi-Task Learning with Exponential Moving Average Loss Weighting Strategies
Authors:
Anish Lakkapragada,
Essam Sleiman,
Saimourya Surabhi,
Dennis P. Wall
Abstract:
Multi-Task Learning (MTL) is a growing subject of interest in deep learning, due to its ability to train models more efficiently on multiple tasks compared to using a group of conventional single-task models. However, MTL can be impractical as certain tasks can dominate training and hurt performance in others, thus making some tasks perform better in a single-task model compared to a multi-task on…
▽ More
Multi-Task Learning (MTL) is a growing subject of interest in deep learning, due to its ability to train models more efficiently on multiple tasks compared to using a group of conventional single-task models. However, MTL can be impractical as certain tasks can dominate training and hurt performance in others, thus making some tasks perform better in a single-task model compared to a multi-task one. Such problems are broadly classified as negative transfer, and many prior approaches in the literature have been made to mitigate these issues. One such current approach to alleviate negative transfer is to weight each of the losses so that they are on the same scale. Whereas current loss balancing approaches rely on either optimization or complex numerical analysis, none directly scale the losses based on their observed magnitudes. We propose multiple techniques for loss balancing based on scaling by the exponential moving average and benchmark them against current best-performing methods on three established datasets. On these datasets, they achieve comparable, if not higher, performance compared to current best-performing methods.
△ Less
Submitted 22 November, 2022;
originally announced November 2022.
-
An Exploration of Active Learning for Affective Digital Phenotyping
Authors:
Peter Washington,
Cezmi Mutlu,
Aaron Kline,
Cathy Hou,
Kaitlyn Dunlap,
Jack Kent,
Arman Husic,
Nate Stockham,
Brianna Chrisman,
Kelley Paskov,
Jae-Yoon Jung,
Dennis P. Wall
Abstract:
Some of the most severe bottlenecks preventing widespread development of machine learning models for human behavior include a dearth of labeled training data and difficulty of acquiring high quality labels. Active learning is a paradigm for using algorithms to computationally select a useful subset of data points to label using metrics for model uncertainty and data similarity. We explore active l…
▽ More
Some of the most severe bottlenecks preventing widespread development of machine learning models for human behavior include a dearth of labeled training data and difficulty of acquiring high quality labels. Active learning is a paradigm for using algorithms to computationally select a useful subset of data points to label using metrics for model uncertainty and data similarity. We explore active learning for naturalistic computer vision emotion data, a particularly heterogeneous and complex data space due to inherently subjective labels. Using frames collected from gameplay acquired from a therapeutic smartphone game for children with autism, we run a simulation of active learning using gameplay prompts as metadata to aid in the active learning process. We find that active learning using information generated during gameplay slightly outperforms random selection of the same number of labeled frames. We next investigate a method to conduct active learning with subjective data, such as in affective computing, and where multiple crowdsourced labels can be acquired for each image. Using the Child Affective Facial Expression (CAFE) dataset, we simulate an active learning process for crowdsourcing many labels and find that prioritizing frames using the entropy of the crowdsourced label distribution results in lower categorical cross-entropy loss compared to random frame selection. Collectively, these results demonstrate pilot evaluations of two novel active learning approaches for subjective affective data collected in noisy settings.
△ Less
Submitted 6 April, 2022; v1 submitted 4 April, 2022;
originally announced April 2022.
-
Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images
Authors:
Peter Washington,
Cezmi Onur Mutlu,
Aaron Kline,
Kelley Paskov,
Nate Tyler Stockham,
Brianna Chrisman,
Nick Deveau,
Mourya Surhabi,
Nick Haber,
Dennis P. Wall
Abstract:
Computer Vision (CV) classifiers which distinguish and detect nonverbal social human behavior and mental state can aid digital diagnostics and therapeutics for psychiatry and the behavioral sciences. While CV classifiers for traditional and structured classification tasks can be developed with standard machine learning pipelines for supervised learning consisting of data labeling, preprocessing, a…
▽ More
Computer Vision (CV) classifiers which distinguish and detect nonverbal social human behavior and mental state can aid digital diagnostics and therapeutics for psychiatry and the behavioral sciences. While CV classifiers for traditional and structured classification tasks can be developed with standard machine learning pipelines for supervised learning consisting of data labeling, preprocessing, and training a convolutional neural network, there are several pain points which arise when attempting this process for behavioral phenotyping. Here, we discuss the challenges and corresponding opportunities in this space, including handling heterogeneous data, avoiding biased models, labeling massive and repetitive data sets, working with ambiguous or compound class labels, managing privacy concerns, creating appropriate representations, and personalizing models. We discuss current state-of-the-art research endeavors in CV such as data curation, data augmentation, crowdsourced labeling, active learning, reinforcement learning, generative models, representation learning, federated learning, and meta-learning. We highlight at least some of the machine learning advancements needed for imaging classifiers to detect human social cues successfully and reliably.
△ Less
Submitted 26 January, 2022;
originally announced January 2022.
-
Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devices
Authors:
Agnik Banerjee,
Peter Washington,
Cezmi Mutlu,
Aaron Kline,
Dennis P. Wall
Abstract:
Implementing automated emotion recognition on mobile devices could provide an accessible diagnostic and therapeutic tool for those who struggle to recognize emotion, including children with developmental behavioral conditions such as autism. Although recent advances have been made in building more accurate emotion classifiers, existing models are too computationally expensive to be deployed on mob…
▽ More
Implementing automated emotion recognition on mobile devices could provide an accessible diagnostic and therapeutic tool for those who struggle to recognize emotion, including children with developmental behavioral conditions such as autism. Although recent advances have been made in building more accurate emotion classifiers, existing models are too computationally expensive to be deployed on mobile devices. In this study, we optimized and profiled various machine learning models designed for inference on edge devices and were able to match previous state of the art results for emotion recognition on children. Our best model, a MobileNet-V2 network pre-trained on ImageNet, achieved 65.11% balanced accuracy and 64.19% F1-score on CAFE, while achieving a 45-millisecond inference latency on a Motorola Moto G6 phone. This balanced accuracy is only 1.79% less than the current state of the art for CAFE, which used a model that contains 26.62x more parameters and was unable to run on the Moto G6, even when fully optimized. This work validates that with specialized design and optimization techniques, machine learning models can become lightweight enough for deployment on mobile devices and still achieve high accuracies on difficult image classification tasks.
△ Less
Submitted 21 August, 2021;
originally announced August 2021.
-
Artificial Intelligence in the Global South (AI4D): Potential and Risks
Authors:
P. J. Wall,
Deepak Saxena,
Suzana Brown
Abstract:
Artificial intelligence is becoming more widely available in all parts of the world. This has created many previously unforeseen possibilities for addressing the challenges outlined in the Sustainable Development Goals in the Global South. However, the use of AI in such contexts brings with it a unique set of risks and challenges. Among these are the potential for Governments to use such technolog…
▽ More
Artificial intelligence is becoming more widely available in all parts of the world. This has created many previously unforeseen possibilities for addressing the challenges outlined in the Sustainable Development Goals in the Global South. However, the use of AI in such contexts brings with it a unique set of risks and challenges. Among these are the potential for Governments to use such technologies to suppress their own people, and the ethical questions arising from implementing AI primarily designed and developed in the Global North into vastly different social, cultural, and political environments in the Global South. This paper examines the key issues and questions arising in the emerging sub-field of AI for global development (AI4D) and the potential and risks associated with using such technologies in the Global South. We propose that although there are many risks associated with the use of AI, the potential benefits are enough to warrant detailed research and investigation of the most appropriate and effective ways to design, develop, implement, and use such technologies in the Global South. We conclude by calling for the wider ICT4D community to continue to conduct detailed research and investigation of all aspects of AI4D.
△ Less
Submitted 23 August, 2021;
originally announced August 2021.
-
Artificial Intelligence Ethics: An Inclusive Global Discourse?
Authors:
Cathy Roche,
Dave Lewis,
P. J. Wall
Abstract:
It is widely accepted that technology is ubiquitous across the planet and has the potential to solve many of the problems existing in the Global South. Moreover, the rapid advancement of artificial intelligence (AI) brings with it the potential to address many of the challenges outlined in the Sustainable Development Goals (SDGs) in ways which were never before possible. However, there are many qu…
▽ More
It is widely accepted that technology is ubiquitous across the planet and has the potential to solve many of the problems existing in the Global South. Moreover, the rapid advancement of artificial intelligence (AI) brings with it the potential to address many of the challenges outlined in the Sustainable Development Goals (SDGs) in ways which were never before possible. However, there are many questions about how such advanced technologies should be managed and governed, and whether or not the emerging ethical frameworks and standards for AI are dominated by the Global North. This research examines the growing body of documentation on AI ethics to examine whether or not there is equality of participation in the ongoing global discourse. Specifically, it seeks to discover if both countries in the Global South and women are underrepresented in this discourse. Findings indicate a dearth of references to both of these themes in the AI ethics documents, suggesting that the associated ethical implications and risks are being neglected. Without adequate input from both countries in the Global South and from women, such ethical frameworks and standards may be discriminatory with the potential to reinforce marginalisation.
△ Less
Submitted 23 August, 2021;
originally announced August 2021.
-
Designing Mobile Health for User Engagement: The Importance of Socio-Technical Approach
Authors:
Tochukwu Ikwunne,
Lucy Hederman,
P. J. Wall
Abstract:
Despite the significance of user engagement for efficacy of mobile health (mHealth) in the Global South, many such interventions do not include user-engaging attributes. This is because socio-technical aspects are frequently not considered during the design, development, and implementation, stages of such initiatives. In addition, there is little discussion in the literature about the role socio-t…
▽ More
Despite the significance of user engagement for efficacy of mobile health (mHealth) in the Global South, many such interventions do not include user-engaging attributes. This is because socio-technical aspects are frequently not considered during the design, development, and implementation, stages of such initiatives. In addition, there is little discussion in the literature about the role socio-technical factors play in user-centered design processes for mHealth. This research posits consideration of socio-technical factors is required as techno-centric approaches to mHealth design and user engagement, as well as those relying on existing universal frameworks for user-centered design, have proven to be ineffective with the result that most mHealth projects in the Global South fail to sustain. This research examines projects in Sierra Leone where semi-structured interviews were conducted with mHealth designers and developers in order to explore their attitudes towards user engagement in this case. Barriers and facilitators to user engagement were identified and classified as either technical or socio-technical. Findings from the study indicate that adoption of a techno-centric approach without consideration of socio-technical factors can negatively affect user's engagement. Based on these findings, we propose to develop a new design framework for more effective inclusion of user-engaging attributes in mHealth.
△ Less
Submitted 24 August, 2021; v1 submitted 22 August, 2021;
originally announced August 2021.
-
Scalable Hypergraph Embedding System
Authors:
Sepideh Maleki,
Donya Saless,
Dennis P. Wall,
Keshav Pingali
Abstract:
Many problems such as node classification and link prediction in network data can be solved using graph embeddings. However, it is difficult to use graphs to capture non-binary relations such as communities of nodes. These kinds of complex relations are expressed more naturally as hypergraphs. While hypergraphs are a generalization of graphs, state-of-the-art graph embedding techniques are not ade…
▽ More
Many problems such as node classification and link prediction in network data can be solved using graph embeddings. However, it is difficult to use graphs to capture non-binary relations such as communities of nodes. These kinds of complex relations are expressed more naturally as hypergraphs. While hypergraphs are a generalization of graphs, state-of-the-art graph embedding techniques are not adequate for solving prediction and classification tasks on large hypergraphs accurately in reasonable time. In this paper, we introduce HyperNetVec, a novel hierarchical framework for scalable unsupervised hypergraph embedding. HyperNetVec exploits shared-memory parallelism and is capable of generating high quality embeddings for real-world hypergraphs with millions of nodes and hyperedges in only a couple of minutes while existing hypergraph systems either fail for such large hypergraphs or may take days to produce the embeddings.
△ Less
Submitted 25 January, 2022; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Activity Recognition with Moving Cameras and Few Training Examples: Applications for Detection of Autism-Related Headbanging
Authors:
Peter Washington,
Aaron Kline,
Onur Cezmi Mutlu,
Emilie Leblanc,
Cathy Hou,
Nate Stockham,
Kelley Paskov,
Brianna Chrisman,
Dennis P. Wall
Abstract:
Activity recognition computer vision algorithms can be used to detect the presence of autism-related behaviors, including what are termed "restricted and repetitive behaviors", or stimming, by diagnostic instruments. The limited data that exist in this domain are usually recorded with a handheld camera which can be shaky or even moving, posing a challenge for traditional feature representation app…
▽ More
Activity recognition computer vision algorithms can be used to detect the presence of autism-related behaviors, including what are termed "restricted and repetitive behaviors", or stimming, by diagnostic instruments. The limited data that exist in this domain are usually recorded with a handheld camera which can be shaky or even moving, posing a challenge for traditional feature representation approaches for activity detection which mistakenly capture the camera's motion as a feature. To address these issues, we first document the advantages and limitations of current feature representation techniques for activity recognition when applied to head banging detection. We then propose a feature representation consisting exclusively of head pose keypoints. We create a computer vision classifier for detecting head banging in home videos using a time-distributed convolutional neural network (CNN) in which a single CNN extracts features from each frame in the input sequence, and these extracted features are fed as input to a long short-term memory (LSTM) network. On the binary task of predicting head banging and no head banging within videos from the Self Stimulatory Behaviour Dataset (SSBD), we reach a mean F1-score of 90.77% using 3-fold cross validation (with individual fold F1-scores of 83.3%, 89.0%, and 100.0%) when ensuring that no child who appeared in the train set was in the test set for all folds. This work documents a successful technique for training a computer vision classifier which can detect human motion with few training examples and even when the camera recording the source clips is unstable. The general methods described here can be applied by designers and developers of interactive systems towards other human motion and pose classification problems used in mobile and ubiquitous interactive systems.
△ Less
Submitted 10 January, 2021;
originally announced January 2021.
-
Improved Digital Therapy for Developmental Pediatrics Using Domain-Specific Artificial Intelligence: Machine Learning Study
Authors:
Peter Washington,
Haik Kalantarian,
John Kent,
Arman Husic,
Aaron Kline,
Emilie Leblanc,
Cathy Hou,
Onur Cezmi Mutlu,
Kaitlyn Dunlap,
Yordan Penev,
Maya Varma,
Nate Tyler Stockham,
Brianna Chrisman,
Kelley Paskov,
Min Woo Sun,
Jae-Yoon Jung,
Catalin Voss,
Nick Haber,
Dennis Paul Wall
Abstract:
Background: Automated emotion classification could aid those who struggle to recognize emotions, including children with developmental behavioral conditions such as autism. However, most computer vision emotion recognition models are trained on adult emotion and therefore underperform when applied to child faces. Objective: We designed a strategy to gamify the collection and labeling of child emot…
▽ More
Background: Automated emotion classification could aid those who struggle to recognize emotions, including children with developmental behavioral conditions such as autism. However, most computer vision emotion recognition models are trained on adult emotion and therefore underperform when applied to child faces. Objective: We designed a strategy to gamify the collection and labeling of child emotion-enriched images to boost the performance of automatic child emotion recognition models to a level closer to what will be needed for digital health care approaches. Methods: We leveraged our prototype therapeutic smartphone game, GuessWhat, which was designed in large part for children with developmental and behavioral conditions, to gamify the secure collection of video data of children expressing a variety of emotions prompted by the game. Independently, we created a secure web interface to gamify the human labeling effort, called HollywoodSquares, tailored for use by any qualified labeler. We gathered and labeled 2155 videos, 39,968 emotion frames, and 106,001 labels on all images. With this drastically expanded pediatric emotion-centric database (>30 times larger than existing public pediatric emotion data sets), we trained a convolutional neural network (CNN) computer vision classifier of happy, sad, surprised, fearful, angry, disgust, and neutral expressions evoked by children. Results: The classifier achieved a 66.9% balanced accuracy and 67.4% F1-score on the entirety of the Child Affective Facial Expression (CAFE) as well as a 79.1% balanced accuracy and 78% F1-score on CAFE Subset A, a subset containing at least 60% human agreement on emotions labels. This performance is at least 10% higher than all previously developed classifiers evaluated against CAFE, the best of which reached a 56% balanced accuracy even when combining "anger" and "disgust" into a single class.
△ Less
Submitted 3 June, 2024; v1 submitted 15 December, 2020;
originally announced December 2020.
-
A Wearable Social Interaction Aid for Children with Autism
Authors:
Nick Haber,
Catalin Voss,
Jena Daniels,
Peter Washington,
Azar Fazel,
Aaron Kline,
Titas De,
Terry Winograd,
Carl Feinstein,
Dennis P. Wall
Abstract:
With most recent estimates giving an incidence rate of 1 in 68 children in the United States, the autism spectrum disorder (ASD) is a growing public health crisis. Many of these children struggle to make eye contact, recognize facial expressions, and engage in social interactions. Today the standard for treatment of the core autism-related deficits focuses on a form of behavior training known as A…
▽ More
With most recent estimates giving an incidence rate of 1 in 68 children in the United States, the autism spectrum disorder (ASD) is a growing public health crisis. Many of these children struggle to make eye contact, recognize facial expressions, and engage in social interactions. Today the standard for treatment of the core autism-related deficits focuses on a form of behavior training known as Applied Behavioral Analysis. To address perceived deficits in expression recognition, ABA approaches routinely involve the use of prompts such as flash cards for repetitive emotion recognition training via memorization. These techniques must be administered by trained practitioners and often at clinical centers that are far outnumbered by and out of reach from the many children and families in need of attention. Waitlists for access are up to 18 months long, and this wait may lead to children regressing down a path of isolation that worsens their long-term prognosis. There is an urgent need to innovate new methods of care delivery that can appropriately empower caregivers of children at risk or with a diagnosis of autism, and that capitalize on mobile tools and wearable devices for use outside of clinical settings.
△ Less
Submitted 19 April, 2020;
originally announced April 2020.
-
Logarithmic means \dots]{On the Nörlund logarithmic means with respect to Vilenkin system in the martingale Hardy space $H_{1}$
Authors:
L. -E. Persson,
G. Tephnadze,
P. Wall
Abstract:
In this paper we prove and discuss a new divergence result of Nörlund logarithmic means with respect to Vilenkin system in Hardy space $H_1. $
In this paper we prove and discuss a new divergence result of Nörlund logarithmic means with respect to Vilenkin system in Hardy space $H_1. $
△ Less
Submitted 13 February, 2018;
originally announced February 2018.
-
Machine learning approach for early detection of autism by combining questionnaire and home video screening
Authors:
Halim Abbas,
Ford Garberson,
Eric Glover,
Dennis P Wall
Abstract:
Existing screening tools for early detection of autism are expensive, cumbersome, time-intensive, and sometimes fall short in predictive value. In this work, we apply Machine Learning (ML) to gold standard clinical data obtained across thousands of children at risk for autism spectrum disorders to create a low-cost, quick, and easy to apply autism screening tool that performs as well or better tha…
▽ More
Existing screening tools for early detection of autism are expensive, cumbersome, time-intensive, and sometimes fall short in predictive value. In this work, we apply Machine Learning (ML) to gold standard clinical data obtained across thousands of children at risk for autism spectrum disorders to create a low-cost, quick, and easy to apply autism screening tool that performs as well or better than most widely used standardized instruments. This new tool combines two screening methods into a single assessment, one based on short, structured parent-report questionnaires and the other on tagging key behaviors from short, semi-structured home videos of children. To overcome the scarcity, sparsity, and imbalance of training data, we apply creative feature selection, feature engineering, and novel feature encoding techniques. We allow for inconclusive determination where appropriate in order to boost screening accuracy when conclusive. We demonstrate a significant accuracy improvement over standard screening tools in a clinical study sample of 162 children.
△ Less
Submitted 15 March, 2017;
originally announced March 2017.
-
Some new $\left(H_{p},L_{p}\right)$ type inequalities of maximal operators of Vilenkin-Nörlund means with non-decreasing coefficients
Authors:
L. E. Persson,
G. Tephnadze,
P. Wall
Abstract:
In this paper we prove and discuss some new $\left(H_{p},L_{p}\right)$ type inequalities of maximal operators of Vilenkin-Nörlund means with non-decreasing coefficients. We also apply these inequalities to prove strong convergence theorems of such Vilenkin-Nörlund means. These inequalities are the best possible in a special sense. As applications, both some well-known and new results are pointed o…
▽ More
In this paper we prove and discuss some new $\left(H_{p},L_{p}\right)$ type inequalities of maximal operators of Vilenkin-Nörlund means with non-decreasing coefficients. We also apply these inequalities to prove strong convergence theorems of such Vilenkin-Nörlund means. These inequalities are the best possible in a special sense. As applications, both some well-known and new results are pointed out.
△ Less
Submitted 21 April, 2015;
originally announced April 2015.
-
Maximal operators of Vilenkin-Nörlund means
Authors:
L. E. Persson,
G. Tephnadze,
P. Wall
Abstract:
In this paper we prove and discuss some new $\left(H_{p},weak-L_{p}\right) $ type inequalities of maximal operators of Vilenkin-Nörlund means with monotone coefficients. We also apply these results to prove a.e. convergence of such Vilenkin-Nörlund means. It is also proved that these results are the best possible in a special sense. As applications, both some well-known and new results are pointed…
▽ More
In this paper we prove and discuss some new $\left(H_{p},weak-L_{p}\right) $ type inequalities of maximal operators of Vilenkin-Nörlund means with monotone coefficients. We also apply these results to prove a.e. convergence of such Vilenkin-Nörlund means. It is also proved that these results are the best possible in a special sense. As applications, both some well-known and new results are pointed out.
△ Less
Submitted 25 January, 2015;
originally announced March 2015.
-
On weak convergence of locally periodic functions
Authors:
Dag Lukkassen,
Peter Wall
Abstract:
We prove a generalization of the fact that periodic functions converge weakly to the mean value as the oscillation increases. Some convergence questions connected to locally periodic nonlinear boundary value problems are also considered.
We prove a generalization of the fact that periodic functions converge weakly to the mean value as the oscillation increases. Some convergence questions connected to locally periodic nonlinear boundary value problems are also considered.
△ Less
Submitted 18 October, 2002;
originally announced October 2002.
-
Some homogenization and corrector results for nonlinear monotone operators
Authors:
Peter Wall
Abstract:
This paper deals with the limit behaviour of the solutions of quasi-linear equations of the form \ $\ds -\limfunc{div}\left(a\left(x, x/{\varepsilon _h},Du_h\right)\right)=f_h$ on $Ω$ with Dirichlet boundary conditions. The sequence $(\varepsilon _h)$ tends to $0$ and the map $a(x,y,ξ)$ is periodic in $y$, monotone in $ξ$ and satisfies suitable continuity conditions. It is proved that…
▽ More
This paper deals with the limit behaviour of the solutions of quasi-linear equations of the form \ $\ds -\limfunc{div}\left(a\left(x, x/{\varepsilon _h},Du_h\right)\right)=f_h$ on $Ω$ with Dirichlet boundary conditions. The sequence $(\varepsilon _h)$ tends to $0$ and the map $a(x,y,ξ)$ is periodic in $y$, monotone in $ξ$ and satisfies suitable continuity conditions. It is proved that $u_h\rightarrow u$ weakly in $H_0^{1,2}(Ω)$, where $u$ is the solution of a homogenized problem \ $-\limfunc{div}(b(x,Du))=f$ on $Ω$. We also prove some corrector results, i.e. we find $(P_h)$ such that $Du_h-P_h(Du)\rightarrow 0$ in $L^2(Ω,R^n)$.
△ Less
Submitted 30 June, 1998;
originally announced July 1998.
