-
Helios 2.0: A Robust, Ultra-Low Power Gesture Recognition System Optimised for Event-Sensor based Wearables
Authors:
Prarthana Bhattacharyya,
Joshua Mitton,
Ryan Page,
Owen Morgan,
Oliver Powell,
Benjamin Menzies,
Gabriel Homewood,
Kemi Jacobs,
Paolo Baesso,
Taru Muhonen,
Richard Vigars,
Louis Berridge
Abstract:
We present an advance in wearable technology: a mobile-optimized, real-time, ultra-low-power event camera system that enables natural hand gesture control for smart glasses, dramatically improving user experience. While hand gesture recognition in computer vision has advanced significantly, critical challenges remain in creating systems that are intuitive, adaptable across diverse users and enviro…
▽ More
We present an advance in wearable technology: a mobile-optimized, real-time, ultra-low-power event camera system that enables natural hand gesture control for smart glasses, dramatically improving user experience. While hand gesture recognition in computer vision has advanced significantly, critical challenges remain in creating systems that are intuitive, adaptable across diverse users and environments, and energy-efficient enough for practical wearable applications. Our approach tackles these challenges through carefully selected microgestures: lateral thumb swipes across the index finger (in both directions) and a double pinch between thumb and index fingertips. These human-centered interactions leverage natural hand movements, ensuring intuitive usability without requiring users to learn complex command sequences. To overcome variability in users and environments, we developed a novel simulation methodology that enables comprehensive domain sampling without extensive real-world data collection. Our power-optimised architecture maintains exceptional performance, achieving F1 scores above 80\% on benchmark datasets featuring diverse users and environments. The resulting models operate at just 6-8 mW when exploiting the Qualcomm Snapdragon Hexagon DSP, with our 2-channel implementation exceeding 70\% F1 accuracy and our 6-channel model surpassing 80\% F1 accuracy across all gesture classes in user studies. These results were achieved using only synthetic training data. This improves on the state-of-the-art for F1 accuracy by 20\% with a power reduction 25x when using DSP. This advancement brings deploying ultra-low-power vision systems in wearable devices closer and opens new possibilities for seamless human-computer interaction.
△ Less
Submitted 10 March, 2025;
originally announced March 2025.
-
Participatory Budget Allocation Method for Approval Ballots
Authors:
Rutvik Page,
Arnav Doifode,
Jitendra Tembhurne,
Aishwarya Sagar Anand Ukey
Abstract:
In this paper, we study the problem of Participatory Budgeting (PB) with approval ballots, inspired by Multi-Winner Voting schemes. We present generalized preference aggregation methods for participatory budgeting, especially for finding seemingly fair budget allocations. To achieve this, we generalize such preference aggregation methods from the well-known methods, namely the Sequential Chamberli…
▽ More
In this paper, we study the problem of Participatory Budgeting (PB) with approval ballots, inspired by Multi-Winner Voting schemes. We present generalized preference aggregation methods for participatory budgeting, especially for finding seemingly fair budget allocations. To achieve this, we generalize such preference aggregation methods from the well-known methods, namely the Sequential Chamberlin Courant rule and the Sequential Monroe Rule in the realm of social choice theory. Further, we provide an experimental evaluation of the preference aggregation methods using an impartial culture method of preference generation and study the extent to which such polynomial time algorithms satisfy one of the most popular notions of fairness called proportional representation.
△ Less
Submitted 9 October, 2024;
originally announced October 2024.
-
Helios: An extremely low power event-based gesture recognition for always-on smart eyewear
Authors:
Prarthana Bhattacharyya,
Joshua Mitton,
Ryan Page,
Owen Morgan,
Ben Menzies,
Gabriel Homewood,
Kemi Jacobs,
Paolo Baesso,
David Trickett,
Chris Mair,
Taru Muhonen,
Rory Clark,
Louis Berridge,
Richard Vigars,
Iain Wallace
Abstract:
This paper introduces Helios, the first extremely low-power, real-time, event-based hand gesture recognition system designed for all-day on smart eyewear. As augmented reality (AR) evolves, current smart glasses like the Meta Ray-Bans prioritize visual and wearable comfort at the expense of functionality. Existing human-machine interfaces (HMIs) in these devices, such as capacitive touch and voice…
▽ More
This paper introduces Helios, the first extremely low-power, real-time, event-based hand gesture recognition system designed for all-day on smart eyewear. As augmented reality (AR) evolves, current smart glasses like the Meta Ray-Bans prioritize visual and wearable comfort at the expense of functionality. Existing human-machine interfaces (HMIs) in these devices, such as capacitive touch and voice controls, present limitations in ergonomics, privacy and power consumption. Helios addresses these challenges by leveraging natural hand interactions for a more intuitive and comfortable user experience. Our system utilizes a extremely low-power and compact 3mmx4mm/20mW event camera to perform natural hand-based gesture recognition for always-on smart eyewear. The camera's output is processed by a convolutional neural network (CNN) running on a NXP Nano UltraLite compute platform, consuming less than 350mW. Helios can recognize seven classes of gestures, including subtle microgestures like swipes and pinches, with 91% accuracy. We also demonstrate real-time performance across 20 users at a remarkably low latency of 60ms. Our user testing results align with the positive feedback we received during our recent successful demo at AWE-USA-2024.
△ Less
Submitted 26 August, 2024; v1 submitted 6 July, 2024;
originally announced July 2024.
-
AnthroNet: Conditional Generation of Humans via Anthropometrics
Authors:
Francesco Picetti,
Shrinath Deshpande,
Jonathan Leban,
Soroosh Shahtalebi,
Jay Patel,
Peifeng Jing,
Chunpu Wang,
Charles Metze III,
Cameron Sun,
Cera Laidlaw,
James Warren,
Kathy Huynh,
River Page,
Jonathan Hogins,
Adam Crespi,
Sujoy Ganguly,
Salehe Erfanian Ebadi
Abstract:
We present a novel human body model formulated by an extensive set of anthropocentric measurements, which is capable of generating a wide range of human body shapes and poses. The proposed model enables direct modeling of specific human identities through a deep generative architecture, which can produce humans in any arbitrary pose. It is the first of its kind to have been trained end-to-end usin…
▽ More
We present a novel human body model formulated by an extensive set of anthropocentric measurements, which is capable of generating a wide range of human body shapes and poses. The proposed model enables direct modeling of specific human identities through a deep generative architecture, which can produce humans in any arbitrary pose. It is the first of its kind to have been trained end-to-end using only synthetically generated data, which not only provides highly accurate human mesh representations but also allows for precise anthropometry of the body. Moreover, using a highly diverse animation library, we articulated our synthetic humans' body and hands to maximize the diversity of the learnable priors for model training. Our model was trained on a dataset of $100k$ procedurally-generated posed human meshes and their corresponding anthropometric measurements. Our synthetic data generator can be used to generate millions of unique human identities and poses for non-commercial academic research purposes.
△ Less
Submitted 7 September, 2023;
originally announced September 2023.
-
Proportionality in Committee Selection with Negative Feelings
Authors:
Nimrod Talmon,
Rutvik Page
Abstract:
We study a class of elections in which the input format is trichotomous and allows voters to elicit their negative feelings explicitly. In particular, we study multiwinner elections with a special proclivity to elect proportionally representative committees. That is, we design various axioms to deal with negative feelings and suggest some structures to these preferences that allow better preferenc…
▽ More
We study a class of elections in which the input format is trichotomous and allows voters to elicit their negative feelings explicitly. In particular, we study multiwinner elections with a special proclivity to elect proportionally representative committees. That is, we design various axioms to deal with negative feelings and suggest some structures to these preferences that allow better preference aggregation rules. We propose two different classes of axioms designed to aggregate trichotomous preferences more efficiently. We propose trichotomous versions of some well known multiwinner voting rules and report their satisfiability of our axioms. Hence, with reports of our simulations as evidence, we build upon the social optimality of our proportionality based axioms to evaluate the quality of voting rules for electing a proportionally representative committee with trichotomous ballots as inputs.
△ Less
Submitted 13 January, 2021; v1 submitted 5 January, 2021;
originally announced January 2021.
-
Electing the Executive Branch
Authors:
Rutvik Page,
Ehud Shapiro,
Nimrod Talmon
Abstract:
The executive branch, or government, is typically not elected directly by the people, but rather formed by another elected body or person such as the parliament or the president. As a result, its members are not directly accountable to the people, individually or as a group. We consider a scenario in which the members of the government are elected directly by the people, and wish to achieve propor…
▽ More
The executive branch, or government, is typically not elected directly by the people, but rather formed by another elected body or person such as the parliament or the president. As a result, its members are not directly accountable to the people, individually or as a group. We consider a scenario in which the members of the government are elected directly by the people, and wish to achieve proportionality while doing so. We propose a formal model consisting of $k$ offices, each with its own disjoint set of candidates, and a set of voters who provide approval ballots for all offices. We wish to identify good aggregation rules that assign one candidate to each office. As using a simple majority vote for each office independently might result in disregarding minority preferences altogether, here we consider an adaptation of the greedy variant of Proportional Approval Voting (GreedyPAV) to our setting, and demonstrate -- through computer-based simulations -- how voting for all offices together using this rule overcomes this weakness. We note that the approach is applicable also to a party that employs direct democracy, where party members elect the party's representatives in a coalition government.
△ Less
Submitted 15 September, 2021; v1 submitted 21 September, 2020;
originally announced September 2020.
-
DeepTag: inferring all-cause diagnoses from clinical notes in under-resourced medical domain
Authors:
Allen Nie,
Ashley Zehnder,
Rodney L. Page,
Arturo L. Pineda,
Manuel A. Rivas,
Carlos D. Bustamante,
James Zou
Abstract:
Large scale veterinary clinical records can become a powerful resource for patient care and research. However, clinicians lack the time and resource to annotate patient records with standard medical diagnostic codes and most veterinary visits are captured in free text notes. The lack of standard coding makes it challenging to use the clinical data to improve patient care. It is also a major impedi…
▽ More
Large scale veterinary clinical records can become a powerful resource for patient care and research. However, clinicians lack the time and resource to annotate patient records with standard medical diagnostic codes and most veterinary visits are captured in free text notes. The lack of standard coding makes it challenging to use the clinical data to improve patient care. It is also a major impediment to cross-species translational research, which relies on the ability to accurately identify patient cohorts with specific diagnostic criteria in humans and animals. In order to reduce the coding burden for veterinary clinical practice and aid translational research, we have developed a deep learning algorithm, DeepTag, which automatically infers diagnostic codes from veterinary free text notes. DeepTag is trained on a newly curated dataset of 112,558 veterinary notes manually annotated by experts. DeepTag extends multi-task LSTM with an improved hierarchical objective that captures the semantic structures between diseases. To foster human-machine collaboration, DeepTag also learns to abstain in examples when it is uncertain and defers them to human experts, resulting in improved performance. DeepTag accurately infers disease codes from free text even in challenging cross-hospital settings where the text comes from different clinical settings than the ones used for training. It enables automated disease annotation across a broad range of clinical diagnoses with minimal pre-processing. The technical framework in this work can be applied in other medical domains that currently lack medical coding resources.
△ Less
Submitted 3 September, 2018; v1 submitted 27 June, 2018;
originally announced June 2018.
-
How Computers Work: Computational Thinking for Everyone
Authors:
Rex Page,
Ruben Gamboa
Abstract:
What would you teach if you had only one course to help students grasp the essence of computation and perhaps inspire a few of them to make computing a subject of further study? Assume they have the standard college prep background. This would include basic algebra, but not necessarily more advanced mathematics. They would have written a few term papers, but would not have written computer program…
▽ More
What would you teach if you had only one course to help students grasp the essence of computation and perhaps inspire a few of them to make computing a subject of further study? Assume they have the standard college prep background. This would include basic algebra, but not necessarily more advanced mathematics. They would have written a few term papers, but would not have written computer programs. They could surf and twitter, but could not exclusive-or and nand. What about computers would interest them or help them place their experience in context? This paper provides one possible answer to this question by discussing a course that has completed its second iteration. Grounded in classical logic, elucidated in digital circuits and computer software, it expands into areas such as CPU components and massive databases. The course has succeeded in garnering the enthusiastic attention of students with a broad range of interests, exercising their problem solving skills, and introducing them to computational thinking.
△ Less
Submitted 22 January, 2013;
originally announced January 2013.
-
Accurate Programming: Thinking about programs in terms of properties
Authors:
Walid Taha,
Veronica Gaspes,
Rex Page
Abstract:
Accurate programming is a practical approach to producing high quality programs. It combines ideas from test-automation, test-driven development, agile programming, and other state of the art software development methods. In addition to building on approaches that have proven effective in practice, it emphasizes concepts that help programmers sharpen their understanding of both the problems they…
▽ More
Accurate programming is a practical approach to producing high quality programs. It combines ideas from test-automation, test-driven development, agile programming, and other state of the art software development methods. In addition to building on approaches that have proven effective in practice, it emphasizes concepts that help programmers sharpen their understanding of both the problems they are solving and the solutions they come up with. This is achieved by encouraging programmers to think about programs in terms of properties.
△ Less
Submitted 4 September, 2011;
originally announced September 2011.
