-
LookAlike: Consistent Distractor Generation in Math MCQs
Authors:
Nisarg Parikh,
Nigel Fernandez,
Alexander Scarlatos,
Simon Woodhead,
Andrew Lan
Abstract:
Large language models (LLMs) are increasingly used to generate distractors for multiple-choice questions (MCQs), especially in domains like math education. However, existing approaches are limited in ensuring that the generated distractors are consistent with common student errors. We propose LookAlike, a method that improves error-distractor consistency via preference optimization. Our two main i…
▽ More
Large language models (LLMs) are increasingly used to generate distractors for multiple-choice questions (MCQs), especially in domains like math education. However, existing approaches are limited in ensuring that the generated distractors are consistent with common student errors. We propose LookAlike, a method that improves error-distractor consistency via preference optimization. Our two main innovations are: (a) mining synthetic preference pairs from model inconsistencies, and (b) alternating supervised fine-tuning (SFT) with Direct Preference Optimization (DPO) to stabilize training. Unlike prior work that relies on heuristics or manually annotated preference data, LookAlike uses its own generation inconsistencies as dispreferred samples, thus enabling scalable and stable training. Evaluated on a real-world dataset of 1,400+ math MCQs, LookAlike achieves 51.6% accuracy in distractor generation and 57.2% in error generation under LLM-as-a-judge evaluation, outperforming an existing state-of-the-art method (45.6% / 47.7%). These improvements highlight the effectiveness of preference-based regularization and inconsistency mining for generating consistent math MCQ distractors at scale.
△ Less
Submitted 3 May, 2025;
originally announced May 2025.
-
Automated Knowledge Component Generation and Knowledge Tracing for Coding Problems
Authors:
Zhangqi Duan,
Nigel Fernandez,
Sri Kanakadandi,
Bita Akram,
Andrew Lan
Abstract:
Knowledge components (KCs) mapped to problems help model student learning, tracking their mastery levels on fine-grained skills thereby facilitating personalized learning and feedback in online learning platforms. However, crafting and tagging KCs to problems, traditionally performed by human domain experts, is highly labor-intensive. We present a fully automated, LLM-based pipeline for KC generat…
▽ More
Knowledge components (KCs) mapped to problems help model student learning, tracking their mastery levels on fine-grained skills thereby facilitating personalized learning and feedback in online learning platforms. However, crafting and tagging KCs to problems, traditionally performed by human domain experts, is highly labor-intensive. We present a fully automated, LLM-based pipeline for KC generation and tagging for open-ended programming problems. We also develop an LLM-based knowledge tracing (KT) framework to leverage these LLM-generated KCs, which we refer to as KCGen-KT. We conduct extensive quantitative and qualitative evaluations validating the effectiveness of KCGen-KT. On a real-world dataset of student code submissions to open-ended programming problems, KCGen-KT outperforms existing KT methods. We investigate the learning curves of generated KCs and show that LLM-generated KCs have a comparable level-of-fit to human-written KCs under the performance factor analysis (PFA) model. We also conduct a human evaluation to show that the KC tagging accuracy of our pipeline is reasonably accurate when compared to that by human domain experts.
△ Less
Submitted 25 February, 2025;
originally announced February 2025.
-
The Art of Misclassification: Too Many Classes, Not Enough Points
Authors:
Mario Franco,
Gerardo Febres,
Nelson Fernández,
Carlos Gershenson
Abstract:
Classification is a ubiquitous and fundamental problem in artificial intelligence and machine learning, with extensive efforts dedicated to developing more powerful classifiers and larger datasets. However, the classification task is ultimately constrained by the intrinsic properties of datasets, independently of computational power or model complexity. In this work, we introduce a formal entropy-…
▽ More
Classification is a ubiquitous and fundamental problem in artificial intelligence and machine learning, with extensive efforts dedicated to developing more powerful classifiers and larger datasets. However, the classification task is ultimately constrained by the intrinsic properties of datasets, independently of computational power or model complexity. In this work, we introduce a formal entropy-based measure of classificability, which quantifies the inherent difficulty of a classification problem by assessing the uncertainty in class assignments given feature representations. This measure captures the degree of class overlap and aligns with human intuition, serving as an upper bound on classification performance for classification problems. Our results establish a theoretical limit beyond which no classifier can improve the classification accuracy, regardless of the architecture or amount of data, in a given problem. Our approach provides a principled framework for understanding when classification is inherently fallible and fundamentally ambiguous.
△ Less
Submitted 11 February, 2025;
originally announced February 2025.
-
Software Bills of Materials in Maven Central
Authors:
Yogya Gamage,
Nadia Gonzalez Fernandez,
Martin Monperrus,
Benoit Baudry
Abstract:
Software Bills of Materials (SBOMs) are essential to ensure the transparency and integrity of the software supply chain. There is a growing body of work that investigates the accuracy of SBOM generation tools and the challenges for producing complete SBOMs. Yet, there is little knowledge about how developers distribute SBOMs. In this work, we mine SBOMs from Maven Central to assess the extent to w…
▽ More
Software Bills of Materials (SBOMs) are essential to ensure the transparency and integrity of the software supply chain. There is a growing body of work that investigates the accuracy of SBOM generation tools and the challenges for producing complete SBOMs. Yet, there is little knowledge about how developers distribute SBOMs. In this work, we mine SBOMs from Maven Central to assess the extent to which developers publish SBOMs along with the artifacts. We develop our work on top of the Goblin framework, which consists of a Maven Central dependency graph and a Weaver that allows augmenting the dependency graph with additional data. For this study, we select a sample of 10% of release nodes from the Maven Central dependency graph and collected 14,071 SBOMs from 7,290 package releases. We then augment the Maven Central dependency graph with the collected SBOMs. We present our methodology to mine SBOMs, as well as novel insights about SBOM publication. Our dataset is the first set of SBOMs collected from a package registry. We make it available as a standalone dataset, which can be used for future research about SBOMs and package distribution.
△ Less
Submitted 23 January, 2025;
originally announced January 2025.
-
Test Case-Informed Knowledge Tracing for Open-ended Coding Tasks
Authors:
Zhangqi Duan,
Nigel Fernandez,
Alexander Hicks,
Andrew Lan
Abstract:
Open-ended coding tasks, which ask students to construct programs according to certain specifications, are common in computer science education. Student modeling can be challenging since their open-ended nature means that student code can be diverse. Traditional knowledge tracing (KT) models that only analyze response correctness may not fully capture nuances in student knowledge from student code…
▽ More
Open-ended coding tasks, which ask students to construct programs according to certain specifications, are common in computer science education. Student modeling can be challenging since their open-ended nature means that student code can be diverse. Traditional knowledge tracing (KT) models that only analyze response correctness may not fully capture nuances in student knowledge from student code. In this paper, we introduce Test case-Informed Knowledge Tracing for Open-ended Coding (TIKTOC), a framework to simultaneously analyze and predict both open-ended student code and whether the code passes each test case. We augment the existing CodeWorkout dataset with the test cases used for a subset of the open-ended coding questions, and propose a multi-task learning KT method to simultaneously analyze and predict 1) whether a student's code submission passes each test case and 2) the student's open-ended code, using a large language model as the backbone. We quantitatively show that these methods outperform existing KT methods for coding that only use the overall score a code submission receives. We also qualitatively demonstrate how test case information, combined with open-ended code, helps us gain fine-grained insights into student knowledge.
△ Less
Submitted 20 December, 2024; v1 submitted 27 September, 2024;
originally announced October 2024.
-
DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions
Authors:
Nigel Fernandez,
Alexander Scarlatos,
Wanyong Feng,
Simon Woodhead,
Andrew Lan
Abstract:
High-quality distractors are crucial to both the assessment and pedagogical value of multiple-choice questions (MCQs), where manually crafting ones that anticipate knowledge deficiencies or misconceptions among real students is difficult. Meanwhile, automated distractor generation, even with the help of large language models (LLMs), remains challenging for subjects like math. It is crucial to not…
▽ More
High-quality distractors are crucial to both the assessment and pedagogical value of multiple-choice questions (MCQs), where manually crafting ones that anticipate knowledge deficiencies or misconceptions among real students is difficult. Meanwhile, automated distractor generation, even with the help of large language models (LLMs), remains challenging for subjects like math. It is crucial to not only identify plausible distractors but also understand the error behind them. In this paper, we introduce DiVERT (Distractor Generation with Variational Errors Represented as Text), a novel variational approach that learns an interpretable representation of errors behind distractors in math MCQs. Through experiments on a real-world math MCQ dataset with 1,434 questions used by hundreds of thousands of students, we show that DiVERT, despite using a base open-source LLM with 7B parameters, outperforms state-of-the-art approaches using GPT-4o on downstream distractor generation. We also conduct a human evaluation with math educators and find that DiVERT leads to error labels that are of comparable quality to human-authored ones.
△ Less
Submitted 7 October, 2024; v1 submitted 27 June, 2024;
originally announced June 2024.
-
Interpreting Latent Student Knowledge Representations in Programming Assignments
Authors:
Nigel Fernandez,
Andrew Lan
Abstract:
Recent advances in artificial intelligence for education leverage generative large language models, including using them to predict open-ended student responses rather than their correctness only. However, the black-box nature of these models limits the interpretability of the learned student knowledge representations. In this paper, we conduct a first exploration into interpreting latent student…
▽ More
Recent advances in artificial intelligence for education leverage generative large language models, including using them to predict open-ended student responses rather than their correctness only. However, the black-box nature of these models limits the interpretability of the learned student knowledge representations. In this paper, we conduct a first exploration into interpreting latent student knowledge representations by presenting InfoOIRT, an Information regularized Open-ended Item Response Theory model, which encourages the latent student knowledge states to be interpretable while being able to generate student-written code for open-ended programming questions. InfoOIRT maximizes the mutual information between a fixed subset of latent knowledge states enforced with simple prior distributions and generated student code, which encourages the model to learn disentangled representations of salient syntactic and semantic code features including syntactic styles, mastery of programming skills, and code structures. Through experiments on a real-world programming education dataset, we show that InfoOIRT can both accurately generate student code and lead to interpretable student knowledge representations.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
The Sustainability Assessment Framework Toolkit: A Decade of Modeling Experience
Authors:
Patricia Lago,
Nelly Condori Fernandez,
Iffat Fatima,
Markus Funke,
Ivano Malavolta
Abstract:
Software intensive systems play a crucial role in most, if not all, aspects of modern society. As such, both their sustainability and their role in supporting sustainable processes, must be realized by design. To this aim, the architecture of software intensive systems should be designed to support sustainability goals; and measured to understand how effectively they do so. In this paper, we prese…
▽ More
Software intensive systems play a crucial role in most, if not all, aspects of modern society. As such, both their sustainability and their role in supporting sustainable processes, must be realized by design. To this aim, the architecture of software intensive systems should be designed to support sustainability goals; and measured to understand how effectively they do so. In this paper, we present the Sustainability Assessment Framework (SAF) Toolkit -- a set of instruments we developed to support software architects and design decision makers in modeling sustainability as a software quality property. The SAF Toolkit is the result of our experience gained in over a decade of cases in collaboration with industrial partners. We illustrate the toolkit with examples stemming from various cases. We extract our lessons learned, and our current research and future plans to extend the SAF Toolkit for further architecture modeling and measurement.
△ Less
Submitted 19 October, 2024; v1 submitted 2 May, 2024;
originally announced May 2024.
-
SyllabusQA: A Course Logistics Question Answering Dataset
Authors:
Nigel Fernandez,
Alexander Scarlatos,
Andrew Lan
Abstract:
Automated teaching assistants and chatbots have significant potential to reduce the workload of human instructors, especially for logistics-related question answering, which is important to students yet repetitive for instructors. However, due to privacy concerns, there is a lack of publicly available datasets. We introduce SyllabusQA, an open-source dataset with 63 real course syllabi covering 36…
▽ More
Automated teaching assistants and chatbots have significant potential to reduce the workload of human instructors, especially for logistics-related question answering, which is important to students yet repetitive for instructors. However, due to privacy concerns, there is a lack of publicly available datasets. We introduce SyllabusQA, an open-source dataset with 63 real course syllabi covering 36 majors, containing 5,078 open-ended course logistics-related question-answer pairs that are diverse in both question types and answer formats. Since many logistics-related questions contain critical information like the date of an exam, it is important to evaluate the factuality of answers. We benchmark several strong baselines on this task, from large language model prompting to retrieval-augmented generation. We introduce Fact-QA, an LLM-based (GPT-4) evaluation metric to evaluate the factuality of predicted answers. We find that despite performing close to humans on traditional metrics of textual similarity, there remains a significant gap between automated approaches and humans in terms of fact precision.
△ Less
Submitted 22 July, 2024; v1 submitted 2 March, 2024;
originally announced March 2024.
-
Pixel-Wise Recognition for Holistic Surgical Scene Understanding
Authors:
Nicolás Ayobi,
Santiago Rodríguez,
Alejandra Pérez,
Isabela Hernández,
Nicolás Aparicio,
Eugénie Dessevres,
Sebastián Peña,
Jessica Santander,
Juan Ignacio Caicedo,
Nicolás Fernández,
Pablo Arbeláez
Abstract:
This paper presents the Holistic and Multi-Granular Surgical Scene Understanding of Prostatectomies (GraSP) dataset, a curated benchmark that models surgical scene understanding as a hierarchy of complementary tasks with varying levels of granularity. Our approach enables a multi-level comprehension of surgical activities, encompassing long-term tasks such as surgical phases and steps recognition…
▽ More
This paper presents the Holistic and Multi-Granular Surgical Scene Understanding of Prostatectomies (GraSP) dataset, a curated benchmark that models surgical scene understanding as a hierarchy of complementary tasks with varying levels of granularity. Our approach enables a multi-level comprehension of surgical activities, encompassing long-term tasks such as surgical phases and steps recognition and short-term tasks including surgical instrument segmentation and atomic visual actions detection. To exploit our proposed benchmark, we introduce the Transformers for Actions, Phases, Steps, and Instrument Segmentation (TAPIS) model, a general architecture that combines a global video feature extractor with localized region proposals from an instrument segmentation model to tackle the multi-granularity of our benchmark. Through extensive experimentation, we demonstrate the impact of including segmentation annotations in short-term recognition tasks, highlight the varying granularity requirements of each task, and establish TAPIS's superiority over previously proposed baselines and conventional CNN-based models. Additionally, we validate the robustness of our method across multiple public benchmarks, confirming the reliability and applicability of our dataset. This work represents a significant step forward in Endoscopic Vision, offering a novel and comprehensive framework for future research towards a holistic understanding of surgical procedures.
△ Less
Submitted 25 January, 2024; v1 submitted 20 January, 2024;
originally announced January 2024.
-
3HAN: A Deep Neural Network for Fake News Detection
Authors:
Sneha Singhania,
Nigel Fernandez,
Shrisha Rao
Abstract:
The rapid spread of fake news is a serious problem calling for AI solutions. We employ a deep learning based automated detector through a three level hierarchical attention network (3HAN) for fast, accurate detection of fake news. 3HAN has three levels, one each for words, sentences, and the headline, and constructs a news vector: an effective representation of an input news article, by processing…
▽ More
The rapid spread of fake news is a serious problem calling for AI solutions. We employ a deep learning based automated detector through a three level hierarchical attention network (3HAN) for fast, accurate detection of fake news. 3HAN has three levels, one each for words, sentences, and the headline, and constructs a news vector: an effective representation of an input news article, by processing an article in an hierarchical bottom-up manner. The headline is known to be a distinguishing feature of fake news, and furthermore, relatively few words and sentences in an article are more important than the rest. 3HAN gives a differential importance to parts of an article, on account of its three layers of attention. By experiments on a large real-world data set, we observe the effectiveness of 3HAN with an accuracy of 96.77%. Unlike some other deep learning models, 3HAN provides an understandable output through the attention weights given to different parts of an article, which can be visualized through a heatmap to enable further manual fact checking.
△ Less
Submitted 21 June, 2023;
originally announced June 2023.
-
Improving Reading Comprehension Question Generation with Data Augmentation and Overgenerate-and-rank
Authors:
Nischal Ashok Kumar,
Nigel Fernandez,
Zichao Wang,
Andrew Lan
Abstract:
Reading comprehension is a crucial skill in many aspects of education, including language learning, cognitive development, and fostering early literacy skills in children. Automated answer-aware reading comprehension question generation has significant potential to scale up learner support in educational activities. One key technical challenge in this setting is that there can be multiple question…
▽ More
Reading comprehension is a crucial skill in many aspects of education, including language learning, cognitive development, and fostering early literacy skills in children. Automated answer-aware reading comprehension question generation has significant potential to scale up learner support in educational activities. One key technical challenge in this setting is that there can be multiple questions, sometimes very different from each other, with the same answer; a trained question generation method may not necessarily know which question human educators would prefer. To address this challenge, we propose 1) a data augmentation method that enriches the training dataset with diverse questions given the same context and answer and 2) an overgenerate-and-rank method to select the best question from a pool of candidates. We evaluate our method on the FairytaleQA dataset, showing a 5% absolute improvement in ROUGE-L over the best existing method. We also demonstrate the effectiveness of our method in generating harder, "implicit" questions, where the answers are not contained in the context as text spans.
△ Less
Submitted 15 June, 2023;
originally announced June 2023.
-
SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models
Authors:
Martin Gonzalez,
Nelson Fernandez,
Thuy Tran,
Elies Gherbi,
Hatem Hajri,
Nader Masmoudi
Abstract:
A potent class of generative models known as Diffusion Probabilistic Models (DPMs) has become prominent. A forward diffusion process adds gradually noise to data, while a model learns to gradually denoise. Sampling from pre-trained DPMs is obtained by solving differential equations (DE) defined by the learnt model, a process which has shown to be prohibitively slow. Numerous efforts on speeding-up…
▽ More
A potent class of generative models known as Diffusion Probabilistic Models (DPMs) has become prominent. A forward diffusion process adds gradually noise to data, while a model learns to gradually denoise. Sampling from pre-trained DPMs is obtained by solving differential equations (DE) defined by the learnt model, a process which has shown to be prohibitively slow. Numerous efforts on speeding-up this process have consisted on crafting powerful ODE solvers. Despite being quick, such solvers do not usually reach the optimal quality achieved by available slow SDE solvers. Our goal is to propose SDE solvers that reach optimal quality without requiring several hundreds or thousands of NFEs to achieve that goal. We propose Stochastic Explicit Exponential Derivative-free Solvers (SEEDS), improving and generalizing Exponential Integrator approaches to the stochastic case on several frameworks. After carefully analyzing the formulation of exact solutions of diffusion SDEs, we craft SEEDS to analytically compute the linear part of such solutions. Inspired by the Exponential Time-Differencing method, SEEDS use a novel treatment of the stochastic components of solutions, enabling the analytical computation of their variance, and contains high-order terms allowing to reach optimal quality sampling $\sim3$-$5\times$ faster than previous SDE methods. We validate our approach on several image generation benchmarks, showing that SEEDS outperform or are competitive with previous SDE solvers. Contrary to the latter, SEEDS are derivative and training free, and we fully prove strong convergence guarantees for them.
△ Less
Submitted 26 October, 2023; v1 submitted 23 May, 2023;
originally announced May 2023.
-
Towards Holistic Surgical Scene Understanding
Authors:
Natalia Valderrama,
Paola Ruiz Puentes,
Isabela Hernández,
Nicolás Ayobi,
Mathilde Verlyk,
Jessica Santander,
Juan Caicedo,
Nicolás Fernández,
Pablo Arbeláez
Abstract:
Most benchmarks for studying surgical interventions focus on a specific challenge instead of leveraging the intrinsic complementarity among different tasks. In this work, we present a new experimental framework towards holistic surgical scene understanding. First, we introduce the Phase, Step, Instrument, and Atomic Visual Action recognition (PSI-AVA) Dataset. PSI-AVA includes annotations for both…
▽ More
Most benchmarks for studying surgical interventions focus on a specific challenge instead of leveraging the intrinsic complementarity among different tasks. In this work, we present a new experimental framework towards holistic surgical scene understanding. First, we introduce the Phase, Step, Instrument, and Atomic Visual Action recognition (PSI-AVA) Dataset. PSI-AVA includes annotations for both long-term (Phase and Step recognition) and short-term reasoning (Instrument detection and novel Atomic Action recognition) in robot-assisted radical prostatectomy videos. Second, we present Transformers for Action, Phase, Instrument, and steps Recognition (TAPIR) as a strong baseline for surgical scene understanding. TAPIR leverages our dataset's multi-level annotations as it benefits from the learned representation on the instrument detection task to improve its classification capacity. Our experimental results in both PSI-AVA and other publicly available databases demonstrate the adequacy of our framework to spur future research on holistic surgical scene understanding.
△ Less
Submitted 25 January, 2024; v1 submitted 8 December, 2022;
originally announced December 2022.
-
Is Soccer a lie or simply a complex system?
Authors:
Nelson Fernandez,
Ricardo Bernal
Abstract:
Understanding soccer as a complex system we base on nature and the collective behavior of many organisms that "do calculations," seeking to generate solutions in a bioinspired way. When soccer mysteries appear, complex systems science emerges as a means to provide explanations. However, given the variety of interpretations that complexity and its associated properties can have and the understandin…
▽ More
Understanding soccer as a complex system we base on nature and the collective behavior of many organisms that "do calculations," seeking to generate solutions in a bioinspired way. When soccer mysteries appear, complex systems science emerges as a means to provide explanations. However, given the variety of interpretations that complexity and its associated properties can have and the understanding of what a complex system is, it is convenient to provide some elements to understand how unpredictability in soccer gives way to hundreds of counterintuitive results and how the science of complexity could contribute to the understanding of many phenomena in this sport. In this context, the manuscript's objective is to synthetically address some of the most important aspects of applied complexity to soccer to bring science and sport closer together
△ Less
Submitted 20 July, 2022; v1 submitted 16 July, 2022;
originally announced July 2022.
-
Automated Scoring for Reading Comprehension via In-context BERT Tuning
Authors:
Nigel Fernandez,
Aritra Ghosh,
Naiming Liu,
Zichao Wang,
Benoît Choffin,
Richard Baraniuk,
Andrew Lan
Abstract:
Automated scoring of open-ended student responses has the potential to significantly reduce human grader effort. Recent advances in automated scoring often leverage textual representations based on pre-trained language models such as BERT and GPT as input to scoring models. Most existing approaches train a separate model for each item/question, which is suitable for scenarios such as essay scoring…
▽ More
Automated scoring of open-ended student responses has the potential to significantly reduce human grader effort. Recent advances in automated scoring often leverage textual representations based on pre-trained language models such as BERT and GPT as input to scoring models. Most existing approaches train a separate model for each item/question, which is suitable for scenarios such as essay scoring where items can be quite different from one another. However, these approaches have two limitations: 1) they fail to leverage item linkage for scenarios such as reading comprehension where multiple items may share a reading passage; 2) they are not scalable since storing one model per item becomes difficult when models have a large number of parameters. In this paper, we report our (grand prize-winning) solution to the National Assessment of Education Progress (NAEP) automated scoring challenge for reading comprehension. Our approach, in-context BERT fine-tuning, produces a single shared scoring model for all items with a carefully-designed input structure to provide contextual information on each item. We demonstrate the effectiveness of our approach via local evaluations using the training dataset provided by the challenge. We also discuss the biases, common error types, and limitations of our approach.
△ Less
Submitted 15 June, 2023; v1 submitted 19 May, 2022;
originally announced May 2022.
-
A Convolutional Neural Network for gaze preference detection: A potential tool for diagnostics of autism spectrum disorder in children
Authors:
Dennis Núñez Fernández,
Franklin Barrientos Porras,
Robert H. Gilman,
Macarena Vittet Mondonedo,
Patricia Sheen,
Mirko Zimic
Abstract:
Early diagnosis of autism spectrum disorder (ASD) is known to improve the quality of life of affected individuals. However, diagnosis is often delayed even in wealthier countries including the US, largely due to the fact that gold standard diagnostic tools such as the Autism Diagnostic Observation Schedule (ADOS) and the Autism Diagnostic Interview-Revised (ADI-R) are time consuming and require ex…
▽ More
Early diagnosis of autism spectrum disorder (ASD) is known to improve the quality of life of affected individuals. However, diagnosis is often delayed even in wealthier countries including the US, largely due to the fact that gold standard diagnostic tools such as the Autism Diagnostic Observation Schedule (ADOS) and the Autism Diagnostic Interview-Revised (ADI-R) are time consuming and require expertise to administer. This trend is even more pronounced lower resources settings due to a lack of trained experts. As a result, alternative, less technical methods that leverage the unique ways in which children with ASD react to visual stimulation in a controlled environment have been developed to help facilitate early diagnosis. Previous studies have shown that, when exposed to a video that presents both social and abstract scenes side by side, a child with ASD will focus their attention towards the abstract images on the screen to a greater extent than a child without ASD. Such differential responses make it possible to implement an algorithm for the rapid diagnosis of ASD based on eye tracking against different visual stimuli. Here we propose a convolutional neural network (CNN) algorithm for gaze prediction using images extracted from a one-minute stimulus video. Our model achieved a high accuracy rate and robustness for prediction of gaze direction with independent persons and employing a different camera than the one used during testing. In addition to this, the proposed algorithm achieves a fast response time, providing a near real-time evaluation of ASD. Thereby, by applying the proposed method, we could significantly reduce the diagnosis time and facilitate the diagnosis of ASD in low resource regions.
△ Less
Submitted 28 July, 2020;
originally announced July 2020.
-
Synthesizing Tasks for Block-based Programming
Authors:
Umair Z. Ahmed,
Maria Christakis,
Aleksandr Efremov,
Nigel Fernandez,
Ahana Ghosh,
Abhik Roychoudhury,
Adish Singla
Abstract:
Block-based visual programming environments play a critical role in introducing computing concepts to K-12 students. One of the key pedagogical challenges in these environments is in designing new practice tasks for a student that match a desired level of difficulty and exercise specific programming concepts. In this paper, we formalize the problem of synthesizing visual programming tasks. In part…
▽ More
Block-based visual programming environments play a critical role in introducing computing concepts to K-12 students. One of the key pedagogical challenges in these environments is in designing new practice tasks for a student that match a desired level of difficulty and exercise specific programming concepts. In this paper, we formalize the problem of synthesizing visual programming tasks. In particular, given a reference visual task $\rm T^{in}$ and its solution code $\rm C^{in}$, we propose a novel methodology to automatically generate a set $\{(\rm T^{out}, \rm C^{out})\}$ of new tasks along with solution codes such that tasks $\rm T^{in}$ and $\rm T^{out}$ are conceptually similar but visually dissimilar. Our methodology is based on the realization that the mapping from the space of visual tasks to their solution codes is highly discontinuous; hence, directly mutating reference task $\rm T^{in}$ to generate new tasks is futile. Our task synthesis algorithm operates by first mutating code $\rm C^{in}$ to obtain a set of codes $\{\rm C^{out}\}$. Then, the algorithm performs symbolic execution over a code $\rm C^{out}$ to obtain a visual task $\rm T^{out}$; this step uses the Monte Carlo Tree Search (MCTS) procedure to guide the search in the symbolic tree. We demonstrate the effectiveness of our algorithm through an extensive empirical evaluation and user study on reference tasks taken from the \emph{Hour of Code: Classic Maze} challenge by \emph{Code.org} and the \emph{Intro to Programming with Karel} course by \emph{CodeHS.com}.
△ Less
Submitted 4 November, 2020; v1 submitted 17 June, 2020;
originally announced June 2020.
-
Development of a hand pose recognition system on an embedded computer using CNNs
Authors:
Dennis Núñez Fernández
Abstract:
Demand of hand pose recognition systems are growing in the last years in technologies like human-machine interfaces. This work suggests an approach for hand pose recognition in embedded computers using hand tracking and CNNs. Results show a fast time response with an accuracy of 94.50% and low power consumption.
Demand of hand pose recognition systems are growing in the last years in technologies like human-machine interfaces. This work suggests an approach for hand pose recognition in embedded computers using hand tracking and CNNs. Results show a fast time response with an accuracy of 94.50% and low power consumption.
△ Less
Submitted 18 October, 2019;
originally announced October 2019.
-
Two-stream convolutional networks for end-to-end learning of self-driving cars
Authors:
Nelson Fernandez
Abstract:
We propose a methodology to extend the concept of Two-Stream Convolutional Networks to perform end-to-end learning for self-driving cars with temporal cues. The system has the ability to learn spatiotemporal features by simultaneously mapping raw images and pre-calculated optical flows directly to steering commands. Although optical flows encode temporal-rich information, we found that 2D-CNNs are…
▽ More
We propose a methodology to extend the concept of Two-Stream Convolutional Networks to perform end-to-end learning for self-driving cars with temporal cues. The system has the ability to learn spatiotemporal features by simultaneously mapping raw images and pre-calculated optical flows directly to steering commands. Although optical flows encode temporal-rich information, we found that 2D-CNNs are prone to capturing features only as spatial representations. We show how the use of Multitask Learning favors the learning of temporal features via inductive transfer from a shared spatiotemporal representation. Preliminary results demonstrate a competitive improvement of 30% in prediction accuracy and stability compared to widely used regression methods trained on the Comma.ai dataset.
△ Less
Submitted 17 December, 2018; v1 submitted 13 November, 2018;
originally announced November 2018.
-
Proportional Justified Representation
Authors:
Luis Sánchez-Fernández,
Edith Elkind,
Martin Lackner,
Norberto Fernández,
Jesús A. Fisteus,
Pablo Basanta Val,
Piotr Skowron
Abstract:
The goal of multi-winner elections is to choose a fixed-size committee based on voters' preferences. An important concern in this setting is representation: large groups of voters with cohesive preferences should be adequately represented by the election winners. Recently, Aziz et al. (2015a;2017) proposed two axioms that aim to capture this idea: justified representation (JR) and its strengthenin…
▽ More
The goal of multi-winner elections is to choose a fixed-size committee based on voters' preferences. An important concern in this setting is representation: large groups of voters with cohesive preferences should be adequately represented by the election winners. Recently, Aziz et al. (2015a;2017) proposed two axioms that aim to capture this idea: justified representation (JR) and its strengthening extended justified representation (EJR). In this paper, we extend the work of Aziz et al. in several directions. First, we answer an open question of Aziz et al., by showing that Reweighted Approval Voting satisfies JR for $k=3, 4, 5$, but fails it for $k\ge 6$. Second, we observe that EJR is incompatible with the Perfect Representation criterion, which is important for many applications of multi-winner voting, and propose a relaxation of EJR, which we call Proportional Justified Representation (PJR). PJR is more demanding than JR, but, unlike EJR, it is compatible with perfect representation, and a committee that provides PJR can be computed in polynomial time if the committee size divides the number of voters. Moreover, just like EJR, PJR can be used to characterize the classic PAV rule in the class of weighted PAV rules. On the other hand, we show that EJR provides stronger guarantees with respect to average voter satisfaction than PJR does.
△ Less
Submitted 29 November, 2016;
originally announced November 2016.
-
Architecting Time-Critical Big-Data Systems
Authors:
Pablo Basanta-Val,
Neil Audsley,
Andy Wellings,
Ian Gray,
Norberto Fernandez
Abstract:
- Current infrastructures for developing big-data applications are able to process --via big-data analytics-huge amounts of data, using clusters of machines that collaborate to perform parallel computations. However, current infrastructures were not designed to work with the requirements of time-critical applications; they are more focused on general-purpose applications rather than time-critical…
▽ More
- Current infrastructures for developing big-data applications are able to process --via big-data analytics-huge amounts of data, using clusters of machines that collaborate to perform parallel computations. However, current infrastructures were not designed to work with the requirements of time-critical applications; they are more focused on general-purpose applications rather than time-critical ones. Addressing this issue from the perspective of the real-time systems community, this paper considers time-critical big-data. It deals with the definition of a time-critical big-data system from the point of view of requirements, analyzing the specific characteristics of some popular big-data applications. This analysis is complemented by the challenges stemmed from the infrastructures that support the applications, proposing an architecture and offering initial performance patterns that connect application costs with infrastructure performance.
△ Less
Submitted 3 November, 2016;
originally announced November 2016.
-
The Maximin Support Method: An Extension of the D'Hondt Method to Approval-Based Multiwinner Elections
Authors:
Luis Sánchez-Fernández,
Norberto Fernández,
Jesús A. Fisteus,
Markus Brill
Abstract:
We propose the maximin support method, a novel extension of the D'Hondt apportionment method to approval-based multiwinner elections. The maximin support method is based on maximizing the support of the least supported elected candidate. It can be computed efficiently and satisfies (adjusted versions of) the main properties of the original D'Hondt method: house monotonicity, population monotonicit…
▽ More
We propose the maximin support method, a novel extension of the D'Hondt apportionment method to approval-based multiwinner elections. The maximin support method is based on maximizing the support of the least supported elected candidate. It can be computed efficiently and satisfies (adjusted versions of) the main properties of the original D'Hondt method: house monotonicity, population monotonicity, and proportional representation. We also establish a close relationship between the maximin support method and Phragmén's voting rules.
△ Less
Submitted 5 September, 2018; v1 submitted 17 September, 2016;
originally announced September 2016.
-
Multi-Agent Modeling of Dynamical Systems: A Self-organized, Emergent, Homeostatic and Autopoietic Approach
Authors:
Nelson Fernandez
Abstract:
This thesis presents the theoretical, conceptual and methodological aspects that support the modeling of dynamical systems (DS) by using several agents. The modeling approach permits the assessment of properties representing order, change, equilibrium, adaptability, and autonomy, in DS. The modeling processes were supported by a conceptual corpus regarding systems dynamics, multi-agent systems, gr…
▽ More
This thesis presents the theoretical, conceptual and methodological aspects that support the modeling of dynamical systems (DS) by using several agents. The modeling approach permits the assessment of properties representing order, change, equilibrium, adaptability, and autonomy, in DS. The modeling processes were supported by a conceptual corpus regarding systems dynamics, multi-agent systems, graph theory, and, particularly, the information theory. Besides to the specification of the dynamical systems as a computational network of agents, metrics that allow characterizing and assessing the inherent complexity of such systems were defined. As a result, properties associated with emergence, self-organization, complexity, homeostasis and autopoiesis were defined, formalized and measured. The validation of the underlying DS model was carried out on discrete systems (boolean networks and cellular automata) and ecological systems. The central contribution of this thesis was the development of a methodological approach for DS modeling. This approach includes a larger set of properties than in traditional studies, what allows us to deepen in questioning essential issues associated with the DS field. All this was achieved from a simple base of calculation and interpretation, which does not require advanced mathematical knowledge, and facilitates their application in different fields of science.
△ Less
Submitted 2 October, 2015;
originally announced June 2016.
-
Measuring the Complexity of Continuous Distributions
Authors:
Guillermo Santamaría-Bonfil,
Nelson Fernández,
Carlos Gershenson
Abstract:
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We find that a broad range of common parameters found in Gaussian and scale-free distributions present high complexity values. We also explore the relationship betwee…
▽ More
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. This allows us to calculate the complexity of phenomena for which distributions are known. We find that a broad range of common parameters found in Gaussian and scale-free distributions present high complexity values. We also explore the relationship between our measure of complexity and information adaptation.
△ Less
Submitted 2 November, 2015;
originally announced November 2015.
-
Measuring the Complexity of Self-organizing Traffic Lights
Authors:
Dario Zubillaga,
Geovany Cruz,
Luis Daniel Aguilar,
Jorge Zapotecatl,
Nelson Fernandez,
Jose Aguilar,
David A. Rosenblueth,
Carlos Gershenson
Abstract:
We apply measures of complexity, emergence and self-organization to an abstract city traffic model for comparing a traditional traffic coordination method with a self-organizing method in two scenarios: cyclic boundaries and non-orientable boundaries. We show that the measures are useful to identify and characterize different dynamical phases. It becomes clear that different operation regimes are…
▽ More
We apply measures of complexity, emergence and self-organization to an abstract city traffic model for comparing a traditional traffic coordination method with a self-organizing method in two scenarios: cyclic boundaries and non-orientable boundaries. We show that the measures are useful to identify and characterize different dynamical phases. It becomes clear that different operation regimes are required for different traffic demands. Thus, not only traffic is a non-stationary problem, which requires controllers to adapt constantly. Controllers must also change drastically the complexity of their behavior depending on the demand. Based on our measures, we can say that the self-organizing method achieves an adaptability level comparable to a living system.
△ Less
Submitted 2 February, 2014;
originally announced February 2014.
-
Information Measures of Complexity, Emergence, Self-organization, Homeostasis, and Autopoiesis
Authors:
Nelson Fernandez,
Carlos Maldonado,
Carlos Gershenson
Abstract:
This chapter reviews measures of emergence, self-organization, complexity, homeostasis, and autopoiesis based on information theory. These measures are derived from proposed axioms and tested in two case studies: random Boolean networks and an Arctic lake ecosystem.
Emergence is defined as the information a system or process produces. Self-organization is defined as the opposite of emergence, wh…
▽ More
This chapter reviews measures of emergence, self-organization, complexity, homeostasis, and autopoiesis based on information theory. These measures are derived from proposed axioms and tested in two case studies: random Boolean networks and an Arctic lake ecosystem.
Emergence is defined as the information a system or process produces. Self-organization is defined as the opposite of emergence, while complexity is defined as the balance between emergence and self-organization. Homeostasis reflects the stability of a system. Autopoiesis is defined as the ratio between the complexity of a system and the complexity of its environment. The proposed measures can be applied at different scales, which can be studied with multi-scale profiles.
△ Less
Submitted 31 July, 2013; v1 submitted 5 April, 2013;
originally announced April 2013.
-
Complexity and Information: Measuring Emergence, Self-organization, and Homeostasis at Multiple Scales
Authors:
Carlos Gershenson,
Nelson Fernandez
Abstract:
Concepts used in the scientific study of complex systems have become so widespread that their use and abuse has led to ambiguity and confusion in their meaning. In this paper we use information theory to provide abstract and concise measures of complexity, emergence, self-organization, and homeostasis. The purpose is to clarify the meaning of these concepts with the aid of the proposed formal meas…
▽ More
Concepts used in the scientific study of complex systems have become so widespread that their use and abuse has led to ambiguity and confusion in their meaning. In this paper we use information theory to provide abstract and concise measures of complexity, emergence, self-organization, and homeostasis. The purpose is to clarify the meaning of these concepts with the aid of the proposed formal measures. In a simplified version of the measures (focusing on the information produced by a system), emergence becomes the opposite of self-organization, while complexity represents their balance. Homeostasis can be seen as a measure of the stability of the system. We use computational experiments on random Boolean networks and elementary cellular automata to illustrate our measures at multiple scales.
△ Less
Submitted 10 August, 2012; v1 submitted 9 May, 2012;
originally announced May 2012.
