Showing 1–7 of 7 results for author: Sharp, M

Predicting Parkinson's disease trajectory using clinical and functional MRI features: a reproduction and replication study

Authors: Elodie Germani, Nikhil Baghwat, Mathieu Dugré, Rémi Gau, Albert Montillo, Kevin Nguyen, Andrzej Sokolowski, Madeleine Sharp, Jean-Baptiste Poline, Tristan Glatard

Abstract: Parkinson's disease (PD) is a common neurodegenerative disorder with a poorly understood physiopathology and no established biomarkers for the diagnosis of early stages and for prediction of disease progression. Several neuroimaging biomarkers have been studied recently, but these are susceptible to several sources of variability related for instance to cohort selection or image analysis. In this… ▽ More Parkinson's disease (PD) is a common neurodegenerative disorder with a poorly understood physiopathology and no established biomarkers for the diagnosis of early stages and for prediction of disease progression. Several neuroimaging biomarkers have been studied recently, but these are susceptible to several sources of variability related for instance to cohort selection or image analysis. In this context, an evaluation of the robustness of such biomarkers to variations in the data processing workflow is essential. This study is part of a larger project investigating the replicability of potential neuroimaging biomarkers of PD. Here, we attempt to reproduce (re-implementing the experiments with the same data, same method) and replicate (different data and/or method) the models described in [1] to predict individual's PD current state and progression using demographic, clinical and neuroimaging features (fALFF and ReHo extracted from resting-state fMRI). We use the Parkinson's Progression Markers Initiative dataset (PPMI, ppmi-info.org), as in [1] and aim to reproduce the original cohort, imaging features and machine learning models as closely as possible using the information available in the paper and the code. We also investigated methodological variations in cohort selection, feature extraction pipelines and sets of input features. Different criteria were used to evaluate the reproduction and compare the reproduced results with the original ones. Notably, we obtained significantly better than chance performance using the analysis pipeline closest to that in the original study (R2 \> 0), which is consistent with its findings. Moreover, using derived data provided by the authors of the original study, we were able to make an exact reproduction and managed to obtain results that were close to the original ones. The challenges encountered while reproducing and replicating the original work are likely explained by the complexity of neuroimaging studies, in particular in clinical settings. We provide recommendations to further facilitate the reproducibility of such studies in the future. △ Less

Submitted 12 February, 2025; v1 submitted 20 February, 2024; originally announced March 2024.

Comments: PLoS ONE, In press

RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture

Authors: Angels Balaguer, Vinamra Benara, Renato Luiz de Freitas Cunha, Roberto de M. Estevão Filho, Todd Hendry, Daniel Holstein, Jennifer Marsman, Nick Mecklenburg, Sara Malvar, Leonardo O. Nunes, Rafael Padilha, Morris Sharp, Bruno Silva, Swati Sharma, Vijay Aski, Ranveer Chandra

Abstract: There are two common ways in which developers are incorporating proprietary and domain-specific data when building applications of Large Language Models (LLMs): Retrieval-Augmented Generation (RAG) and Fine-Tuning. RAG augments the prompt with the external data, while fine-Tuning incorporates the additional knowledge into the model itself. However, the pros and cons of both approaches are not well… ▽ More There are two common ways in which developers are incorporating proprietary and domain-specific data when building applications of Large Language Models (LLMs): Retrieval-Augmented Generation (RAG) and Fine-Tuning. RAG augments the prompt with the external data, while fine-Tuning incorporates the additional knowledge into the model itself. However, the pros and cons of both approaches are not well understood. In this paper, we propose a pipeline for fine-tuning and RAG, and present the tradeoffs of both for multiple popular LLMs, including Llama2-13B, GPT-3.5, and GPT-4. Our pipeline consists of multiple stages, including extracting information from PDFs, generating questions and answers, using them for fine-tuning, and leveraging GPT-4 for evaluating the results. We propose metrics to assess the performance of different stages of the RAG and fine-Tuning pipeline. We conduct an in-depth study on an agricultural dataset. Agriculture as an industry has not seen much penetration of AI, and we study a potentially disruptive application - what if we could provide location-specific insights to a farmer? Our results show the effectiveness of our dataset generation pipeline in capturing geographic-specific knowledge, and the quantitative and qualitative benefits of RAG and fine-tuning. We see an accuracy increase of over 6 p.p. when fine-tuning the model and this is cumulative with RAG, which increases accuracy by 5 p.p. further. In one particular experiment, we also demonstrate that the fine-tuned model leverages information from across geographies to answer specific questions, increasing answer similarity from 47% to 72%. Overall, the results point to how systems built using LLMs can be adapted to respond and incorporate knowledge across a dimension that is critical for a specific industry, paving the way for further applications of LLMs in other industrial domains. △ Less

Submitted 30 January, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

KPI Extraction from Maintenance Work Orders -- A Comparison of Expert Labeling, Text Classification and AI-Assisted Tagging for Computing Failure Rates of Wind Turbines

Authors: Marc-Alexander Lutz, Bastian Schäfermeier, Rachael Sexton, Michael Sharp, Alden Dima, Stefan Faulstich, Jagan Mohini Aluri

Abstract: Maintenance work orders are commonly used to document information about wind turbine operation and maintenance. This includes details about proactive and reactive wind turbine downtimes, such as preventative and corrective maintenance. However, the information contained in maintenance work orders is often unstructured and difficult to analyze, presenting challenges for decision-makers wishing to u… ▽ More Maintenance work orders are commonly used to document information about wind turbine operation and maintenance. This includes details about proactive and reactive wind turbine downtimes, such as preventative and corrective maintenance. However, the information contained in maintenance work orders is often unstructured and difficult to analyze, presenting challenges for decision-makers wishing to use it for optimizing operation and maintenance. To address this issue, this work compares three different approaches to calculate reliability by performance indicators from maintenance work orders. The first approach involves manual labeling of the maintenance work orders by domain experts, using the schema defined in an industrial guideline to assign the label accordingly. The second approach involves the development of a model that automatically labels the maintenance work orders using text classification methods. Through this method, we are able to achieve macro average and weighted average F1-Scores of 0.75 and 0.85 respectively. The third technique uses an AI-assisted tagging tool to tag and structure the raw maintenance information, together with a novel rule-based approach for extracting relevant maintenance work orders for failure rate calculation. In our experiments the AI-assisted tool leads to a 88% drop in tagging time in comparison to the other two approaches, while expert labeling and text classification are more accurate in KPI extraction. Overall, our findings make extracting maintenance information from maintenance work orders more efficient, enable the assessment of reliability key performance indicators and therefore support the optimization of wind turbine operation and maintenance. △ Less

Submitted 6 December, 2023; v1 submitted 7 November, 2023; originally announced November 2023.

ACM Class: I.2.7; I.7

Journal ref: Energies (2023), 16, 7937

The Impacts of Dimensionality, Diffusion, and Directedness on Intrinsic Universality in the abstract Tile Assembly Model

Authors: Daniel Hader, Aaron Koch, Matthew J. Patitz, Michael Sharp

Abstract: We present a series of results related to mathematical models of self-assembling tiles and the impacts that three diverse properties have on their dynamics. We expand upon a series of prior results which showed that (1) the abstract Tile Assembly Model (aTAM) is intrinsically universal (IU) [FOCS 2012], and (2) the class of directed aTAM systems is not IU [FOCS 2016]. IU for a model (or class of s… ▽ More We present a series of results related to mathematical models of self-assembling tiles and the impacts that three diverse properties have on their dynamics. We expand upon a series of prior results which showed that (1) the abstract Tile Assembly Model (aTAM) is intrinsically universal (IU) [FOCS 2012], and (2) the class of directed aTAM systems is not IU [FOCS 2016]. IU for a model (or class of systems within a model) means that there is a universal tile set which can be used to simulate an arbitrary system within that model (or class). Furthermore, the simulation must not only produce the same resultant structures, it must also maintain the full dynamics of the systems being simulated modulo only a scale factor. While the FOCS 2012 result showed the standard, two-dimensional (2D) aTAM is IU, here we show this is also the case for the 3D version. Conversely, the FOCS 2016 result showed the class of aTAM systems which are directed (a.k.a. deterministic, or confluent) is not IU, implying that nondeterminism is fundamentally required for such simulations. Here, however, we show that in 3D the class of directed aTAM systems is actually IU, i.e. there is a universal directed simulator for them. We then consider the influence of more rigid notions of dimensionality. Namely, we introduce the Planar aTAM, where tiles are not only restricted to binding in the plane, but also to traveling in the plane, and prove that the Planar aTAM is not IU, and that the class of directed systems within the Planar aTAM also is not IU. Finally, analogous to the Planar aTAM, we introduce the Spatial aTAM, its 3D counterpart, and prove that it is IU. To prove our positive results we have not only designed, but also implemented what we believe to be the first IU tile set ever implemented and simulated in any tile assembly model. We've made it and a simulator which can demonstrate it freely available. △ Less

Submitted 9 October, 2019; originally announced October 2019.

Comments: A shortened version of this paper was accepted to SODA 2020

Anteater: Interactive Visualization of Program Execution Values in Context

Authors: Rebecca Faust, Katherine Isaacs, William Z. Bernstein, Michael Sharp, Carlos Scheidegger

Abstract: Debugging is famously one the hardest parts in programming. In this paper, we tackle the question: what does a debugging environment look like when we take interactive visualization as a central design principle? We introduce Anteater, an interactive visualization system for tracing and exploring the execution of Python programs. Existing systems often have visualization components built on top of… ▽ More Debugging is famously one the hardest parts in programming. In this paper, we tackle the question: what does a debugging environment look like when we take interactive visualization as a central design principle? We introduce Anteater, an interactive visualization system for tracing and exploring the execution of Python programs. Existing systems often have visualization components built on top of an existing infrastructure. In contrast, Anteater's organization of trace data enables an intermediate representation which can be leveraged to automatically synthesize a variety of visualizations and interactions. These interactive visualizations help with tasks such as discovering important structures in the execution and understanding and debugging unexpected behaviors. To assess the utility of Anteater, we conducted a participant study where programmers completed tasks on their own python programs using Anteater. Finally, we discuss limitations and where further research is needed. △ Less

Submitted 26 February, 2024; v1 submitted 5 July, 2019; originally announced July 2019.

Comments: 31 pages, 9 figures, 3 tables

Self-Assembly of 3-D Structures Using 2-D Folding Tiles

Authors: Jerome Durand-Lose, Jacob Hendricks, Matthew J. Patitz, Ian Perkins, Michael Sharp

Abstract: Self-assembly is a process which is ubiquitous in natural, especially biological systems. It occurs when groups of relatively simple components spontaneously combine to form more complex structures. While such systems have inspired a large amount of research into designing theoretical models of self-assembling systems, and even laboratory-based implementations of them, these artificial models and… ▽ More Self-assembly is a process which is ubiquitous in natural, especially biological systems. It occurs when groups of relatively simple components spontaneously combine to form more complex structures. While such systems have inspired a large amount of research into designing theoretical models of self-assembling systems, and even laboratory-based implementations of them, these artificial models and systems often tend to be lacking in one of the powerful features of natural systems (e.g. the assembly and folding of proteins), namely the dynamic reconfigurability of structures. In this paper, we present a new mathematical model of self-assembly, based on the abstract Tile Assembly Model (aTAM), called the Flexible Tile Assembly Model (FTAM). In the FTAM, the individual components are 2-dimensional square tiles as in the aTAM, but in the FTAM, bonds between the edges of tiles can be flexible, allowing bonds to flex and entire structures to reconfigure, thus allowing 2-dimensional components to form 3-dimensional structures. We analyze the powers and limitations of FTAM systems by (1) demonstrating how flexibility can be controlled to carefully build desired structures, and (2) showing how flexibility can be beneficially harnessed to form structures which can "efficiently" reconfigure into many different configurations and/or greatly varying configurations. We also show that with such power comes a heavy burden in terms of computational complexity of simulation and prediction by proving that, for important properties of FTAM systems, determining their existence is intractable, even for properties which are easily computed for systems in less dynamic models. △ Less

Submitted 17 July, 2018; v1 submitted 12 July, 2018; originally announced July 2018.

Thermodynamically Favorable Computation via Tile Self-assembly

Authors: Cameron Chalk, Jacob Hendricks, Matthew J. Patitz, Michael Sharp

Abstract: The recently introduced Thermodynamic Binding Networks (TBN) model was developed with the purpose of studying self-assembling systems by focusing on their thermodynamically favorable final states, and ignoring the kinetic pathways through which they evolve. The model was intentionally developed to abstract away not only the notion of time, but also the constraints of geometry. Collections of monom… ▽ More The recently introduced Thermodynamic Binding Networks (TBN) model was developed with the purpose of studying self-assembling systems by focusing on their thermodynamically favorable final states, and ignoring the kinetic pathways through which they evolve. The model was intentionally developed to abstract away not only the notion of time, but also the constraints of geometry. Collections of monomers with binding domains which allow them to form polymers via complementary bonds are analyzed to determine their final, stable configurations, which are those which maximize the number of bonds formed (i.e. enthalpy) and the number of independent components (i.e. entropy). In this paper, we first develop a definition of what it means for a TBN to perform a computation, and then present a set of constructions which are capable of performing computations by simulating the behaviors of space-bounded Turing machines and boolean circuits. In contrast to previous TBN results, these constructions are robust to great variability in the counts of monomers existing in the systems and the numbers of polymers that form in parallel. Although the Turing machine simulating TBNs are inefficient in terms of the numbers of unique monomer types required, as compared to algorithmic self-assembling systems in the abstract Tile Assembly Model (aTAM), we then show that a general strategy of porting those aTAM system designs to TBNs produces TBNs which incorrectly simulate computations. Finally, we present a refinement of the TBN model which we call the Geometric Thermodynamic Binding Networks (GTBN) model in which monomers are defined with rigid geometries and form rigid bonds. Utilizing the constraints imposed by geometry, we then provide a GTBN construction capable of simulating Turing machines as efficiently as in the aTAM. △ Less

Submitted 7 February, 2018; originally announced February 2018.