-
LegalScore: Development of a Benchmark for Evaluating AI Models in Legal Career Exams in Brazil
Authors:
Roberto Caparroz,
Marcelo Roitman,
Beatriz G. Chow,
Caroline Giusti,
Larissa Torhacs,
Pedro A. Sola,
João H. M. Diogo,
Luiza Balby,
Carolina D. L. Vasconcelos,
Leonardo R. Caparroz,
Albano P. Franco
Abstract:
This research introduces LegalScore, a specialized index for assessing how generative artificial intelligence models perform in a selected range of career exams that require a legal background in Brazil. The index evaluates fourteen different types of artificial intelligence models' performance, from proprietary to open-source models, in answering objective questions applied to these exams. The re…
▽ More
This research introduces LegalScore, a specialized index for assessing how generative artificial intelligence models perform in a selected range of career exams that require a legal background in Brazil. The index evaluates fourteen different types of artificial intelligence models' performance, from proprietary to open-source models, in answering objective questions applied to these exams. The research uncovers the response of the models when applying English-trained large language models to Brazilian legal contexts, leading us to reflect on the importance and the need for Brazil-specific training data in generative artificial intelligence models. Performance analysis shows that while proprietary and most known models achieved better results overall, local and smaller models indicated promising performances due to their Brazilian context alignment in training. By establishing an evaluation framework with metrics including accuracy, confidence intervals, and normalized scoring, LegalScore enables systematic assessment of artificial intelligence performance in legal examinations in Brazil. While the study demonstrates artificial intelligence's potential value for exam preparation and question development, it concludes that significant improvements are needed before AI can match human performance in advanced legal assessments. The benchmark creates a foundation for continued research, highlighting the importance of local adaptation in artificial intelligence development.
△ Less
Submitted 17 January, 2025;
originally announced February 2025.
-
Unsupervised stratification of patients with myocardial infarction based on imaging and in-silico biomarkers
Authors:
Dolors Serra,
Pau Romero,
Paula Franco,
Ignacio Bernat,
Miguel Lozano,
Ignacio Garcia-Fernandez,
David Soto,
Antonio Berruezo,
Oscar Camara,
Rafael Sebastian
Abstract:
This study presents a novel methodology for stratifying post-myocardial infarction patients at risk of ventricular arrhythmias using patient-specific 3D cardiac models derived from late gadolinium enhancement cardiovascular magnetic resonance (LGE-CMR) images. The method integrates imaging and computational simulation with a simplified cellular automaton model, Arrhythmic3D, enabling rapid and acc…
▽ More
This study presents a novel methodology for stratifying post-myocardial infarction patients at risk of ventricular arrhythmias using patient-specific 3D cardiac models derived from late gadolinium enhancement cardiovascular magnetic resonance (LGE-CMR) images. The method integrates imaging and computational simulation with a simplified cellular automaton model, Arrhythmic3D, enabling rapid and accurate VA risk assessment in clinical timeframes. Applied to 51 patients, the model generated thousands of personalized simulations to evaluate arrhythmia inducibility and predict VA risk. Key findings include the identification of slow conduction channels (SCCs) within scar tissue as critical to reentrant arrhythmias and the localization of high-risk zones for potential intervention. The Arrhythmic Risk Score (ARRISK), developed from simulation results, demonstrated strong concordance with clinical outcomes and outperformed traditional imaging-based risk stratification. The methodology is fully automated, requiring minimal user intervention, and offers a promising tool for improving precision medicine in cardiac care by enhancing patient-specific arrhythmia risk assessment and guiding treatment strategies.
△ Less
Submitted 10 September, 2024;
originally announced September 2024.
-
Machine Learning Pipeline for Pulsar Star Dataset
Authors:
Alexander Ylnner Choquenaira Florez,
Braulio Valentin Sanchez Vinces,
Diana Carolina Roca Arroyo,
Josimar Edinson Chire Saire,
Patrıcia Batista Franco
Abstract:
This work brings together some of the most common machine learning (ML) algorithms, and the objective is to make a comparison at the level of obtained results from a set of unbalanced data. This dataset is composed of almost 17 thousand observations made to astronomical objects to identify pulsars (HTRU2). The methodological proposal based on evaluating the accuracy of these different models on th…
▽ More
This work brings together some of the most common machine learning (ML) algorithms, and the objective is to make a comparison at the level of obtained results from a set of unbalanced data. This dataset is composed of almost 17 thousand observations made to astronomical objects to identify pulsars (HTRU2). The methodological proposal based on evaluating the accuracy of these different models on the same database treated with two different strategies for unbalanced data. The results show that in spite of the noise and unbalance of classes present in this type of data, it is possible to apply them on standard ML algorithms and obtain promising accuracy ratios.
△ Less
Submitted 3 May, 2020;
originally announced May 2020.
-
The Unit-Demand Envy-Free Pricing Problem
Authors:
Cristina G. Fernandes,
Carlos E. Ferreira,
Álvaro J. P. Franco,
Rafael C. S. Schouery
Abstract:
We consider the unit-demand envy-free pricing problem, which is a unit-demand auction where each bidder receives an item that maximizes his utility, and the goal is to maximize the auctioneer's profit. This problem is NP-hard and unlikely to be in APX. We present four new MIP formulations for it and experimentally compare them to a previous one due to Shioda, Tunçel, and Myklebust. We describe thr…
▽ More
We consider the unit-demand envy-free pricing problem, which is a unit-demand auction where each bidder receives an item that maximizes his utility, and the goal is to maximize the auctioneer's profit. This problem is NP-hard and unlikely to be in APX. We present four new MIP formulations for it and experimentally compare them to a previous one due to Shioda, Tunçel, and Myklebust. We describe three models to generate different random instances for general unit-demand auctions, that we designed for the computational experiments. Each model has a nice economic interpretation. Aiming approximation results, we consider the variant of the problem where the item prices are restricted to be chosen from a geometric series, and prove that an optimal solution for this variant has value that is a fraction (depending on the series used) of the optimal value of the original problem. So this variant is also unlikely to be in APX.
△ Less
Submitted 30 September, 2013;
originally announced October 2013.
-
Algorithms for Junctions in Directed Acyclic Graphs
Authors:
Carlos Eduardo Ferreira,
Álvaro Junio Pereira Franco
Abstract:
Given a pair of distinct vertices u, v in a graph G, we say that s is a junction of u, v if there are in G internally vertex disjoint directed paths from s to u and from s to v. We show how to characterize junctions in directed acyclic graphs. We also consider the two problems in the following and derive efficient algorithms to solve them. Given a directed acyclic graph G and a vertex s in G, how…
▽ More
Given a pair of distinct vertices u, v in a graph G, we say that s is a junction of u, v if there are in G internally vertex disjoint directed paths from s to u and from s to v. We show how to characterize junctions in directed acyclic graphs. We also consider the two problems in the following and derive efficient algorithms to solve them. Given a directed acyclic graph G and a vertex s in G, how can we find all pairs of vertices of G such that s is a junction of them? And given a directed acyclic graph G and k pairs of vertices of G, how can we preprocess G such that all junctions of k given pairs of vertices could be listed quickly? All junctions of k pairs problem arises in an application in Anthropology and we apply our algorithm to find such junctions on kinship networks of some brazilian indian ethnic groups.
△ Less
Submitted 13 April, 2012;
originally announced April 2012.
