-
An ELIXIR scoping review on domain-specific evaluation metrics for synthetic data in life sciences
Authors:
Styliani-Christina Fragkouli,
Somya Iqbal,
Lisa Crossman,
Barbara Gravel,
Nagat Masued,
Mark Onders,
Devesh Haseja,
Alex Stikkelman,
Alfonso Valencia,
Tom Lenaerts,
Fotis Psomopoulos,
Pilib Ó Broin,
Núria Queralt-Rosinach,
Davide Cirillo
Abstract:
Synthetic data has emerged as a powerful resource in life sciences, offering solutions for data scarcity, privacy protection and accessibility constraints. By creating artificial datasets that mirror the characteristics of real data, allows researchers to develop and validate computational methods in controlled environments. Despite its promise, the adoption of synthetic data in Life Sciences hing…
▽ More
Synthetic data has emerged as a powerful resource in life sciences, offering solutions for data scarcity, privacy protection and accessibility constraints. By creating artificial datasets that mirror the characteristics of real data, allows researchers to develop and validate computational methods in controlled environments. Despite its promise, the adoption of synthetic data in Life Sciences hinges on rigorous evaluation metrics designed to assess their fidelity and reliability. To explore the current landscape of synthetic data evaluation metrics in several Life Sciences domains, the ELIXIR Machine Learning Focus Group performed a systematic review of the scientific literature following the PRISMA guidelines. Six critical domains were examined to identify current practices for assessing synthetic data. Findings reveal that, while generation methods are rapidly evolving, systematic evaluation is often overlooked, limiting researchers ability to compare, validate, and trust synthetic datasets across different domains. This systematic review underscores the urgent need for robust, standardized evaluation approaches that not only bolster confidence in synthetic data but also guide its effective and responsible implementation. By laying the groundwork for establishing domain-specific yet interoperable standards, this scoping review paves the way for future initiatives aimed at enhancing the role of synthetic data in scientific discovery, clinical practice and beyond.
△ Less
Submitted 17 June, 2025;
originally announced June 2025.
-
Open and Sustainable AI: challenges, opportunities and the road ahead in the life sciences
Authors:
Gavin Farrell,
Eleni Adamidi,
Rafael Andrade Buono,
Mihail Anton,
Omar Abdelghani Attafi,
Salvador Capella Gutierrez,
Emidio Capriotti,
Leyla Jael Castro,
Davide Cirillo,
Lisa Crossman,
Christophe Dessimoz,
Alexandros Dimopoulos,
Raul Fernandez-Diaz,
Styliani-Christina Fragkouli,
Carole Goble,
Wei Gu,
John M. Hancock,
Alireza Khanteymoori,
Tom Lenaerts,
Fabio G. Liberante,
Peter Maccallum,
Alexander Miguel Monzon,
Magnus Palmblad,
Lucy Poveda,
Ovidiu Radulescu
, et al. (5 additional authors not shown)
Abstract:
Artificial intelligence (AI) has recently seen transformative breakthroughs in the life sciences, expanding possibilities for researchers to interpret biological information at an unprecedented capacity, with novel applications and advances being made almost daily. In order to maximise return on the growing investments in AI-based life science research and accelerate this progress, it has become u…
▽ More
Artificial intelligence (AI) has recently seen transformative breakthroughs in the life sciences, expanding possibilities for researchers to interpret biological information at an unprecedented capacity, with novel applications and advances being made almost daily. In order to maximise return on the growing investments in AI-based life science research and accelerate this progress, it has become urgent to address the exacerbation of long-standing research challenges arising from the rapid adoption of AI methods. We review the increased erosion of trust in AI research outputs, driven by the issues of poor reusability and reproducibility, and highlight their consequent impact on environmental sustainability. Furthermore, we discuss the fragmented components of the AI ecosystem and lack of guiding pathways to best support Open and Sustainable AI (OSAI) model development. In response, this perspective introduces a practical set of OSAI recommendations directly mapped to over 300 components of the AI ecosystem. Our work connects researchers with relevant AI resources, facilitating the implementation of sustainable, reusable and transparent AI. Built upon life science community consensus and aligned to existing efforts, the outputs of this perspective are designed to aid the future development of policy and structured pathways for guiding AI implementation.
△ Less
Submitted 22 May, 2025;
originally announced May 2025.
-
DOME Registry: Implementing community-wide recommendations for reporting supervised machine learning in biology
Authors:
Omar Abdelghani Attafi,
Damiano Clementel,
Konstantinos Kyritsis,
Emidio Capriotti,
Gavin Farrell,
Styliani-Christina Fragkouli,
Leyla Jael Castro,
András Hatos,
Tom Lenaerts,
Stanislav Mazurenko,
Soroush Mozaffari,
Franco Pradelli,
Patrick Ruch,
Castrense Savojardo,
Paola Turina,
Federico Zambelli,
Damiano Piovesan,
Alexander Miguel Monzon,
Fotis Psomopoulos,
Silvio C. E. Tosatto
Abstract:
Supervised machine learning (ML) is used extensively in biology and deserves closer scrutiny. The DOME recommendations aim to enhance the validation and reproducibility of ML research by establishing standards for key aspects such as data handling and processing, optimization, evaluation, and model interpretability. The recommendations help to ensure that key details are reported transparently by…
▽ More
Supervised machine learning (ML) is used extensively in biology and deserves closer scrutiny. The DOME recommendations aim to enhance the validation and reproducibility of ML research by establishing standards for key aspects such as data handling and processing, optimization, evaluation, and model interpretability. The recommendations help to ensure that key details are reported transparently by providing a structured set of questions. Here, we introduce the DOME Registry (URL: registry.dome-ml.org), a database that allows scientists to manage and access comprehensive DOME-related information on published ML studies. The registry uses external resources like ORCID, APICURON and the Data Stewardship Wizard to streamline the annotation process and ensure comprehensive documentation. By assigning unique identifiers and DOME scores to publications, the registry fosters a standardized evaluation of ML methods. Future plans include continuing to grow the registry through community curation, improving the DOME score definition and encouraging publishers to adopt DOME standards, promoting transparency and reproducibility of ML in the life sciences.
△ Less
Submitted 16 August, 2024; v1 submitted 14 August, 2024;
originally announced August 2024.
-
Synthetic data: How could it be used for infectious disease research?
Authors:
Styliani-Christina Fragkouli,
Dhwani Solanki,
Leyla J Castro,
Fotis E Psomopoulos,
Núria Queralt-Rosinach,
Davide Cirillo,
Lisa C Crossman
Abstract:
Over the last three to five years, it has become possible to generate machine learning synthetic data for healthcare-related uses. However, concerns have been raised about potential negative factors associated with the possibilities of artificial dataset generation. These include the potential misuse of generative artificial intelligence (AI) in fields such as cybercrime, the use of deepfakes and…
▽ More
Over the last three to five years, it has become possible to generate machine learning synthetic data for healthcare-related uses. However, concerns have been raised about potential negative factors associated with the possibilities of artificial dataset generation. These include the potential misuse of generative artificial intelligence (AI) in fields such as cybercrime, the use of deepfakes and fake news to deceive or manipulate, and displacement of human jobs across various market sectors.
Here, we consider both current and future positive advances and possibilities with synthetic datasets. Synthetic data offers significant benefits, particularly in data privacy, research, in balancing datasets and reducing bias in machine learning models. Generative AI is an artificial intelligence genre capable of creating text, images, video or other data using generative models. The recent explosion of interest in GenAI was heralded by the invention and speedy move to use of large language models (LLM). These computational models are able to achieve general-purpose language generation and other natural language processing tasks and are based on transformer architectures, which made an evolutionary leap from previous neural network architectures.
Fuelled by the advent of improved GenAI techniques and wide scale usage, this is surely the time to consider how synthetic data can be used to advance infectious disease research. In this commentary we aim to create an overview of the current and future position of synthetic data in infectious disease research.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
DOME: Recommendations for supervised machine learning validation in biology
Authors:
Ian Walsh,
Dmytro Fishman,
Dario Garcia-Gasulla,
Tiina Titma,
Gianluca Pollastri,
The ELIXIR Machine Learning focus group,
Jen Harrow,
Fotis E. Psomopoulos,
Silvio C. E. Tosatto
Abstract:
Modern biology frequently relies on machine learning to provide predictions and improve decision processes. There have been recent calls for more scrutiny on machine learning performance and possible limitations. Here we present a set of community-wide recommendations aiming to help establish standards of supervised machine learning validation in biology. Adopting a structured methods description…
▽ More
Modern biology frequently relies on machine learning to provide predictions and improve decision processes. There have been recent calls for more scrutiny on machine learning performance and possible limitations. Here we present a set of community-wide recommendations aiming to help establish standards of supervised machine learning validation in biology. Adopting a structured methods description for machine learning based on data, optimization, model, evaluation (DOME) will aim to help both reviewers and readers to better understand and assess the performance and limitations of a method or outcome. The recommendations are formulated as questions to anyone wishing to pursue implementation of a machine learning algorithm. Answers to these questions can be easily included in the supplementary material of published papers.
△ Less
Submitted 7 January, 2021; v1 submitted 25 June, 2020;
originally announced June 2020.
-
DNA coding and Gödel numbering
Authors:
Argyris Nicolaidis,
Fotis Psomopoulos
Abstract:
Evolution consists of distinct stages: cosmological, biological, linguistic. Since biology verges on natural sciences and linguistics, we expect that it shares structures and features from both forms of knowledge. Indeed, in DNA we encounter the biological "atoms", the four nucleotide molecules. At the same time these four nucleotides may be considered as the "letters" of an alphabet. These four "…
▽ More
Evolution consists of distinct stages: cosmological, biological, linguistic. Since biology verges on natural sciences and linguistics, we expect that it shares structures and features from both forms of knowledge. Indeed, in DNA we encounter the biological "atoms", the four nucleotide molecules. At the same time these four nucleotides may be considered as the "letters" of an alphabet. These four "letters", through a genetic code, generate biological "words", "phrases", "sentences" (aminoacids, proteins, cells, living organisms). In this spirit we may consider equally well a DNA strand as a mathematical statement. Inspired by the work of Kurt Gödel, we attach to each DNA strand a Gödel's number, a product of prime numbers raised to appropriate powers. To each DNA chain corresponds a single Gödel's number $G$, and inversely given a Gödel's number $G$, we can specify the DNA chain it stands for. Next, considering a single DNA strand composed of $N$ bases, we study the statistical distribution of $g$, the logarithm of $G$. Our assumption is that the choice of the $m$-th term is random and with equal probability for the four possible outcomes. The "experiment", to some extent, appears as throwing $N$ times a four-faces die. Through the moment generating function we obtain the discrete and then the continuum distribution of $g$. There is an excellent agreement between our formalism and simulated data. At the end we compare our formalism to actual data, to specify the presence of traces of non-random dynamics.
△ Less
Submitted 30 September, 2019;
originally announced September 2019.
