-
Mask-Guided Multi-Channel SwinUNETR Framework for Robust MRI Classification
Authors:
Smriti Joshi,
Lidia Garrucho,
Richard Osuala,
Oliver Diaz,
Karim Lekadir
Abstract:
Breast cancer is one of the leading causes of cancer-related mortality in women, and early detection is essential for improving outcomes. Magnetic resonance imaging (MRI) is a highly sensitive tool for breast cancer detection, particularly in women at high risk or with dense breast tissue, where mammography is less effective. The ODELIA consortium organized a multi-center challenge to foster AI-ba…
▽ More
Breast cancer is one of the leading causes of cancer-related mortality in women, and early detection is essential for improving outcomes. Magnetic resonance imaging (MRI) is a highly sensitive tool for breast cancer detection, particularly in women at high risk or with dense breast tissue, where mammography is less effective. The ODELIA consortium organized a multi-center challenge to foster AI-based solutions for breast cancer diagnosis and classification. The dataset included 511 studies from six European centers, acquired on scanners from multiple vendors at both 1.5 T and 3 T. Each study was labeled for the left and right breast as no lesion, benign lesion, or malignant lesion. We developed a SwinUNETR-based deep learning framework that incorporates breast region masking, extensive data augmentation, and ensemble learning to improve robustness and generalizability. Our method achieved second place on the challenge leaderboard, highlighting its potential to support clinical breast MRI interpretation. We publicly share our codebase at https://github.com/smriti-joshi/bcnaim-odelia-challenge.git.
△ Less
Submitted 28 August, 2025;
originally announced August 2025.
-
Comparing Conditional Diffusion Models for Synthesizing Contrast-Enhanced Breast MRI from Pre-Contrast Images
Authors:
Sebastian Ibarra,
Javier del Riego,
Alessandro Catanese,
Julian Cuba,
Julian Cardona,
Nataly Leon,
Jonathan Infante,
Karim Lekadir,
Oliver Diaz,
Richard Osuala
Abstract:
Dynamic contrast-enhanced (DCE) MRI is essential for breast cancer diagnosis and treatment. However, its reliance on contrast agents introduces safety concerns, contraindications, increased cost, and workflow complexity. To this end, we present pre-contrast conditioned denoising diffusion probabilistic models to synthesize DCE-MRI, introducing, evaluating, and comparing a total of 22 generative mo…
▽ More
Dynamic contrast-enhanced (DCE) MRI is essential for breast cancer diagnosis and treatment. However, its reliance on contrast agents introduces safety concerns, contraindications, increased cost, and workflow complexity. To this end, we present pre-contrast conditioned denoising diffusion probabilistic models to synthesize DCE-MRI, introducing, evaluating, and comparing a total of 22 generative model variants in both single-breast and full breast settings. Towards enhancing lesion fidelity, we introduce both tumor-aware loss functions and explicit tumor segmentation mask conditioning. Using a public multicenter dataset and comparing to respective pre-contrast baselines, we observe that subtraction image-based models consistently outperform post-contrast-based models across five complementary evaluation metrics. Apart from assessing the entire image, we also separately evaluate the region of interest, where both tumor-aware losses and segmentation mask inputs improve evaluation metrics. The latter notably enhance qualitative results capturing contrast uptake, albeit assuming access to tumor localization inputs that are not guaranteed to be available in screening settings. A reader study involving 2 radiologists and 4 MRI technologists confirms the high realism of the synthetic images, indicating an emerging clinical potential of generative contrast-enhancement. We share our codebase at https://github.com/sebastibar/conditional-diffusion-breast-MRI.
△ Less
Submitted 15 September, 2025; v1 submitted 19 August, 2025;
originally announced August 2025.
-
Clinically-guided Data Synthesis for Laryngeal Lesion Detection
Authors:
Chiara Baldini,
Kaisar Kushibar,
Richard Osuala,
Simone Balocco,
Oliver Diaz,
Karim Lekadir,
Leonardo S. Mattos
Abstract:
Although computer-aided diagnosis (CADx) and detection (CADe) systems have made significant progress in various medical domains, their application is still limited in specialized fields such as otorhinolaryngology. In the latter, current assessment methods heavily depend on operator expertise, and the high heterogeneity of lesions complicates diagnosis, with biopsy persisting as the gold standard…
▽ More
Although computer-aided diagnosis (CADx) and detection (CADe) systems have made significant progress in various medical domains, their application is still limited in specialized fields such as otorhinolaryngology. In the latter, current assessment methods heavily depend on operator expertise, and the high heterogeneity of lesions complicates diagnosis, with biopsy persisting as the gold standard despite its substantial costs and risks. A critical bottleneck for specialized endoscopic CADx/e systems is the lack of well-annotated datasets with sufficient variability for real-world generalization. This study introduces a novel approach that exploits a Latent Diffusion Model (LDM) coupled with a ControlNet adapter to generate laryngeal endoscopic image-annotation pairs, guided by clinical observations. The method addresses data scarcity by conditioning the diffusion process to produce realistic, high-quality, and clinically relevant image features that capture diverse anatomical conditions. The proposed approach can be leveraged to expand training datasets for CADx/e models, empowering the assessment process in laryngology. Indeed, during a downstream task of detection, the addition of only 10% synthetic data improved the detection rate of laryngeal lesions by 9% when the model was internally tested and 22.1% on out-of-domain external data. Additionally, the realism of the generated images was evaluated by asking 5 expert otorhinolaryngologists with varying expertise to rate their confidence in distinguishing synthetic from real images. This work has the potential to accelerate the development of automated tools for laryngeal disease diagnosis, offering a solution to data scarcity and demonstrating the applicability of synthetic data in real-world scenarios.
△ Less
Submitted 8 August, 2025;
originally announced August 2025.
-
Single Image Test-Time Adaptation via Multi-View Co-Training
Authors:
Smriti Joshi,
Richard Osuala,
Lidia Garrucho,
Kaisar Kushibar,
Dimitri Kessler,
Oliver Diaz,
Karim Lekadir
Abstract:
Test-time adaptation enables a trained model to adjust to a new domain during inference, making it particularly valuable in clinical settings where such on-the-fly adaptation is required. However, existing techniques depend on large target domain datasets, which are often impractical and unavailable in medical scenarios that demand per-patient, real-time inference. Moreover, current methods common…
▽ More
Test-time adaptation enables a trained model to adjust to a new domain during inference, making it particularly valuable in clinical settings where such on-the-fly adaptation is required. However, existing techniques depend on large target domain datasets, which are often impractical and unavailable in medical scenarios that demand per-patient, real-time inference. Moreover, current methods commonly focus on two-dimensional images, failing to leverage the volumetric richness of medical imaging data. Bridging this gap, we propose a Patch-Based Multi-View Co-Training method for Single Image Test-Time adaptation. Our method enforces feature and prediction consistency through uncertainty-guided self-training, enabling effective volumetric segmentation in the target domain with only a single test-time image. Validated on three publicly available breast magnetic resonance imaging datasets for tumor segmentation, our method achieves performance close to the upper bound supervised benchmark while also outperforming all existing state-of-the-art methods, on average by a Dice Similarity Coefficient of 3.75%. We publicly share our accessible codebase, readily integrable with the popular nnUNet framework, at https://github.com/smriti-joshi/muvi.git.
△ Less
Submitted 30 June, 2025;
originally announced June 2025.
-
Segmentation variability and radiomics stability for predicting Triple-Negative Breast Cancer subtype using Magnetic Resonance Imaging
Authors:
Isabella Cama,
Alejandro Guzmán,
Cristina Campi,
Michele Piana,
Karim Lekadir,
Sara Garbarino,
Oliver Díaz
Abstract:
Most papers caution against using predictive models for disease stratification based on unselected radiomic features, as these features are affected by contouring variability. Instead, they advocate for the use of the Intraclass Correlation Coefficient (ICC) as a measure of stability for feature selection. However, the direct effect of segmentation variability on the predictive models is rarely st…
▽ More
Most papers caution against using predictive models for disease stratification based on unselected radiomic features, as these features are affected by contouring variability. Instead, they advocate for the use of the Intraclass Correlation Coefficient (ICC) as a measure of stability for feature selection. However, the direct effect of segmentation variability on the predictive models is rarely studied. This study investigates the impact of segmentation variability on feature stability and predictive performance in radiomics-based prediction of Triple-Negative Breast Cancer (TNBC) subtype using Magnetic Resonance Imaging. A total of 244 images from the Duke dataset were used, with segmentation variability introduced through modifications of manual segmentations. For each mask, explainable radiomic features were selected using the Shapley Additive exPlanations method and used to train logistic regression models. Feature stability across segmentations was assessed via ICC, Pearson's correlation, and reliability scores quantifying the relationship between feature stability and segmentation variability. Results indicate that segmentation accuracy does not significantly impact predictive performance. While incorporating peritumoral information may reduce feature reproducibility, it does not diminish feature predictive capability. Moreover, feature selection in predictive models is not inherently tied to feature stability with respect to segmentation, suggesting that an overreliance on ICC or reliability scores for feature selection might exclude valuable predictive features.
△ Less
Submitted 2 April, 2025;
originally announced April 2025.
-
Federated nnU-Net for Privacy-Preserving Medical Image Segmentation
Authors:
Grzegorz Skorupko,
Fotios Avgoustidis,
Carlos Martín-Isla,
Lidia Garrucho,
Dimitri A. Kessler,
Esmeralda Ruiz Pujadas,
Oliver Díaz,
Maciej Bobowicz,
Katarzyna Gwoździewicz,
Xavier Bargalló,
Paulius Jaruševičius,
Richard Osuala,
Kaisar Kushibar,
Karim Lekadir
Abstract:
The nnU-Net framework has played a crucial role in medical image segmentation and has become the gold standard in multitudes of applications targeting different diseases, organs, and modalities. However, so far it has been used primarily in a centralized approach where the collected data is stored in the same location where nnU-Net is trained. This centralized approach has various limitations, suc…
▽ More
The nnU-Net framework has played a crucial role in medical image segmentation and has become the gold standard in multitudes of applications targeting different diseases, organs, and modalities. However, so far it has been used primarily in a centralized approach where the collected data is stored in the same location where nnU-Net is trained. This centralized approach has various limitations, such as potential leakage of sensitive patient information and violation of patient privacy. Federated learning has emerged as a key approach for training segmentation models in a decentralized manner, enabling collaborative development while prioritising patient privacy. In this paper, we propose FednnU-Net, a plug-and-play, federated learning extension of the nnU-Net framework. To this end, we contribute two federated methodologies to unlock decentralized training of nnU-Net, namely, Federated Fingerprint Extraction (FFE) and Asymmetric Federated Averaging (AsymFedAvg). We conduct a comprehensive set of experiments demonstrating high and consistent performance of our methods for breast, cardiac and fetal segmentation based on a multi-modal collection of 6 datasets representing samples from 18 different institutions. To democratize research as well as real-world deployments of decentralized training in clinical centres, we publicly share our framework at https://github.com/faildeny/FednnUNet .
△ Less
Submitted 28 August, 2025; v1 submitted 4 March, 2025;
originally announced March 2025.
-
Fréchet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets
Authors:
Nicholas Konz,
Richard Osuala,
Preeti Verma,
Yuwen Chen,
Hanxue Gu,
Haoyu Dong,
Yaqian Chen,
Andrew Marshall,
Lidia Garrucho,
Kaisar Kushibar,
Daniel M. Lang,
Gene S. Kim,
Lars J. Grimm,
John M. Lewin,
James S. Duncan,
Julia A. Schnabel,
Oliver Diaz,
Karim Lekadir,
Maciej A. Mazurowski
Abstract:
Determining whether two sets of images belong to the same or different distributions or domains is a crucial task in modern medical image analysis and deep learning; for example, to evaluate the output quality of image generative models. Currently, metrics used for this task either rely on the (potentially biased) choice of some downstream task, such as segmentation, or adopt task-independent perc…
▽ More
Determining whether two sets of images belong to the same or different distributions or domains is a crucial task in modern medical image analysis and deep learning; for example, to evaluate the output quality of image generative models. Currently, metrics used for this task either rely on the (potentially biased) choice of some downstream task, such as segmentation, or adopt task-independent perceptual metrics (e.g., Fréchet Inception Distance/FID) from natural imaging, which we show insufficiently capture anatomical features. To this end, we introduce a new perceptual metric tailored for medical images, FRD (Fréchet Radiomic Distance), which utilizes standardized, clinically meaningful, and interpretable image features. We show that FRD is superior to other image distribution metrics for a range of medical imaging applications, including out-of-domain (OOD) detection, the evaluation of image-to-image translation (by correlating more with downstream task performance as well as anatomical consistency and realism), and the evaluation of unconditional image generation. Moreover, FRD offers additional benefits such as stability and computational efficiency at low sample sizes, sensitivity to image corruptions and adversarial attacks, feature interpretability, and correlation with radiologist-perceived image quality. Additionally, we address key gaps in the literature by presenting an extensive framework for the multifaceted evaluation of image similarity metrics in medical imaging -- including the first large-scale comparative study of generative models for medical image translation -- and release an accessible codebase to facilitate future research. Our results are supported by thorough experiments spanning a variety of datasets, modalities, and downstream tasks, highlighting the broad potential of FRD for medical image analysis.
△ Less
Submitted 6 June, 2025; v1 submitted 2 December, 2024;
originally announced December 2024.
-
Streamlining the review process: AI-generated annotations in research manuscripts
Authors:
Oscar Díaz,
Xabier Garmendia,
Juanan Pereira
Abstract:
The increasing volume of research paper submissions poses a significant challenge to the traditional academic peer-review system, leading to an overwhelming workload for reviewers. This study explores the potential of integrating Large Language Models (LLMs) into the peer-review process to enhance efficiency without compromising effectiveness. We focus on manuscript annotations, particularly excer…
▽ More
The increasing volume of research paper submissions poses a significant challenge to the traditional academic peer-review system, leading to an overwhelming workload for reviewers. This study explores the potential of integrating Large Language Models (LLMs) into the peer-review process to enhance efficiency without compromising effectiveness. We focus on manuscript annotations, particularly excerpt highlights, as a potential area for AI-human collaboration. While LLMs excel in certain tasks like aspect coverage and informativeness, they often lack high-level analysis and critical thinking, making them unsuitable for replacing human reviewers entirely. Our approach involves using LLMs to assist with specific aspects of the review process. This paper introduces AnnotateGPT, a platform that utilizes GPT-4 for manuscript review, aiming to improve reviewers' comprehension and focus. We evaluate AnnotateGPT using a Technology Acceptance Model (TAM) questionnaire with nine participants and generalize the findings. Our work highlights annotation as a viable middle ground for AI-human collaboration in academic review, offering insights into integrating LLMs into the review process and tuning traditional annotation tools for LLM incorporation.
△ Less
Submitted 29 November, 2024;
originally announced December 2024.
-
Simulating Dynamic Tumor Contrast Enhancement in Breast MRI using Conditional Generative Adversarial Networks
Authors:
Richard Osuala,
Smriti Joshi,
Apostolia Tsirikoglou,
Lidia Garrucho,
Walter H. L. Pinaya,
Daniel M. Lang,
Julia A. Schnabel,
Oliver Diaz,
Karim Lekadir
Abstract:
This paper presents a method for virtual contrast enhancement in breast MRI, offering a promising non-invasive alternative to traditional contrast agent-based DCE-MRI acquisition. Using a conditional generative adversarial network, we predict DCE-MRI images, including jointly-generated sequences of multiple corresponding DCE-MRI timepoints, from non-contrast-enhanced MRIs, enabling tumor localizat…
▽ More
This paper presents a method for virtual contrast enhancement in breast MRI, offering a promising non-invasive alternative to traditional contrast agent-based DCE-MRI acquisition. Using a conditional generative adversarial network, we predict DCE-MRI images, including jointly-generated sequences of multiple corresponding DCE-MRI timepoints, from non-contrast-enhanced MRIs, enabling tumor localization and characterization without the associated health risks. Furthermore, we qualitatively and quantitatively evaluate the synthetic DCE-MRI images, proposing a multi-metric Scaled Aggregate Measure (SAMe), assessing their utility in a tumor segmentation downstream task, and conclude with an analysis of the temporal patterns in multi-sequence DCE-MRI generation. Our approach demonstrates promising results in generating realistic and useful DCE-MRI sequences, highlighting the potential of virtual contrast enhancement for improving breast cancer diagnosis and treatment, particularly for patients where contrast agent administration is contraindicated.
△ Less
Submitted 14 May, 2025; v1 submitted 27 September, 2024;
originally announced September 2024.
-
Enhancing the Utility of Privacy-Preserving Cancer Classification using Synthetic Data
Authors:
Richard Osuala,
Daniel M. Lang,
Anneliese Riess,
Georgios Kaissis,
Zuzanna Szafranowska,
Grzegorz Skorupko,
Oliver Diaz,
Julia A. Schnabel,
Karim Lekadir
Abstract:
Deep learning holds immense promise for aiding radiologists in breast cancer detection. However, achieving optimal model performance is hampered by limitations in availability and sharing of data commonly associated to patient privacy concerns. Such concerns are further exacerbated, as traditional deep learning models can inadvertently leak sensitive training information. This work addresses these…
▽ More
Deep learning holds immense promise for aiding radiologists in breast cancer detection. However, achieving optimal model performance is hampered by limitations in availability and sharing of data commonly associated to patient privacy concerns. Such concerns are further exacerbated, as traditional deep learning models can inadvertently leak sensitive training information. This work addresses these challenges exploring and quantifying the utility of privacy-preserving deep learning techniques, concretely, (i) differentially private stochastic gradient descent (DP-SGD) and (ii) fully synthetic training data generated by our proposed malignancy-conditioned generative adversarial network. We assess these methods via downstream malignancy classification of mammography masses using a transformer model. Our experimental results depict that synthetic data augmentation can improve privacy-utility tradeoffs in differentially private model training. Further, model pretraining on synthetic data achieves remarkable performance, which can be further increased with DP-SGD fine-tuning across all privacy guarantees. With this first in-depth exploration of privacy-preserving deep learning in breast imaging, we address current and emerging clinical privacy requirements and pave the way towards the adoption of private high-utility deep diagnostic models. Our reproducible codebase is publicly available at https://github.com/RichardObi/mammo_dp.
△ Less
Submitted 17 July, 2024;
originally announced July 2024.
-
A large-scale multicenter breast cancer DCE-MRI benchmark dataset with expert segmentations
Authors:
Lidia Garrucho,
Kaisar Kushibar,
Claire-Anne Reidel,
Smriti Joshi,
Richard Osuala,
Apostolia Tsirikoglou,
Maciej Bobowicz,
Javier del Riego,
Alessandro Catanese,
Katarzyna Gwoździewicz,
Maria-Laura Cosaka,
Pasant M. Abo-Elhoda,
Sara W. Tantawy,
Shorouq S. Sakrana,
Norhan O. Shawky-Abdelfatah,
Amr Muhammad Abdo-Salem,
Androniki Kozana,
Eugen Divjak,
Gordana Ivanac,
Katerina Nikiforaki,
Michail E. Klontzas,
Rosa García-Dosdá,
Meltem Gulsun-Akpinar,
Oğuz Lafcı,
Ritse Mann
, et al. (8 additional authors not shown)
Abstract:
Artificial Intelligence (AI) research in breast cancer Magnetic Resonance Imaging (MRI) faces challenges due to limited expert-labeled segmentations. To address this, we present a multicenter dataset of 1506 pre-treatment T1-weighted dynamic contrast-enhanced MRI cases, including expert annotations of primary tumors and non-mass-enhanced regions. The dataset integrates imaging data from four colle…
▽ More
Artificial Intelligence (AI) research in breast cancer Magnetic Resonance Imaging (MRI) faces challenges due to limited expert-labeled segmentations. To address this, we present a multicenter dataset of 1506 pre-treatment T1-weighted dynamic contrast-enhanced MRI cases, including expert annotations of primary tumors and non-mass-enhanced regions. The dataset integrates imaging data from four collections in The Cancer Imaging Archive (TCIA), where only 163 cases with expert segmentations were initially available. To facilitate the annotation process, a deep learning model was trained to produce preliminary segmentations for the remaining cases. These were subsequently corrected and verified by 16 breast cancer experts (averaging 9 years of experience), creating a fully annotated dataset. Additionally, the dataset includes 49 harmonized clinical and demographic variables, as well as pre-trained weights for a baseline nnU-Net model trained on the annotated data. This resource addresses a critical gap in publicly available breast cancer datasets, enabling the development, validation, and benchmarking of advanced deep learning models, thus driving progress in breast cancer diagnostics, treatment response prediction, and personalized care.
△ Less
Submitted 21 February, 2025; v1 submitted 19 June, 2024;
originally announced June 2024.
-
Mitigating annotation shift in cancer classification using single image generative models
Authors:
Marta Buetas Arcas,
Richard Osuala,
Karim Lekadir,
Oliver Díaz
Abstract:
Artificial Intelligence (AI) has emerged as a valuable tool for assisting radiologists in breast cancer detection and diagnosis. However, the success of AI applications in this domain is restricted by the quantity and quality of available data, posing challenges due to limited and costly data annotation procedures that often lead to annotation shifts. This study simulates, analyses and mitigates a…
▽ More
Artificial Intelligence (AI) has emerged as a valuable tool for assisting radiologists in breast cancer detection and diagnosis. However, the success of AI applications in this domain is restricted by the quantity and quality of available data, posing challenges due to limited and costly data annotation procedures that often lead to annotation shifts. This study simulates, analyses and mitigates annotation shifts in cancer classification in the breast mammography domain. First, a high-accuracy cancer risk prediction model is developed, which effectively distinguishes benign from malignant lesions. Next, model performance is used to quantify the impact of annotation shift. We uncover a substantial impact of annotation shift on multiclass classification performance particularly for malignant lesions. We thus propose a training data augmentation approach based on single-image generative models for the affected class, requiring as few as four in-domain annotations to considerably mitigate annotation shift, while also addressing dataset imbalance. Lastly, we further increase performance by proposing and validating an ensemble architecture based on multiple models trained under different data augmentation regimes. Our study offers key insights into annotation shift in deep learning breast cancer classification and explores the potential of single-image generative models to overcome domain shift challenges.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
Towards Learning Contrast Kinetics with Multi-Condition Latent Diffusion Models
Authors:
Richard Osuala,
Daniel M. Lang,
Preeti Verma,
Smriti Joshi,
Apostolia Tsirikoglou,
Grzegorz Skorupko,
Kaisar Kushibar,
Lidia Garrucho,
Walter H. L. Pinaya,
Oliver Diaz,
Julia A. Schnabel,
Karim Lekadir
Abstract:
Contrast agents in dynamic contrast enhanced magnetic resonance imaging allow to localize tumors and observe their contrast kinetics, which is essential for cancer characterization and respective treatment decision-making. However, contrast agent administration is not only associated with adverse health risks, but also restricted for patients during pregnancy, and for those with kidney malfunction…
▽ More
Contrast agents in dynamic contrast enhanced magnetic resonance imaging allow to localize tumors and observe their contrast kinetics, which is essential for cancer characterization and respective treatment decision-making. However, contrast agent administration is not only associated with adverse health risks, but also restricted for patients during pregnancy, and for those with kidney malfunction, or other adverse reactions. With contrast uptake as key biomarker for lesion malignancy, cancer recurrence risk, and treatment response, it becomes pivotal to reduce the dependency on intravenous contrast agent administration. To this end, we propose a multi-conditional latent diffusion model capable of acquisition time-conditioned image synthesis of DCE-MRI temporal sequences. To evaluate medical image synthesis, we additionally propose and validate the Fréchet radiomics distance as an image quality measure based on biomarker variability between synthetic and real imaging data. Our results demonstrate our method's ability to generate realistic multi-sequence fat-saturated breast DCE-MRI and uncover the emerging potential of deep learning based contrast kinetics simulation. We publicly share our accessible codebase at https://github.com/RichardObi/ccnet and provide a user-friendly library for Fréchet radiomics distance calculation at https://pypi.org/project/frd-score.
△ Less
Submitted 17 July, 2024; v1 submitted 20 March, 2024;
originally announced March 2024.
-
Pre- to Post-Contrast Breast MRI Synthesis for Enhanced Tumour Segmentation
Authors:
Richard Osuala,
Smriti Joshi,
Apostolia Tsirikoglou,
Lidia Garrucho,
Walter H. L. Pinaya,
Oliver Diaz,
Karim Lekadir
Abstract:
Despite its benefits for tumour detection and treatment, the administration of contrast agents in dynamic contrast-enhanced MRI (DCE-MRI) is associated with a range of issues, including their invasiveness, bioaccumulation, and a risk of nephrogenic systemic fibrosis. This study explores the feasibility of producing synthetic contrast enhancements by translating pre-contrast T1-weighted fat-saturat…
▽ More
Despite its benefits for tumour detection and treatment, the administration of contrast agents in dynamic contrast-enhanced MRI (DCE-MRI) is associated with a range of issues, including their invasiveness, bioaccumulation, and a risk of nephrogenic systemic fibrosis. This study explores the feasibility of producing synthetic contrast enhancements by translating pre-contrast T1-weighted fat-saturated breast MRI to their corresponding first DCE-MRI sequence leveraging the capabilities of a generative adversarial network (GAN). Additionally, we introduce a Scaled Aggregate Measure (SAMe) designed for quantitatively evaluating the quality of synthetic data in a principled manner and serving as a basis for selecting the optimal generative model. We assess the generated DCE-MRI data using quantitative image quality metrics and apply them to the downstream task of 3D breast tumour segmentation. Our results highlight the potential of post-contrast DCE-MRI synthesis in enhancing the robustness of breast tumour segmentation models via data augmentation. Our code is available at https://github.com/RichardObi/pre_post_synthesis.
△ Less
Submitted 31 May, 2024; v1 submitted 17 November, 2023;
originally announced November 2023.
-
FUTURE-AI: International consensus guideline for trustworthy and deployable artificial intelligence in healthcare
Authors:
Karim Lekadir,
Aasa Feragen,
Abdul Joseph Fofanah,
Alejandro F Frangi,
Alena Buyx,
Anais Emelie,
Andrea Lara,
Antonio R Porras,
An-Wen Chan,
Arcadi Navarro,
Ben Glocker,
Benard O Botwe,
Bishesh Khanal,
Brigit Beger,
Carol C Wu,
Celia Cintas,
Curtis P Langlotz,
Daniel Rueckert,
Deogratias Mzurikwao,
Dimitrios I Fotiadis,
Doszhan Zhussupov,
Enzo Ferrante,
Erik Meijering,
Eva Weicken,
Fabio A González
, et al. (95 additional authors not shown)
Abstract:
Despite major advances in artificial intelligence (AI) for medicine and healthcare, the deployment and adoption of AI technologies remain limited in real-world clinical practice. In recent years, concerns have been raised about the technical, clinical, ethical and legal risks associated with medical AI. To increase real world adoption, it is essential that medical AI tools are trusted and accepted…
▽ More
Despite major advances in artificial intelligence (AI) for medicine and healthcare, the deployment and adoption of AI technologies remain limited in real-world clinical practice. In recent years, concerns have been raised about the technical, clinical, ethical and legal risks associated with medical AI. To increase real world adoption, it is essential that medical AI tools are trusted and accepted by patients, clinicians, health organisations and authorities. This work describes the FUTURE-AI guideline as the first international consensus framework for guiding the development and deployment of trustworthy AI tools in healthcare. The FUTURE-AI consortium was founded in 2021 and currently comprises 118 inter-disciplinary experts from 51 countries representing all continents, including AI scientists, clinicians, ethicists, and social scientists. Over a two-year period, the consortium defined guiding principles and best practices for trustworthy AI through an iterative process comprising an in-depth literature review, a modified Delphi survey, and online consensus meetings. The FUTURE-AI framework was established based on 6 guiding principles for trustworthy AI in healthcare, i.e. Fairness, Universality, Traceability, Usability, Robustness and Explainability. Through consensus, a set of 28 best practices were defined, addressing technical, clinical, legal and socio-ethical dimensions. The recommendations cover the entire lifecycle of medical AI, from design, development and validation to regulation, deployment, and monitoring. FUTURE-AI is a risk-informed, assumption-free guideline which provides a structured approach for constructing medical AI tools that will be trusted, deployed and adopted in real-world practice. Researchers are encouraged to take the recommendations into account in proof-of-concept stages to facilitate future translation towards clinical practice of medical AI.
△ Less
Submitted 8 July, 2024; v1 submitted 11 August, 2023;
originally announced September 2023.
-
How can feature usage be tracked across product variants? Implicit Feedback in Software Product Lines
Authors:
Oscar Díaz,
Raul Medeiros,
Mustafa Al-Hajjaji
Abstract:
Implicit feedback is collecting information about software usage to understand how and when the software is used. This research tackles implicit feedback in Software Product Lines (SPLs). The need for platform-centric feedback makes SPL feedback depart from one-off-application feedback in both the artefact to be tracked (the platform vs the variant) as well as the tracking approach (indirect codin…
▽ More
Implicit feedback is collecting information about software usage to understand how and when the software is used. This research tackles implicit feedback in Software Product Lines (SPLs). The need for platform-centric feedback makes SPL feedback depart from one-off-application feedback in both the artefact to be tracked (the platform vs the variant) as well as the tracking approach (indirect coding vs direct coding). Traditionally, product feedback is achieved by embedding `usage trackers' into the software's code. Yet, products are now members of the SPL portfolio, and hence, this approach conflicts with one of the main SPL tenants: reducing, if not eliminating, coding directly into the variant's code. Thus, we advocate for Product Derivation to be subject to a second transformation that precedes the construction of the variant based on the configuration model. This approach is tested through FEACKER, an extension to pure::variants. We resorted to a TAM evaluation on pure-systems GmbH employees(n=8). Observed divergences were next tackled through a focus group (n=3). The results reveal agreement in the interest in conducting feedback analysis at the platform level (perceived usefulness) while regarding FEACKER as a seamless
△ Less
Submitted 15 September, 2023; v1 submitted 8 September, 2023;
originally announced September 2023.
-
Revisiting Skin Tone Fairness in Dermatological Lesion Classification
Authors:
Thorsten Kalb,
Kaisar Kushibar,
Celia Cintas,
Karim Lekadir,
Oliver Diaz,
Richard Osuala
Abstract:
Addressing fairness in lesion classification from dermatological images is crucial due to variations in how skin diseases manifest across skin tones. However, the absence of skin tone labels in public datasets hinders building a fair classifier. To date, such skin tone labels have been estimated prior to fairness analysis in independent studies using the Individual Typology Angle (ITA). Briefly, I…
▽ More
Addressing fairness in lesion classification from dermatological images is crucial due to variations in how skin diseases manifest across skin tones. However, the absence of skin tone labels in public datasets hinders building a fair classifier. To date, such skin tone labels have been estimated prior to fairness analysis in independent studies using the Individual Typology Angle (ITA). Briefly, ITA calculates an angle based on pixels extracted from skin images taking into account the lightness and yellow-blue tints. These angles are then categorised into skin tones that are subsequently used to analyse fairness in skin cancer classification. In this work, we review and compare four ITA-based approaches of skin tone classification on the ISIC18 dataset, a common benchmark for assessing skin cancer classification fairness in the literature. Our analyses reveal a high disagreement among previously published studies demonstrating the risks of ITA-based skin tone estimation methods. Moreover, we investigate the causes of such large discrepancy among these approaches and find that the lack of diversity in the ISIC18 dataset limits its use as a testbed for fairness analysis. Finally, we recommend further research on robust ITA estimation and diverse dataset acquisition with skin tone annotation to facilitate conclusive fairness assessments of artificial intelligence tools in dermatology. Our code is available at https://github.com/tkalbl/RevisitingSkinToneFairness.
△ Less
Submitted 18 August, 2023;
originally announced August 2023.
-
Optimizing Floors in First Price Auctions: an Empirical Study of Yahoo Advertising
Authors:
Miguel Alcobendas,
Jonathan Ji,
Hemakumar Gokulakannan,
Dawit Wami,
Boris Kapchits,
Emilien Pouradier Duteil,
Korby Satow,
Maria Rosario Levy Roman,
Oriol Diaz,
Amado A. Diaz Jr.,
Rabi Kavoori
Abstract:
Floors (also known as reserve prices) help publishers to increase the expected revenue of their ad space, which is usually sold via auctions. Floors are defined as the minimum bid that a seller (it can be a publisher or an ad exchange) is willing to accept for the inventory opportunity. In this paper, we present a model to set floors in first price auctions, and discuss the impact of its implement…
▽ More
Floors (also known as reserve prices) help publishers to increase the expected revenue of their ad space, which is usually sold via auctions. Floors are defined as the minimum bid that a seller (it can be a publisher or an ad exchange) is willing to accept for the inventory opportunity. In this paper, we present a model to set floors in first price auctions, and discuss the impact of its implementation on Yahoo sites. The model captures important characteristics of the online advertising industry. For instance, some bidders impose restrictions on how ad exchanges can handle data from bidders, conditioning the model choice to set reserve prices. Our solution induces bidders to change their bidding behavior as a response to the floors enclosed in the bid request, helping online publishers to increase their ad revenue.
The outlined methodology has been implemented at Yahoo with remarkable results. The annualized incremental revenue is estimated at +1.3% on Yahoo display inventory, and +2.5% on video ad inventory. These are non-negligible numbers in the multi-million Yahoo ad business.
△ Less
Submitted 9 February, 2024; v1 submitted 12 February, 2023;
originally announced February 2023.
-
medigan: a Python library of pretrained generative models for medical image synthesis
Authors:
Richard Osuala,
Grzegorz Skorupko,
Noussair Lazrak,
Lidia Garrucho,
Eloy García,
Smriti Joshi,
Socayna Jouide,
Michael Rutherford,
Fred Prior,
Kaisar Kushibar,
Oliver Diaz,
Karim Lekadir
Abstract:
Synthetic data generated by generative models can enhance the performance and capabilities of data-hungry deep learning models in medical imaging. However, there is (1) limited availability of (synthetic) datasets and (2) generative models are complex to train, which hinders their adoption in research and clinical applications. To reduce this entry barrier, we propose medigan, a one-stop shop for…
▽ More
Synthetic data generated by generative models can enhance the performance and capabilities of data-hungry deep learning models in medical imaging. However, there is (1) limited availability of (synthetic) datasets and (2) generative models are complex to train, which hinders their adoption in research and clinical applications. To reduce this entry barrier, we propose medigan, a one-stop shop for pretrained generative models implemented as an open-source framework-agnostic Python library. medigan allows researchers and developers to create, increase, and domain-adapt their training data in just a few lines of code. Guided by design decisions based on gathered end-user requirements, we implement medigan based on modular components for generative model (i) execution, (ii) visualisation, (iii) search & ranking, and (iv) contribution. The library's scalability and design is demonstrated by its growing number of integrated and readily-usable pretrained generative models consisting of 21 models utilising 9 different Generative Adversarial Network architectures trained on 11 datasets from 4 domains, namely, mammography, endoscopy, x-ray, and MRI. Furthermore, 3 applications of medigan are analysed in this work, which include (a) enabling community-wide sharing of restricted data, (b) investigating generative model evaluation metrics, and (c) improving clinical downstream tasks. In (b), extending on common medical image synthesis assessment and reporting standards, we show Fréchet Inception Distance variability based on image normalisation and radiology-specific feature extraction.
△ Less
Submitted 23 February, 2023; v1 submitted 28 September, 2022;
originally announced September 2022.
-
High-resolution synthesis of high-density breast mammograms: Application to improved fairness in deep learning based mass detection
Authors:
Lidia Garrucho,
Kaisar Kushibar,
Richard Osuala,
Oliver Diaz,
Alessandro Catanese,
Javier del Riego,
Maciej Bobowicz,
Fredrik Strand,
Laura Igual,
Karim Lekadir
Abstract:
Computer-aided detection systems based on deep learning have shown good performance in breast cancer detection. However, high-density breasts show poorer detection performance since dense tissues can mask or even simulate masses. Therefore, the sensitivity of mammography for breast cancer detection can be reduced by more than 20% in dense breasts. Additionally, extremely dense cases reported an in…
▽ More
Computer-aided detection systems based on deep learning have shown good performance in breast cancer detection. However, high-density breasts show poorer detection performance since dense tissues can mask or even simulate masses. Therefore, the sensitivity of mammography for breast cancer detection can be reduced by more than 20% in dense breasts. Additionally, extremely dense cases reported an increased risk of cancer compared to low-density breasts. This study aims to improve the mass detection performance in highdensity breasts using synthetic high-density full-field digital mammograms (FFDM) as data augmentation during breast mass detection model training. To this end, a total of five cycle-consistent GAN (CycleGAN) models using three FFDM datasets were trained for low-to-high-density image translation in highresolution mammograms. The training images were split by breast density BIRADS categories, being BI-RADS A almost entirely fatty and BI-RADS D extremely dense breasts. Our results showed that the proposed data augmentation technique improved the sensitivity and precision of mass detection in models trained with small datasets and improved the domain generalization of the models trained with large databases. In addition, the clinical realism of the synthetic images was evaluated in a reader study involving two expert radiologists and one surgical oncologist.
△ Less
Submitted 24 January, 2023; v1 submitted 20 September, 2022;
originally announced September 2022.
-
Sharing Generative Models Instead of Private Data: A Simulation Study on Mammography Patch Classification
Authors:
Zuzanna Szafranowska,
Richard Osuala,
Bennet Breier,
Kaisar Kushibar,
Karim Lekadir,
Oliver Diaz
Abstract:
Early detection of breast cancer in mammography screening via deep-learning based computer-aided detection systems shows promising potential in improving the curability and mortality rates of breast cancer. However, many clinical centres are restricted in the amount and heterogeneity of available data to train such models to (i) achieve promising performance and to (ii) generalise well across acqu…
▽ More
Early detection of breast cancer in mammography screening via deep-learning based computer-aided detection systems shows promising potential in improving the curability and mortality rates of breast cancer. However, many clinical centres are restricted in the amount and heterogeneity of available data to train such models to (i) achieve promising performance and to (ii) generalise well across acquisition protocols and domains. As sharing data between centres is restricted due to patient privacy concerns, we propose a potential solution: sharing trained generative models between centres as substitute for real patient data. In this work, we use three well known mammography datasets to simulate three different centres, where one centre receives the trained generator of Generative Adversarial Networks (GANs) from the two remaining centres in order to augment the size and heterogeneity of its training dataset. We evaluate the utility of this approach on mammography patch classification on the test set of the GAN-receiving centre using two different classification models, (a) a convolutional neural network and (b) a transformer neural network. Our experiments demonstrate that shared GANs notably increase the performance of both transformer and convolutional classification models and highlight this approach as a viable alternative to inter-centre data sharing.
△ Less
Submitted 15 April, 2022; v1 submitted 8 March, 2022;
originally announced March 2022.
-
Domain generalization in deep learning-based mass detection in mammography: A large-scale multi-center study
Authors:
Lidia Garrucho,
Kaisar Kushibar,
Socayna Jouide,
Oliver Diaz,
Laura Igual,
Karim Lekadir
Abstract:
Computer-aided detection systems based on deep learning have shown great potential in breast cancer detection. However, the lack of domain generalization of artificial neural networks is an important obstacle to their deployment in changing clinical environments. In this work, we explore the domain generalization of deep learning methods for mass detection in digital mammography and analyze in-dep…
▽ More
Computer-aided detection systems based on deep learning have shown great potential in breast cancer detection. However, the lack of domain generalization of artificial neural networks is an important obstacle to their deployment in changing clinical environments. In this work, we explore the domain generalization of deep learning methods for mass detection in digital mammography and analyze in-depth the sources of domain shift in a large-scale multi-center setting. To this end, we compare the performance of eight state-of-the-art detection methods, including Transformer-based models, trained in a single domain and tested in five unseen domains. Moreover, a single-source mass detection training pipeline is designed to improve the domain generalization without requiring images from the new domain. The results show that our workflow generalizes better than state-of-the-art transfer learning-based approaches in four out of five domains while reducing the domain shift caused by the different acquisition protocols and scanner manufacturers. Subsequently, an extensive analysis is performed to identify the covariate shifts with bigger effects on the detection performance, such as due to differences in patient age, breast density, mass size, and mass malignancy. Ultimately, this comprehensive study provides key insights and best practices for future research on domain generalization in deep learning-based breast cancer detection.
△ Less
Submitted 24 January, 2023; v1 submitted 27 January, 2022;
originally announced January 2022.
-
Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging
Authors:
Richard Osuala,
Kaisar Kushibar,
Lidia Garrucho,
Akis Linardos,
Zuzanna Szafranowska,
Stefan Klein,
Ben Glocker,
Oliver Diaz,
Karim Lekadir
Abstract:
Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in Generative Adversarial Networks…
▽ More
Despite technological and medical advances, the detection, interpretation, and treatment of cancer based on imaging data continue to pose significant challenges. These include inter-observer variability, class imbalance, dataset shifts, inter- and intra-tumour heterogeneity, malignancy determination, and treatment effect uncertainty. Given the recent advancements in Generative Adversarial Networks (GANs), data synthesis, and adversarial training, we assess the potential of these technologies to address a number of key challenges of cancer imaging. We categorise these challenges into (a) data scarcity and imbalance, (b) data access and privacy, (c) data annotation and segmentation, (d) cancer detection and diagnosis, and (e) tumour profiling, treatment planning and monitoring. Based on our analysis of 164 publications that apply adversarial training techniques in the context of cancer imaging, we highlight multiple underexplored solutions with research potential. We further contribute the Synthesis Study Trustworthiness Test (SynTRUST), a meta-analysis framework for assessing the validation rigour of medical image synthesis studies. SynTRUST is based on 26 concrete measures of thoroughness, reproducibility, usefulness, scalability, and tenability. Based on SynTRUST, we analyse 16 of the most promising cancer imaging challenge solutions and observe a high validation rigour in general, but also several desirable improvements. With this work, we strive to bridge the gap between the needs of the clinical cancer imaging community and the current and prospective research on data synthesis and adversarial networks in the artificial intelligence community.
△ Less
Submitted 27 November, 2022; v1 submitted 20 July, 2021;
originally announced July 2021.
-
Onboarding in Software Product Lines: ConceptMaps as Welcome Guides
Authors:
Maider Azanza,
Arantza Irastorza,
Raul Medeiros,
Oscar Díaz
Abstract:
With a volatile labour and technological market, onboarding is becoming increasingly important. The process of incorporating a new developer, a.k.a. the newcomer, into a software development team is reckoned to be lengthy, frustrating and expensive. Newcomers face personal, interpersonal, process and technical barriers during their incorporation, which, in turn, affects the overall productivity of…
▽ More
With a volatile labour and technological market, onboarding is becoming increasingly important. The process of incorporating a new developer, a.k.a. the newcomer, into a software development team is reckoned to be lengthy, frustrating and expensive. Newcomers face personal, interpersonal, process and technical barriers during their incorporation, which, in turn, affects the overall productivity of the whole team. This problem exacerbates for Software Product Lines (SPLs), where their size and variability combine to make onboarding even more challenging, even more so for developers that are transferred from the Application Engineering team into the Domain Engineering team, who will be our target newcomers. This work presents concept maps on the role of sensemaking scaffolds to help to introduce these newcomers into the SPL domain. Concept maps, used as knowledge visualisation tools, have been proven to be helpful for meaningful learning. Our main insight is to capture concepts of the SPL domain and their interrelationships in a concept map, and then, present them incrementally, helping newcomers grasp the SPL and aiding them in exploring it in a guided manner while avoiding information overload. This work's contributions are four-fold. First, concept maps are proposed as a representation to introduce newcomers into the SPL domain. Second, concept maps are presented as the means for a guided exploration of the SPL core assets. Third, a feature-driven concept map construction process is introduced. Last, the usefulness of concept maps as guides for SPL onboarding is tested through a formative evaluation.
Link to the online demo: url="https://rebrand.ly/wacline-cmap"
△ Less
Submitted 14 April, 2021; v1 submitted 5 March, 2021;
originally announced March 2021.
-
Deep learning reconstruction of digital breast tomosynthesis images for accurate breast density and patient-specific radiation dose estimation
Authors:
Jonas Teuwen,
Nikita Moriakov,
Christian Fedon,
Marco Caballo,
Ingrid Reiser,
Pedrag Bakic,
Eloy García,
Oliver Diaz,
Koen Michielsen,
Ioannis Sechopoulos
Abstract:
The two-dimensional nature of mammography makes estimation of the overall breast density challenging, and estimation of the true patient-specific radiation dose impossible. Digital breast tomosynthesis (DBT), a pseudo-3D technique, is now commonly used in breast cancer screening and diagnostics. Still, the severely limited 3rd dimension information in DBT has not been used, until now, to estimate…
▽ More
The two-dimensional nature of mammography makes estimation of the overall breast density challenging, and estimation of the true patient-specific radiation dose impossible. Digital breast tomosynthesis (DBT), a pseudo-3D technique, is now commonly used in breast cancer screening and diagnostics. Still, the severely limited 3rd dimension information in DBT has not been used, until now, to estimate the true breast density or the patient-specific dose. This study proposes a reconstruction algorithm for DBT based on deep learning specifically optimized for these tasks. The algorithm, which we name DBToR, is based on unrolling a proximal-dual optimization method. The proximal operators are replaced with convolutional neural networks and prior knowledge is included in the model. This extends previous work on a deep learning-based reconstruction model by providing both the primal and the dual blocks with breast thickness information, which is available in DBT. Training and testing of the model were performed using virtual patient phantoms from two different sources. Reconstruction performance, and accuracy in estimation of breast density and radiation dose, were estimated, showing high accuracy (density <+/-3%; dose <+/-20%) without bias, significantly improving on the current state-of-the-art. This work also lays the groundwork for developing a deep learning-based reconstruction algorithm for the task of image interpretation by radiologists.
△ Less
Submitted 29 March, 2021; v1 submitted 11 June, 2020;
originally announced June 2020.
-
Quality analysis of DCGAN-generated mammography lesions
Authors:
Basel Alyafi,
Oliver Diaz,
Joan C Vilanova,
Javier del Riego,
Robert Marti
Abstract:
Medical image synthesis has gained a great focus recently, especially after the introduction of Generative Adversarial Networks (GANs). GANs have been used widely to provide anatomically-plausible and diverse samples for augmentation and other applications, including segmentation and super resolution. In our previous work, Deep Convolutional GANs were used to generate synthetic mammogram lesions,…
▽ More
Medical image synthesis has gained a great focus recently, especially after the introduction of Generative Adversarial Networks (GANs). GANs have been used widely to provide anatomically-plausible and diverse samples for augmentation and other applications, including segmentation and super resolution. In our previous work, Deep Convolutional GANs were used to generate synthetic mammogram lesions, masses mainly, that could enhance the classification performance in imbalanced datasets. In this new work, a deeper investigation was carried out to explore other aspects of the generated images evaluation, i.e., realism, feature space distribution, and observers studies. t-Stochastic Neighbor Embedding (t-SNE) was used to reduce the dimensionality of real and fake images to enable 2D visualisations. Additionally, two expert radiologists performed a realism-evaluation study. Visualisations showed that the generated images have a similar feature distribution of the real ones, avoiding outliers. Moreover, Receiver Operating Characteristic (ROC) curve showed that the radiologists could not, in many cases, distinguish between synthetic and real lesions, giving 48% and 61% accuracies in a balanced sample set.
△ Less
Submitted 6 February, 2020; v1 submitted 28 November, 2019;
originally announced November 2019.
-
DCGANs for Realistic Breast Mass Augmentation in X-ray Mammography
Authors:
Basel Alyafi,
Oliver Diaz,
Robert Marti
Abstract:
Early detection of breast cancer has a major contribution to curability, and using mammographic images, this can be achieved non-invasively. Supervised deep learning, the dominant CADe tool currently, has played a great role in object detection in computer vision, but it suffers from a limiting property: the need of a large amount of labelled data. This becomes stricter when it comes to medical da…
▽ More
Early detection of breast cancer has a major contribution to curability, and using mammographic images, this can be achieved non-invasively. Supervised deep learning, the dominant CADe tool currently, has played a great role in object detection in computer vision, but it suffers from a limiting property: the need of a large amount of labelled data. This becomes stricter when it comes to medical datasets which require high-cost and time-consuming annotations. Furthermore, medical datasets are usually imbalanced, a condition that often hinders classifiers performance. The aim of this paper is to learn the distribution of the minority class to synthesise new samples in order to improve lesion detection in mammography. Deep Convolutional Generative Adversarial Networks (DCGANs) can efficiently generate breast masses. They are trained on increasing-size subsets of one mammographic dataset and used to generate diverse and realistic breast masses. The effect of including the generated images and/or applying horizontal and vertical flipping is tested in an environment where a 1:10 imbalanced dataset of masses and normal tissue patches is classified by a fully-convolutional network. A maximum of ~ 0:09 improvement of F1 score is reported by using DCGANs along with flipping augmentation over using the original images. We show that DCGANs can be used for synthesising photo-realistic breast mass patches with considerable diversity. It is demonstrated that appending synthetic images in this environment, along with flipping, outperforms the traditional augmentation method of flipping solely, offering faster improvements as a function of the training set size.
△ Less
Submitted 4 September, 2019;
originally announced September 2019.
