In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
Authors:
Amelia Jiménez-Sánchez,
Natalia-Rozalia Avlona,
Sarah de Boer,
Víctor M. Campello,
Aasa Feragen,
Enzo Ferrante,
Melanie Ganz,
Judy Wawira Gichoya,
Camila González,
Steff Groefsema,
Alessa Hering,
Adam Hulman,
Leo Joskowicz,
Dovile Juodelyte,
Melih Kandemir,
Thijs Kooi,
Jorge del Pozo Lérida,
Livie Yumeng Li,
Andre Pacheco,
Tim Rädsch,
Mauricio Reyes,
Théo Sourget,
Bram van Ginneken,
David Wen,
Nina Weng
, et al. (4 additional authors not shown)
Abstract:
Datasets play a critical role in medical imaging research, yet issues such as label quality, shortcuts, and metadata are often overlooked. This lack of attention may harm the generalizability of algorithms and, consequently, negatively impact patient outcomes. While existing medical imaging literature reviews mostly focus on machine learning (ML) methods, with only a few focusing on datasets for s…
▽ More
Datasets play a critical role in medical imaging research, yet issues such as label quality, shortcuts, and metadata are often overlooked. This lack of attention may harm the generalizability of algorithms and, consequently, negatively impact patient outcomes. While existing medical imaging literature reviews mostly focus on machine learning (ML) methods, with only a few focusing on datasets for specific applications, these reviews remain static -- they are published once and not updated thereafter. This fails to account for emerging evidence, such as biases, shortcuts, and additional annotations that other researchers may contribute after the dataset is published. We refer to these newly discovered findings of datasets as research artifacts. To address this gap, we propose a living review that continuously tracks public datasets and their associated research artifacts across multiple medical imaging applications. Our approach includes a framework for the living review to monitor data documentation artifacts, and an SQL database to visualize the citation relationships between research artifact and dataset. Lastly, we discuss key considerations for creating medical imaging datasets, review best practices for data annotation, discuss the significance of shortcuts and demographic diversity, and emphasize the importance of managing datasets throughout their entire lifecycle. Our demo is publicly available at http://inthepicture.itu.dk/.
△ Less
Submitted 2 June, 2025; v1 submitted 18 January, 2025;
originally announced January 2025.
