Showing 1–19 of 19 results for author: Fuentes, D

Project Riley: Multimodal Multi-Agent LLM Collaboration with Emotional Reasoning and Voting

Authors: Ana Rita Ortigoso, Gabriel Vieira, Daniel Fuentes, Luis Frazão, Nuno Costa, António Pereira

Abstract: This paper presents Project Riley, a novel multimodal and multi-model conversational AI architecture oriented towards the simulation of reasoning influenced by emotional states. Drawing inspiration from Pixar's Inside Out, the system comprises five distinct emotional agents - Joy, Sadness, Fear, Anger, and Disgust - that engage in structured multi-round dialogues to generate, criticise, and iterat… ▽ More This paper presents Project Riley, a novel multimodal and multi-model conversational AI architecture oriented towards the simulation of reasoning influenced by emotional states. Drawing inspiration from Pixar's Inside Out, the system comprises five distinct emotional agents - Joy, Sadness, Fear, Anger, and Disgust - that engage in structured multi-round dialogues to generate, criticise, and iteratively refine responses. A final reasoning mechanism synthesises the contributions of these agents into a coherent output that either reflects the dominant emotion or integrates multiple perspectives. The architecture incorporates both textual and visual large language models (LLMs), alongside advanced reasoning and self-refinement processes. A functional prototype was deployed locally in an offline environment, optimised for emotional expressiveness and computational efficiency. From this initial prototype, another one emerged, called Armando, which was developed for use in emergency contexts, delivering emotionally calibrated and factually accurate information through the integration of Retrieval-Augmented Generation (RAG) and cumulative context tracking. The Project Riley prototype was evaluated through user testing, in which participants interacted with the chatbot and completed a structured questionnaire assessing three dimensions: Emotional Appropriateness, Clarity and Utility, and Naturalness and Human-likeness. The results indicate strong performance in structured scenarios, particularly with respect to emotional alignment and communicative clarity. △ Less

Submitted 26 May, 2025; originally announced May 2025.

Comments: 28 pages, 5 figures. Submitted for review to Information Fusion

ACM Class: I.2.7; I.2.1; H.5.2

HashKitty: Distributed Password Analysis

Authors: Pedro Antunes, Tomás Santos, Daniel Fuentes, Luís Frazão

Abstract: This article documents the HashKitty platform, a distributed solution for password analysis based on the hashcat tool, designed to improve efficiency in both offensive and defensive security operations. The main objectives of this work are to utilise and characterise the hashcat tool, to develop a central platform that connects various computational nodes, to allow the use of nodes with different… ▽ More This article documents the HashKitty platform, a distributed solution for password analysis based on the hashcat tool, designed to improve efficiency in both offensive and defensive security operations. The main objectives of this work are to utilise and characterise the hashcat tool, to develop a central platform that connects various computational nodes, to allow the use of nodes with different equipment and manufacturers, to distribute tasks among the nodes through a web platform, and to perform distributed password analysis. The results show that the presented solution achieves the proposed objectives, demonstrating effectiveness in workload distribution and password analysis using different types of nodes based on various operating systems and architectures. The architecture of HashKitty is based on a scalable and modular distributed architecture, composed of several components such as computational nodes, integration and control software, a web platform that implements our API, and database servers. In order to achieve a fast and organised development process for our application we used multiple frameworks, runtimes and libraries. For the communication between the computational nodes and the other software we made use of websockets so that we have real-time updates between them. △ Less

Submitted 9 May, 2025; originally announced May 2025.

Comments: 6 pages, 3 figures

ACM Class: C.2.4; D.4.6; K.6.5

A Priori Generalizability Estimate for a CNN

Authors: Cito Balsells, Beatrice Riviere, David Fuentes

Abstract: We formulate truncated singular value decompositions of entire convolutional neural networks. We demonstrate the computed left and right singular vectors are useful in identifying which images the convolutional neural network is likely to perform poorly on. To create this diagnostic tool, we define two metrics: the Right Projection Ratio and the Left Projection Ratio. The Right (Left) Projection R… ▽ More We formulate truncated singular value decompositions of entire convolutional neural networks. We demonstrate the computed left and right singular vectors are useful in identifying which images the convolutional neural network is likely to perform poorly on. To create this diagnostic tool, we define two metrics: the Right Projection Ratio and the Left Projection Ratio. The Right (Left) Projection Ratio evaluates the fidelity of the projection of an image (label) onto the computed right (left) singular vectors. We observe that both ratios are able to identify the presence of class imbalance for an image classification problem. Additionally, the Right Projection Ratio, which only requires unlabeled data, is found to be correlated to the model's performance when applied to image segmentation. This suggests the Right Projection Ratio could be a useful metric to estimate how likely the model is to perform well on a sample. △ Less

Submitted 24 February, 2025; originally announced February 2025.

Learning Discontinuous Galerkin Solutions to Elliptic Problems via Small Linear Convolutional Neural Networks

Authors: Adrian Celaya, Yimo Wang, David Fuentes, Beatrice Riviere

Abstract: In recent years, there has been an increasing interest in using deep learning and neural networks to tackle scientific problems, particularly in solving partial differential equations (PDEs). However, many neural network-based methods, such as physics-informed neural networks, depend on automatic differentiation and the sampling of collocation points, which can result in a lack of interpretability… ▽ More In recent years, there has been an increasing interest in using deep learning and neural networks to tackle scientific problems, particularly in solving partial differential equations (PDEs). However, many neural network-based methods, such as physics-informed neural networks, depend on automatic differentiation and the sampling of collocation points, which can result in a lack of interpretability and lower accuracy compared to traditional numerical methods. To address this issue, we propose two approaches for learning discontinuous Galerkin solutions to PDEs using small linear convolutional neural networks. Our first approach is supervised and depends on labeled data, while our second approach is unsupervised and does not rely on any training data. In both cases, our methods use substantially fewer parameters than similar numerics-based neural networks while also demonstrating comparable accuracy to the true and DG solutions for elliptic problems. △ Less

Submitted 12 February, 2025; originally announced February 2025.

An Adaptive Collocation Point Strategy For Physics Informed Neural Networks via the QR Discrete Empirical Interpolation Method

Authors: Adrian Celaya, David Fuentes, Beatrice Riviere

Abstract: Physics-informed neural networks (PINNs) have gained significant attention for solving forward and inverse problems related to partial differential equations (PDEs). While advancements in loss functions and network architectures have improved PINN accuracy, the impact of collocation point sampling on their performance remains underexplored. Fixed sampling methods, such as uniform random sampling a… ▽ More Physics-informed neural networks (PINNs) have gained significant attention for solving forward and inverse problems related to partial differential equations (PDEs). While advancements in loss functions and network architectures have improved PINN accuracy, the impact of collocation point sampling on their performance remains underexplored. Fixed sampling methods, such as uniform random sampling and equispaced grids, can fail to capture critical regions with high solution gradients, limiting their effectiveness for complex PDEs. Adaptive methods, inspired by adaptive mesh refinement from traditional numerical methods, address this by dynamically updating collocation points during training but may overlook residual dynamics between updates, potentially losing valuable information. To overcome this limitation, we propose an adaptive collocation point selection strategy utilizing the QR Discrete Empirical Interpolation Method (QR-DEIM), a reduced-order modeling technique for efficiently approximating nonlinear functions. Our results on benchmark PDEs, including the wave, Allen-Cahn, and Burgers' equations, demonstrate that our QR-DEIM-based approach improves PINN accuracy compared to existing methods, offering a promising direction for adaptive collocation point strategies. △ Less

Submitted 18 January, 2025; v1 submitted 13 January, 2025; originally announced January 2025.

Two Stage Segmentation of Cervical Tumors using PocketNet

Authors: Awj Twam, Adrian E. Celaya, Megan C. Jacobsen, Rachel Glenn, Peng Wei, Jia Sun, Ann Klopp, Aradhana M. Venkatesan, David Fuentes

Abstract: Cervical cancer remains the fourth most common malignancy amongst women worldwide.1 Concurrent chemoradiotherapy (CRT) serves as the mainstay definitive treatment regimen for locally advanced cervical cancers and includes external beam radiation followed by brachytherapy.2 Integral to radiotherapy treatment planning is the routine contouring of both the target tumor at the level of the cervix, ass… ▽ More Cervical cancer remains the fourth most common malignancy amongst women worldwide.1 Concurrent chemoradiotherapy (CRT) serves as the mainstay definitive treatment regimen for locally advanced cervical cancers and includes external beam radiation followed by brachytherapy.2 Integral to radiotherapy treatment planning is the routine contouring of both the target tumor at the level of the cervix, associated gynecologic anatomy and the adjacent organs at risk (OARs). However, manual contouring of these structures is both time and labor intensive and associated with known interobserver variability that can impact treatment outcomes. While multiple tools have been developed to automatically segment OARs and the high-risk clinical tumor volume (HR-CTV) using computed tomography (CT) images,3,4,5,6 the development of deep learning-based tumor segmentation tools using routine T2-weighted (T2w) magnetic resonance imaging (MRI) addresses an unmet clinical need to improve the routine contouring of both anatomical structures and cervical cancers, thereby increasing quality and consistency of radiotherapy planning. This work applied a novel deep-learning model (PocketNet) to segment the cervix, vagina, uterus, and tumor(s) on T2w MRI. The performance of the PocketNet architecture was evaluated, when trained on data via five-fold cross validation. PocketNet achieved a mean Dice-Sorensen similarity coefficient (DSC) exceeding 70% for tumor segmentation and 80% for organ segmentation. Validation on a publicly available dataset from The Cancer Imaging Archive (TCIA) demonstrated the models robustness, achieving DSC scores of 67.3% for tumor segmentation and 80.8% for organ segmentation. These results suggest that PocketNet is robust to variations in contrast protocols, providing reliable segmentation of the regions of interest. △ Less

Submitted 12 February, 2025; v1 submitted 17 September, 2024; originally announced September 2024.

MIST: A Simple and Scalable End-To-End 3D Medical Imaging Segmentation Framework

Authors: Adrian Celaya, Evan Lim, Rachel Glenn, Brayden Mi, Alex Balsells, Dawid Schellingerhout, Tucker Netherton, Caroline Chung, Beatrice Riviere, David Fuentes

Abstract: Medical imaging segmentation is a highly active area of research, with deep learning-based methods achieving state-of-the-art results in several benchmarks. However, the lack of standardized tools for training, testing, and evaluating new methods makes the comparison of methods difficult. To address this, we introduce the Medical Imaging Segmentation Toolkit (MIST), a simple, modular, and end-to-e… ▽ More Medical imaging segmentation is a highly active area of research, with deep learning-based methods achieving state-of-the-art results in several benchmarks. However, the lack of standardized tools for training, testing, and evaluating new methods makes the comparison of methods difficult. To address this, we introduce the Medical Imaging Segmentation Toolkit (MIST), a simple, modular, and end-to-end medical imaging segmentation framework designed to facilitate consistent training, testing, and evaluation of deep learning-based medical imaging segmentation methods. MIST standardizes data analysis, preprocessing, and evaluation pipelines, accommodating multiple architectures and loss functions. This standardization ensures reproducible and fair comparisons across different methods. We detail MIST's data format requirements, pipelines, and auxiliary features and demonstrate its efficacy using the BraTS Adult Glioma Post-Treatment Challenge dataset. Our results highlight MIST's ability to produce accurate segmentation masks and its scalability across multiple GPUs, showcasing its potential as a powerful tool for future medical imaging research and development. △ Less

Submitted 18 November, 2024; v1 submitted 31 July, 2024; originally announced July 2024.

Comments: Submitted to BraTS 2024

Sifting out communities in large sparse networks

Authors: Sharlee Climer, Kenneth Smith Jr, Wei Yang, Lisa de las Fuentes, Victor G. Dávila-Román, C. Charles Gu

Abstract: Research data sets are growing to unprecedented sizes and network modeling is commonly used to extract complex relationships in diverse domains, such as genetic interactions involved in disease, logistics, and social communities. As the number of nodes increases in a network, an increasing sparsity of edges is a practical limitation due to memory restrictions. Moreover, many of these sparse networ… ▽ More Research data sets are growing to unprecedented sizes and network modeling is commonly used to extract complex relationships in diverse domains, such as genetic interactions involved in disease, logistics, and social communities. As the number of nodes increases in a network, an increasing sparsity of edges is a practical limitation due to memory restrictions. Moreover, many of these sparse networks exhibit very large numbers of nodes with no adjacent edges, as well as disjoint components of nodes with no edges connecting them. A prevalent aim in network modeling is the identification of clusters, or communities, of nodes that are highly interrelated. Several definitions of strong community structure have been introduced to facilitate this task, each with inherent assumptions and biases. We introduce an intuitive objective function for quantifying the quality of clustering results in large sparse networks. We utilize a two-step method for identifying communities which is especially well-suited for this domain as the first step efficiently divides the network into the disjoint components, while the second step optimizes clustering of the produced components based on the new objective. Using simulated networks, optimization based on the new objective function consistently yields significantly higher accuracy than those based on the modularity function, with the widest gaps appearing for the noisiest networks. Additionally, applications to benchmark problems illustrate the intuitive correctness of our approach. Finally, the practicality of our approach is demonstrated in real-world data in which we identify complex genetic interactions in large-scale networks comprised of tens of thousands of nodes. Based on these three different types of trials, our results clearly demonstrate the usefulness of our two-step procedure and the accuracy of our simple objective. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Heterogeneous Image-based Classification Using Distributional Data Analysis

Authors: Alec Reinhardt, Newsha Nikzad, Raven J. Hollis, Galia Jacobson, Millicent A. Roach, Mohamed Badawy, Peter Chul Park, Laura Beretta, Prasun K Jalal, David T. Fuentes, Eugene J. Koay, Suprateek Kundu

Abstract: Diagnostic imaging has gained prominence as potential biomarkers for early detection and diagnosis in a diverse array of disorders including cancer. However, existing methods routinely face challenges arising from various factors such as image heterogeneity. We develop a novel imaging-based distributional data analysis (DDA) approach that incorporates the probability (quantile) distribution of the… ▽ More Diagnostic imaging has gained prominence as potential biomarkers for early detection and diagnosis in a diverse array of disorders including cancer. However, existing methods routinely face challenges arising from various factors such as image heterogeneity. We develop a novel imaging-based distributional data analysis (DDA) approach that incorporates the probability (quantile) distribution of the pixel-level features as covariates. The proposed approach uses a smoothed quantile distribution (via a suitable basis representation) as functional predictors in a scalar-on-functional quantile regression model. Some distinctive features of the proposed approach include the ability to: (i) account for heterogeneity within the image; (ii) incorporate granular information spanning the entire distribution; and (iii) tackle variability in image sizes for unregistered images in cancer applications. Our primary goal is risk prediction in Hepatocellular carcinoma that is achieved via predicting the change in tumor grades at post-diagnostic visits using pre-diagnostic enhancement pattern mapping (EPM) images of the liver. Along the way, the proposed DDA approach is also used for case versus control diagnosis and risk stratification objectives. Our analysis reveals that when coupled with global structural radiomics features derived from the corresponding T1-MRI scans, the proposed smoothed quantile distributions derived from EPM images showed considerable improvements in sensitivity and comparable specificity in contrast to classification based on routinely used summary measures that do not account for image heterogeneity. Given that there are limited predictive modeling approaches based on heterogeneous images in cancer, the proposed method is expected to provide considerable advantages in image-based early detection and risk prediction. △ Less

Submitted 11 March, 2024; originally announced March 2024.

Comments: 16, 2 figures, 3 tables

Solutions to Elliptic and Parabolic Problems via Finite Difference Based Unsupervised Small Linear Convolutional Neural Networks

Authors: Adrian Celaya, Keegan Kirk, David Fuentes, Beatrice Riviere

Abstract: In recent years, there has been a growing interest in leveraging deep learning and neural networks to address scientific problems, particularly in solving partial differential equations (PDEs). However, many neural network-based methods like PINNs rely on auto differentiation and sampling collocation points, leading to a lack of interpretability and lower accuracy than traditional numerical method… ▽ More In recent years, there has been a growing interest in leveraging deep learning and neural networks to address scientific problems, particularly in solving partial differential equations (PDEs). However, many neural network-based methods like PINNs rely on auto differentiation and sampling collocation points, leading to a lack of interpretability and lower accuracy than traditional numerical methods. As a result, we propose a fully unsupervised approach, requiring no training data, to estimate finite difference solutions for PDEs directly via small linear convolutional neural networks. Our proposed approach uses substantially fewer parameters than similar finite difference-based approaches while also demonstrating comparable accuracy to the true solution for several selected elliptic and parabolic problems compared to the finite difference method. △ Less

Submitted 22 April, 2024; v1 submitted 31 October, 2023; originally announced November 2023.

Comments: Submitted to CMA, under review

Distributed Conditional GAN (discGAN) For Synthetic Healthcare Data Generation

Authors: David Fuentes, Diana McSpadden, Sodiq Adewole

Abstract: In this paper, we propose a distributed Generative Adversarial Networks (discGANs) to generate synthetic tabular data specific to the healthcare domain. While using GANs to generate images has been well studied, little to no attention has been given to generation of tabular data. Modeling distributions of discrete and continuous tabular data is a non-trivial task with high utility. We applied disc… ▽ More In this paper, we propose a distributed Generative Adversarial Networks (discGANs) to generate synthetic tabular data specific to the healthcare domain. While using GANs to generate images has been well studied, little to no attention has been given to generation of tabular data. Modeling distributions of discrete and continuous tabular data is a non-trivial task with high utility. We applied discGAN to model non-Gaussian multi-modal healthcare data. We generated 249,000 synthetic records from original 2,027 eICU dataset. We evaluated the performance of the model using machine learning efficacy, the Kolmogorov-Smirnov (KS) test for continuous variables and chi-squared test for discrete variables. Our results show that discGAN was able to generate data with distributions similar to the real data. △ Less

Submitted 9 April, 2023; originally announced April 2023.

FMG-Net and W-Net: Multigrid Inspired Deep Learning Architectures For Medical Imaging Segmentation

Authors: Adrian Celaya, Beatrice Riviere, David Fuentes

Abstract: Accurate medical imaging segmentation is critical for precise and effective medical interventions. However, despite the success of convolutional neural networks (CNNs) in medical image segmentation, they still face challenges in handling fine-scale features and variations in image scales. These challenges are particularly evident in complex and challenging segmentation tasks, such as the BraTS mul… ▽ More Accurate medical imaging segmentation is critical for precise and effective medical interventions. However, despite the success of convolutional neural networks (CNNs) in medical image segmentation, they still face challenges in handling fine-scale features and variations in image scales. These challenges are particularly evident in complex and challenging segmentation tasks, such as the BraTS multi-label brain tumor segmentation challenge. In this task, accurately segmenting the various tumor sub-components, which vary significantly in size and shape, remains a significant challenge, with even state-of-the-art methods producing substantial errors. Therefore, we propose two architectures, FMG-Net and W-Net, that incorporate the principles of geometric multigrid methods for solving linear systems of equations into CNNs to address these challenges. Our experiments on the BraTS 2020 dataset demonstrate that both FMG-Net and W-Net outperform the widely used U-Net architecture regarding tumor subcomponent segmentation accuracy and training efficiency. These findings highlight the potential of incorporating the principles of multigrid methods into CNNs to improve the accuracy and efficiency of medical imaging segmentation. △ Less

Submitted 10 November, 2023; v1 submitted 5 April, 2023; originally announced April 2023.

Comments: To appear in LatinX in AI (LXAI) Research Workshop @ NeurIPS 2023

EGFR mutation prediction using F18-FDG PET-CT based radiomics features in non-small cell lung cancer

Authors: Hector Henriquez, Diana Fuentes, Francisco Suarez, Patricio Gonzalez

Abstract: Lung cancer is the leading cause of cancer death in the world. Accurate determination of the EGFR (epidermal growth factor receptor) mutation status is highly relevant for the proper treatment of this patients. Purpose: The aim of this study was to predict the mutational status of the EGFR in non-small cell lung cancer patients using radiomics features extracted from PET-CT images. Methods: Retros… ▽ More Lung cancer is the leading cause of cancer death in the world. Accurate determination of the EGFR (epidermal growth factor receptor) mutation status is highly relevant for the proper treatment of this patients. Purpose: The aim of this study was to predict the mutational status of the EGFR in non-small cell lung cancer patients using radiomics features extracted from PET-CT images. Methods: Retrospective study that involve 34 patients with lung cancer confirmed by histology and EGFR status mutation assessment. A total of 2.205 radiomics features were extracted from manual segmentation of the PET-CT images using pyradiomics library. Both computed tomography and positron emission tomography images were used. All images were acquired with intravenous iodinated contrast and F18-FDG. Preprocessing includes resampling, normalization, and discretization of the pixel intensity. Three methods were used for the feature selection process: backward selection (set 1), forward selection (set 2), and feature importance analysis of random forest model (set 3). Nine machine learning methods were used for radiomics model building. Results: 35.2% of patients had EGFR mutation, without significant differences in age, gender, tumor size and SUVmax. After the feature selection process 6, 7 and 17 radiomics features were selected, respectively in each group. The best performances were obtained by Ridge Regression in set 1: AUC of 0.826 (95% CI, 0.811 - 0.839), Random Forest in set 2: AUC of 0.823 (95% CI, 0.808 - 0.838) and Neural Network in set 3: AUC of 0.821 (95% CI, 0.808 - 0.835). Conclusion: The radiomics features analysis has the potential of predicting clinically relevant mutations in lung cancer patients through a non-invasive methodology. △ Less

Submitted 13 March, 2023; originally announced March 2023.

Comments: Conference paper. International Conference on Biomedical and Health Informatics - ICBHI 2022

A Generalized Surface Loss for Reducing the Hausdorff Distance in Medical Imaging Segmentation

Authors: Adrian Celaya, Beatrice Riviere, David Fuentes

Abstract: Within medical imaging segmentation, the Dice coefficient and Hausdorff-based metrics are standard measures of success for deep learning models. However, modern loss functions for medical image segmentation often only consider the Dice coefficient or similar region-based metrics during training. As a result, segmentation architectures trained over such loss functions run the risk of achieving high… ▽ More Within medical imaging segmentation, the Dice coefficient and Hausdorff-based metrics are standard measures of success for deep learning models. However, modern loss functions for medical image segmentation often only consider the Dice coefficient or similar region-based metrics during training. As a result, segmentation architectures trained over such loss functions run the risk of achieving high accuracy for the Dice coefficient but low accuracy for Hausdorff-based metrics. Low accuracy on Hausdorff-based metrics can be problematic for applications such as tumor segmentation, where such benchmarks are crucial. For example, high Dice scores accompanied by significant Hausdorff errors could indicate that the predictions fail to detect small tumors. We propose the Generalized Surface Loss function, a novel loss function to minimize Hausdorff-based metrics with more desirable numerical properties than current methods and with weighting terms for class imbalance. Our loss function outperforms other losses when tested on the LiTS and BraTS datasets using the state-of-the-art nnUNet architecture. These results suggest we can improve medical imaging segmentation accuracy with our novel loss function. △ Less

Submitted 23 January, 2024; v1 submitted 7 February, 2023; originally announced February 2023.

Mutual-Information Based Optimal Experimental Design for Hyperpolarized $^{13}$C-Pyruvate MRI

Authors: Prashant K. Jha, Christopher Walker, Drew Mitchell, J. Tinsley Oden, Dawid Schellingerhout, James A. Bankson, David T. Fuentes

Abstract: A key parameter of interest recovered from hyperpolarized (HP) MRI measurements is the apparent pyruvate-to-lactate exchange rate, $k_{PL}$, for measuring tumor metabolism. This manuscript presents an information-theory-based optimal experimental design (OED) approach that minimizes the uncertainty in the rate parameter, $k_{PL}$, recovered from HP-MRI measurements. Mutual information (MI) is empl… ▽ More A key parameter of interest recovered from hyperpolarized (HP) MRI measurements is the apparent pyruvate-to-lactate exchange rate, $k_{PL}$, for measuring tumor metabolism. This manuscript presents an information-theory-based optimal experimental design (OED) approach that minimizes the uncertainty in the rate parameter, $k_{PL}$, recovered from HP-MRI measurements. Mutual information (MI) is employed to measure the information content of the HP measurements with respect to the first-order exchange kinetics of the pyruvate conversion to lactate. Flip angles of the pulse sequence acquisition are optimized with respect to the mutual information. Further, a spatially varying model (high-fidelity) based on the Block-Torrey equations is proposed and utilized as a control. A time-varying flip angle scheme leads to a higher parameter optimization that can further improve the quantitative value of mutual information over a constant flip angle scheme. However, the constant flip angle scheme leads to the best accuracy and precision when considering inference from noise-corrupted data. For the particular MRI data examined here, pyruvate and lactate flip angles of 35 and 28 degrees, respectively, were the best choice in terms of accuracy and precision of the parameter recovery. Moreover, the recovery of rate parameter $k_{PL}$ from the data generated from the high-fidelity model highlights the influence of diffusion and strength of vascular source on the recovered rate parameter. Since the existing pharmacokinetic models for HP-MRI do not account for spatial variation, the optimized design parameters may not be fully optimal in a more general 3D setting. △ Less

Submitted 7 July, 2022; v1 submitted 24 June, 2022; originally announced June 2022.

Comments: 20 pages, 8 figures

MSC Class: 62B10; 92B05; 62P10

Automation of Radiation Treatment Planning for Rectal Cancer

Authors: Kai Huang, Prajnan Das, Adenike M. Olanrewaju, Carlos Cardenas, David Fuentes, Lifei Zhang, Donald Hancock, Hannah Simonds, Dong Joo Rhee, Sam Beddar, Tina Marie Briere, Laurence Court

Abstract: To develop an automated workflow for rectal cancer three-dimensional conformal radiotherapy treatment planning that combines deep-learning(DL) aperture predictions and forward-planning algorithms. We designed an algorithm to automate the clinical workflow for planning with field-in-field. DL models were trained, validated, and tested on 555 patients to automatically generate aperture shapes for pr… ▽ More To develop an automated workflow for rectal cancer three-dimensional conformal radiotherapy treatment planning that combines deep-learning(DL) aperture predictions and forward-planning algorithms. We designed an algorithm to automate the clinical workflow for planning with field-in-field. DL models were trained, validated, and tested on 555 patients to automatically generate aperture shapes for primary and boost fields. Network inputs were digitally reconstructed radiography, gross tumor volume(GTV), and nodal GTV. A physician scored each aperture for 20 patients on a 5-point scale(>3 acceptable). A planning algorithm was then developed to create a homogeneous dose using a combination of wedges and subfields. The algorithm iteratively identifies a hotspot volume, creates a subfield, and optimizes beam weight all without user intervention. The algorithm was tested on 20 patients using clinical apertures with different settings, and the resulting plans(4 plans/patient) were scored by a physician. The end-to-end workflow was tested and scored by a physician on 39 patients using DL-generated apertures and planning algorithms. The predicted apertures had Dice scores of 0.95, 0.94, and 0.90 for posterior-anterior, laterals, and boost fields, respectively. 100%, 95%, and 87.5% of the posterior-anterior, laterals, and boost apertures were scored as clinically acceptable, respectively. Wedged and non-wedged plans were clinically acceptable for 85% and 50% of patients, respectively. The final plans hotspot dose percentage was reduced from 121%($\pm$ 14%) to 109%($\pm$ 5%) of prescription dose. The integrated end-to-end workflow of automatically generated apertures and optimized field-in-field planning gave clinically acceptable plans for 38/39(97%) of patients. We have successfully automated the clinical workflow for generating radiotherapy plans for rectal cancer for our institution. △ Less

Submitted 18 July, 2022; v1 submitted 26 April, 2022; originally announced April 2022.

Journal ref: Journal of Applied Clinical Medical Physics 2022

Correlation between image quality metrics of magnetic resonance images and the neural network segmentation accuracy

Authors: Rajarajeswari Muthusivarajan, Adrian Celaya, Joshua P. Yung, Satish Viswanath, Daniel S. Marcus, Caroline Chung, David Fuentes

Abstract: Deep neural networks with multilevel connections process input data in complex ways to learn the information.A networks learning efficiency depends not only on the complex neural network architecture but also on the input training images.Medical image segmentation with deep neural networks for skull stripping or tumor segmentation from magnetic resonance images enables learning both global and loc… ▽ More Deep neural networks with multilevel connections process input data in complex ways to learn the information.A networks learning efficiency depends not only on the complex neural network architecture but also on the input training images.Medical image segmentation with deep neural networks for skull stripping or tumor segmentation from magnetic resonance images enables learning both global and local features of the images.Though medical images are collected in a controlled environment,there may be artifacts or equipment based variance that cause inherent bias in the input set.In this study, we investigated the correlation between the image quality metrics of MR images with the neural network segmentation accuracy.For that we have used the 3D DenseNet architecture and let the network trained on the same input but applying different methodologies to select the training data set based on the IQM values.The difference in the segmentation accuracy between models based on the random training inputs with IQM based training inputs shed light on the role of image quality metrics on segmentation accuracy.By running the image quality metrics to choose the training inputs,further we may tune the learning efficiency of the network and the segmentation accuracy. △ Less

Submitted 1 November, 2021; originally announced November 2021.

PocketNet: A Smaller Neural Network for Medical Image Analysis

Authors: Adrian Celaya, Jonas A. Actor, Rajarajeswari Muthusivarajan, Evan Gates, Caroline Chung, Dawid Schellingerhout, Beatrice Riviere, David Fuentes

Abstract: Medical imaging deep learning models are often large and complex, requiring specialized hardware to train and evaluate these models. To address such issues, we propose the PocketNet paradigm to reduce the size of deep learning models by throttling the growth of the number of channels in convolutional neural networks. We demonstrate that, for a range of segmentation and classification tasks, Pocket… ▽ More Medical imaging deep learning models are often large and complex, requiring specialized hardware to train and evaluate these models. To address such issues, we propose the PocketNet paradigm to reduce the size of deep learning models by throttling the growth of the number of channels in convolutional neural networks. We demonstrate that, for a range of segmentation and classification tasks, PocketNet architectures produce results comparable to that of conventional neural networks while reducing the number of parameters by multiple orders of magnitude, using up to 90% less GPU memory, and speeding up training times by up to 40%, thereby allowing such models to be trained and deployed in resource-constrained settings. △ Less

Submitted 18 September, 2022; v1 submitted 21 April, 2021; originally announced April 2021.

Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge

Authors: Spyridon Bakas, Mauricio Reyes, Andras Jakab, Stefan Bauer, Markus Rempfler, Alessandro Crimi, Russell Takeshi Shinohara, Christoph Berger, Sung Min Ha, Martin Rozycki, Marcel Prastawa, Esther Alberts, Jana Lipkova, John Freymann, Justin Kirby, Michel Bilello, Hassan Fathallah-Shaykh, Roland Wiest, Jan Kirschke, Benedikt Wiestler, Rivka Colen, Aikaterini Kotrotsou, Pamela Lamontagne, Daniel Marcus, Mikhail Milchenko , et al. (402 additional authors not shown)

Abstract: Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles dissem… ▽ More Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset. △ Less

Submitted 23 April, 2019; v1 submitted 5 November, 2018; originally announced November 2018.

Comments: The International Multimodal Brain Tumor Segmentation (BraTS) Challenge