Showing 1–50 of 183 results for author: Grant, E

Small-Cell-Based Fast Active Learning of Machine Learning Interatomic Potentials

Authors: Zijian Meng, Hao Sun, Edmanuel Torres, Christopher Maxwell, Ryan Eric Grant, Laurent Karim Béland

Abstract: Machine learning interatomic potentials (MLIPs) are often trained with on-the-fly active learning, where sampled configurations from atomistic simulations are added to the training set. However, this approach is limited by the high computational cost of ab initio calculations for large systems. Recent works have shown that MLIPs trained on small cells (1-8 atoms) rival the accuracy of large-cell m… ▽ More Machine learning interatomic potentials (MLIPs) are often trained with on-the-fly active learning, where sampled configurations from atomistic simulations are added to the training set. However, this approach is limited by the high computational cost of ab initio calculations for large systems. Recent works have shown that MLIPs trained on small cells (1-8 atoms) rival the accuracy of large-cell models (100s of atoms) at far lower computational cost. Herein, we refer to these as small-cell and large-cell training, respectively. In this work, we iterate on earlier small-cell training approaches and characterize our resultant small-cell protocol. Potassium and sodium-potassium systems were studied: the former, a simpler system benchmarked in detail; the latter, a more complex binary system for further validation. Our small-cell training approach achieves up to two orders of magnitude of cost savings compared to large-cell (54-atom) training, with some training runs requiring fewer than 120 core-hours. Static and thermodynamic properties predicted using the MLIPs were evaluated, with small-cell training in both systems yielding strong ab initio agreement. Small cells appear to encode the necessary information to model complex large-scale phenomena--solid-liquid interfaces, critical exponents, diverse concentrations--even when the training cells themselves are too small to accommodate these phenomena. Based on these tests, we provide analysis and recommendations. △ Less

Submitted 9 April, 2025; originally announced April 2025.

Comments: 19 pages, 15 figures. Pre-print with appendices and supplementary information

Constraints on Non-Thermal Pressure at galaxy cluster outskirts from a Joint SPT and XMM-Newton Analysis

Authors: Arnab Sarkar, Michael McDonald, Lindsey Bleem, Mark Bautz, Bradford A. Benson, Priyanka Chakraborty, Catherine E. Grant, Christine Jones, Florian Kéruzoré, Eric D. Miller, Scott Randall, Charles Romero, Taweewat Somboonpanyakul, Yuanyuan Su

Abstract: We present joint South Pole Telescope (SPT) and XMM-Newton observations of 8 massive galaxy clusters (0.8--1.7$\times$10$^{15}$ M$_{\odot}$) spanning a redshift range of 0.16 to 0.35. Employing a novel SZ+X-ray fitting technique, we effectively constrain the thermodynamic properties of these clusters out to the virial radius. The resulting best-fit electron density, deprojected temperature, and de… ▽ More We present joint South Pole Telescope (SPT) and XMM-Newton observations of 8 massive galaxy clusters (0.8--1.7$\times$10$^{15}$ M$_{\odot}$) spanning a redshift range of 0.16 to 0.35. Employing a novel SZ+X-ray fitting technique, we effectively constrain the thermodynamic properties of these clusters out to the virial radius. The resulting best-fit electron density, deprojected temperature, and deprojected pressure profiles are in good agreement with previous observations of massive clusters. For the majority of the cluster sample (5 out of 8 clusters), the entropy profiles exhibit a self-similar behavior near the virial radius. We further derive hydrostatic mass, gas mass, and gas fraction profiles for all clusters up to the virial radius. Comparing the enclosed gas fraction profiles with the universal gas fraction profile, we obtain non-thermal pressure fraction profiles for our cluster sample at $>$$R_{500}$, demonstrating a steeper increase between $R_{500}$ and $R_{200}$ that is consistent with the hydrodynamical simulations. Our analysis yields non-thermal pressure fraction ranges of 8--28% (median: 15 $\pm$ 11%) at $R_{500}$ and 21--35% (median: 27 $\pm$ 12%) at $R_{200}$. Notably, weak-lensing mass measurements are available for only four clusters in our sample, and our recovered total cluster masses, after accounting for non-thermal pressure, are consistent with these measurements. △ Less

Submitted 31 March, 2025; originally announced April 2025.

Comments: 6 figures, three tables. Accepted for publication in the Astrophysical Journal Letters

Using the XMM-Newton small window mode to investigate systematic uncertainties in the particle background of X-ray charge-coupled device detectors

Authors: Gerrit Schellenberger, Ralph Kraft, Paul Nulsen, Eric D. Miller, Marshall W. Bautz, Catherine E. Grant, Dan Wilkins, Steven Allen, Silvano Molendi, David N. Burrows, Abraham D. Falcone, Valentina Fioretti, Richard F. Foster, David Hall, Michael W. J. Hubbard, Emanuele Perinati, Artem Poliszczuk, Arne Rau, Arnab Sarkar, Benjamin Schneider

Abstract: The level and uncertainty of the particle induced background in CCD detectors plays a crucial role for future X-ray instruments, such as the Wide Field Imager (WFI) onboard Athena. To mitigate the background systematic uncertainties, which will limit the Athena science goals, we aim to understand the relationship between the energetic charged particles interacting in the detector and satellite, an… ▽ More The level and uncertainty of the particle induced background in CCD detectors plays a crucial role for future X-ray instruments, such as the Wide Field Imager (WFI) onboard Athena. To mitigate the background systematic uncertainties, which will limit the Athena science goals, we aim to understand the relationship between the energetic charged particles interacting in the detector and satellite, and the instrumental science background to an unprecedented level. These particles produce easily identified "cosmic-ray tracks" along with less easily identified signals produced by secondary particles, e.g., X-rays generated by particle interactions with the instrument and indistinguishable from genuine sky X-rays. We utilize the Small Window Mode of the PN camera onboard XMM-Newton to understand the time, spatial and energy dependence of the various background components, particularly the particle induced background. While the distribution of particle events follows expected detector readout patterns, we find a particle track length distribution inconsistent with the simple, isotropic model. We also find that the detector mode-specific readout results in a shifted Cu fluorescent line. We illustrate that on long timescales the variability of the particle background correlates well with the solar cycle. This 20-year lightcurve, can be reproduced by a particle detector onboard Chandra, the HRC anti-coincidence shield. We conclude that the self-anti-coincidence method of removing X-ray-like events near detected particle tracks in the same frame can be optimized with the inclusion of additional information, such as the energy of the X-ray. The results presented here are relevant for any future pixelated X-ray imaging detector, and could allow the WFI to probe to truly faint X-ray surface brightness. △ Less

Submitted 18 March, 2025; v1 submitted 6 March, 2025; originally announced March 2025.

Comments: 34 pages, 16 figures. Accepted for publication in JATIS. Includes minor corrections

Spatial regularisation for improved accuracy and interpretability in keypoint-based registration

Authors: Benjamin Billot, Ramya Muthukrishnan, Esra Abaci-Turk, P. Ellen Grant, Nicholas Ayache, Hervé Delingette, Polina Golland

Abstract: Unsupervised registration strategies bypass requirements in ground truth transforms or segmentations by optimising similarity metrics between fixed and moved volumes. Among these methods, a recent subclass of approaches based on unsupervised keypoint detection stand out as very promising for interpretability. Specifically, these methods train a network to predict feature maps for fixed and moving… ▽ More Unsupervised registration strategies bypass requirements in ground truth transforms or segmentations by optimising similarity metrics between fixed and moved volumes. Among these methods, a recent subclass of approaches based on unsupervised keypoint detection stand out as very promising for interpretability. Specifically, these methods train a network to predict feature maps for fixed and moving images, from which explainable centres of mass are computed to obtain point clouds, that are then aligned in closed-form. However, the features returned by the network often yield spatially diffuse patterns that are hard to interpret, thus undermining the purpose of keypoint-based registration. Here, we propose a three-fold loss to regularise the spatial distribution of the features. First, we use the KL divergence to model features as point spread functions that we interpret as probabilistic keypoints. Then, we sharpen the spatial distributions of these features to increase the precision of the detected landmarks. Finally, we introduce a new repulsive loss across keypoints to encourage spatial diversity. Overall, our loss considerably improves the interpretability of the features, which now correspond to precise and anatomically meaningful landmarks. We demonstrate our three-fold loss in foetal rigid motion tracking and brain MRI affine registration tasks, where it not only outperforms state-of-the-art unsupervised strategies, but also bridges the gap with state-of-the-art supervised methods. Our code is available at https://github.com/BenBillot/spatial_regularisation. △ Less

Submitted 7 March, 2025; v1 submitted 6 March, 2025; originally announced March 2025.

Comments: under review

Nonlinear dynamics of localization in neural receptive fields

Authors: Leon Lufkin, Andrew M. Saxe, Erin Grant

Abstract: Localized receptive fields -- neurons that are selective for certain contiguous spatiotemporal features of their input -- populate early sensory regions of the mammalian brain. Unsupervised learning algorithms that optimize explicit sparsity or independence criteria replicate features of these localized receptive fields, but fail to explain directly how localization arises through learning without… ▽ More Localized receptive fields -- neurons that are selective for certain contiguous spatiotemporal features of their input -- populate early sensory regions of the mammalian brain. Unsupervised learning algorithms that optimize explicit sparsity or independence criteria replicate features of these localized receptive fields, but fail to explain directly how localization arises through learning without efficient coding, as occurs in early layers of deep neural networks and might occur in early sensory regions of biological systems. We consider an alternative model in which localized receptive fields emerge without explicit top-down efficiency constraints -- a feedforward neural network trained on a data model inspired by the structure of natural images. Previous work identified the importance of non-Gaussian statistics to localization in this setting but left open questions about the mechanisms driving dynamical emergence. We address these questions by deriving the effective learning dynamics for a single nonlinear neuron, making precise how higher-order statistical properties of the input data drive emergent localization, and we demonstrate that the predictions of these effective dynamics extend to the many-neuron setting. Our analysis provides an alternative explanation for the ubiquity of localization as resulting from the nonlinear dynamics of learning in neural circuits. △ Less

Submitted 28 January, 2025; originally announced January 2025.

Comments: Appeared at the 38th Conference on Neural Information Processing Systems (NeurIPS 2024); spotlight presentation

International Astrophysical Consortium for High-energy Calibration: Summary of the 16th IACHEC Workshop

Authors: C. E. Grant, K. K. Madsen, V. Burwitz, K. Forster, M. Guainazzi, V. L. Kashyap, H. L. Marshall, C. B. Markwardt, E. D. Miller, L. Natalucci, P. P. Plucinsky, M. Shidatsu, Y. Terada

Abstract: In this report we summarize the activities of the International Astronomical Consortium for High Energy Calibration (IACHEC) from the 16th IACHEC Workshop at Parador de La Granja, Spain. Sixty-one scientists directly involved in the calibration of operational and future high-energy missions gathered during 3.5 days to discuss the status of the cross-calibration between the current international co… ▽ More In this report we summarize the activities of the International Astronomical Consortium for High Energy Calibration (IACHEC) from the 16th IACHEC Workshop at Parador de La Granja, Spain. Sixty-one scientists directly involved in the calibration of operational and future high-energy missions gathered during 3.5 days to discuss the status of the cross-calibration between the current international complement of X-ray observatories, and the possibilities to improve it. This summary consists of reports from the Working Groups with topics ranging across: the identification and characterization of standard calibration sources, multi-observatory cross-calibration campaigns, appropriate and new statistical techniques, calibration of instruments and characterization of background, preservation of knowledge, and results for the benefit of the astronomical community. △ Less

Submitted 27 January, 2025; originally announced January 2025.

Comments: 11 pages; Summary of the 16th IACHEC meeting (May 2024). arXiv admin note: text overlap with arXiv:2407.09432

RAMQA: A Unified Framework for Retrieval-Augmented Multi-Modal Question Answering

Authors: Yang Bai, Christan Earl Grant, Daisy Zhe Wang

Abstract: Multi-modal retrieval-augmented Question Answering (MRAQA), integrating text and images, has gained significant attention in information retrieval (IR) and natural language processing (NLP). Traditional ranking methods rely on small encoder-based language models, which are incompatible with modern decoder-based generative large language models (LLMs) that have advanced various NLP tasks. To bridge… ▽ More Multi-modal retrieval-augmented Question Answering (MRAQA), integrating text and images, has gained significant attention in information retrieval (IR) and natural language processing (NLP). Traditional ranking methods rely on small encoder-based language models, which are incompatible with modern decoder-based generative large language models (LLMs) that have advanced various NLP tasks. To bridge this gap, we propose RAMQA, a unified framework combining learning-to-rank methods with generative permutation-enhanced ranking techniques. We first train a pointwise multi-modal ranker using LLaVA as the backbone. Then, we apply instruction tuning to train a LLaMA model for re-ranking the top-k documents using an innovative autoregressive multi-task learning approach. Our generative ranking model generates re-ranked document IDs and specific answers from document candidates in various permutations. Experiments on two MRAQA benchmarks, WebQA and MultiModalQA, show significant improvements over strong baselines, highlighting the effectiveness of our approach. Code and data are available at: https://github.com/TonyBY/RAMQA △ Less

Submitted 22 January, 2025; originally announced January 2025.

Comments: Accepted by NAACL 2025 Findings

Relation U-Net

Authors: Sheng He, Rina Bao, P. Ellen Grant, Yangming Ou

Abstract: Towards clinical interpretations, this paper presents a new ''output-with-confidence'' segmentation neural network with multiple input images and multiple output segmentation maps and their pairwise relations. A confidence score of the test image without ground-truth can be estimated from the difference among the estimated relation maps. We evaluate the method based on the widely used vanilla U-Ne… ▽ More Towards clinical interpretations, this paper presents a new ''output-with-confidence'' segmentation neural network with multiple input images and multiple output segmentation maps and their pairwise relations. A confidence score of the test image without ground-truth can be estimated from the difference among the estimated relation maps. We evaluate the method based on the widely used vanilla U-Net for segmentation and our new model is named Relation U-Net which can output segmentation maps of the input images as well as an estimated confidence score of the test image without ground-truth. Experimental results on four public datasets show that Relation U-Net can not only provide better accuracy than vanilla U-Net but also estimate a confidence score which is linearly correlated to the segmentation accuracy on test images. △ Less

Submitted 15 January, 2025; originally announced January 2025.

Comments: ISIB 2025

Focal Plane of the Arcus Probe X-Ray Spectrograph

Authors: Catherine E. Grant, Marshall W. Bautz, Eric D. Miller, Richard F. Foster, Beverly LaMarr, Andrew Malonis, Gregory Prigozhin, Benjamin Schneider, Christopher Leitz, Abraham D. Falcone

Abstract: The Arcus Probe mission concept provides high-resolution soft X-ray and UV spectroscopy to reveal feedback-driven structure and evolution throughout the universe with an agile response capability ideal for probing the physics of time-dependent phenomena. The X-ray Spectrograph (XRS) utilizes two nearly identical CCD focal planes to detect and record X-ray photons from the dispersed spectra and zer… ▽ More The Arcus Probe mission concept provides high-resolution soft X-ray and UV spectroscopy to reveal feedback-driven structure and evolution throughout the universe with an agile response capability ideal for probing the physics of time-dependent phenomena. The X-ray Spectrograph (XRS) utilizes two nearly identical CCD focal planes to detect and record X-ray photons from the dispersed spectra and zero-order of the critical angle transmission gratings. In this paper we describe the Arcus focal plane instrument and the CCDs, including laboratory performance results, which meet observatory requirements. △ Less

Submitted 20 December, 2024; originally announced December 2024.

Comments: 11 pages, 7 figures; to be published in JATIS

AGE2HIE: Transfer Learning from Brain Age to Predicting Neurocognitive Outcome for Infant Brain Injury

Authors: Rina Bao, Sheng He, Ellen Grant, Yangming Ou

Abstract: Hypoxic-Ischemic Encephalopathy (HIE) affects 1 to 5 out of every 1,000 newborns, with 30% to 50% of cases resulting in adverse neurocognitive outcomes. However, these outcomes can only be reliably assessed as early as age 2. Therefore, early and accurate prediction of HIE-related neurocognitive outcomes using deep learning models is critical for improving clinical decision-making, guiding treatme… ▽ More Hypoxic-Ischemic Encephalopathy (HIE) affects 1 to 5 out of every 1,000 newborns, with 30% to 50% of cases resulting in adverse neurocognitive outcomes. However, these outcomes can only be reliably assessed as early as age 2. Therefore, early and accurate prediction of HIE-related neurocognitive outcomes using deep learning models is critical for improving clinical decision-making, guiding treatment decisions and assessing novel therapies. However, a major challenge in developing deep learning models for this purpose is the scarcity of large, annotated HIE datasets. We have assembled the first and largest public dataset, however it contains only 156 cases with 2-year neurocognitive outcome labels. In contrast, we have collected 8,859 normal brain black Magnetic Resonance Imagings (MRIs) with 0-97 years of age that are available for brain age estimation using deep learning models. In this paper, we introduce AGE2HIE to transfer knowledge learned by deep learning models from healthy controls brain MRIs to a diseased cohort, from structural to diffusion MRIs, from regression of continuous age estimation to prediction of the binary neurocognitive outcomes, and from lifespan age (0-97 years) to infant (0-2 weeks). Compared to training from scratch, transfer learning from brain age estimation significantly improves not only the prediction accuracy (3% or 2% improvement in same or multi-site), but also the model generalization across different sites (5% improvement in cross-site validation). △ Less

Submitted 7 November, 2024; originally announced November 2024.

Comments: Submitted to ISBI 2025

Foundation AI Model for Medical Image Segmentation

Authors: Rina Bao, Erfan Darzi, Sheng He, Chuan-Heng Hsiao, Mohammad Arafat Hussain, Jingpeng Li, Atle Bjornerud, Ellen Grant, Yangming Ou

Abstract: Foundation models refer to artificial intelligence (AI) models that are trained on massive amounts of data and demonstrate broad generalizability across various tasks with high accuracy. These models offer versatile, one-for-many or one-for-all solutions, eliminating the need for developing task-specific AI models. Examples of such foundation models include the Chat Generative Pre-trained Transfor… ▽ More Foundation models refer to artificial intelligence (AI) models that are trained on massive amounts of data and demonstrate broad generalizability across various tasks with high accuracy. These models offer versatile, one-for-many or one-for-all solutions, eliminating the need for developing task-specific AI models. Examples of such foundation models include the Chat Generative Pre-trained Transformer (ChatGPT) and the Segment Anything Model (SAM). These models have been trained on millions to billions of samples and have shown wide-ranging and accurate applications in numerous tasks such as text processing (using ChatGPT) and natural image segmentation (using SAM). In medical image segmentation - finding target regions in medical images - there is a growing need for these one-for-many or one-for-all foundation models. Such models could obviate the need to develop thousands of task-specific AI models, which is currently standard practice in the field. They can also be adapted to tasks with datasets too small for effective training. We discuss two paths to achieve foundation models for medical image segmentation and comment on progress, challenges, and opportunities. One path is to adapt or fine-tune existing models, originally developed for natural images, for use with medical images. The second path entails building models from scratch, exclusively training on medical images. △ Less

Submitted 4 November, 2024; originally announced November 2024.

Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis

Authors: Neel Dey, Benjamin Billot, Hallee E. Wong, Clinton J. Wang, Mengwei Ren, P. Ellen Grant, Adrian V. Dalca, Polina Golland

Abstract: Current volumetric biomedical foundation models struggle to generalize as public 3D datasets are small and do not cover the broad diversity of medical procedures, conditions, anatomical regions, and imaging protocols. We address this by creating a representation learning method that instead anticipates strong domain shifts at training time itself. We first propose a data engine that synthesizes hi… ▽ More Current volumetric biomedical foundation models struggle to generalize as public 3D datasets are small and do not cover the broad diversity of medical procedures, conditions, anatomical regions, and imaging protocols. We address this by creating a representation learning method that instead anticipates strong domain shifts at training time itself. We first propose a data engine that synthesizes highly variable training samples that would enable generalization to new biomedical contexts. To then train a single 3D network for any voxel-level task, we develop a contrastive learning method that pretrains the network to be stable against nuisance imaging variation simulated by the data engine, a key inductive bias for generalization. This network's features can be used as robust representations of input images for downstream tasks and its weights provide a strong, dataset-agnostic initialization for finetuning on new datasets. As a result, we set new standards across both multimodality registration and few-shot segmentation, a first for any 3D biomedical vision model, all without (pre-)training on any existing dataset of real images. △ Less

Submitted 2 March, 2025; v1 submitted 4 November, 2024; originally announced November 2024.

Comments: ICLR 2025: International Conference on Learning Representations. Code and model weights available at https://github.com/neel-dey/anatomix. Keywords: synthetic data, representation learning, medical image analysis, image registration, image segmentation

Identifiability of Polynomial Models from First Principles and via a Gröbner Basis Approach

Authors: Janet D. Godolphin, James D. E. Grant

Abstract: The relationship between a set of design points and the class of hierarchical polynomial models identifiable from the design is investigated. Saturated models are of particular interest. Necessary and sufficient conditions are derived on the set of design points for specific terms to be included in leaves of the statistical fan. A practitioner led approach to building hierarchical saturated models… ▽ More The relationship between a set of design points and the class of hierarchical polynomial models identifiable from the design is investigated. Saturated models are of particular interest. Necessary and sufficient conditions are derived on the set of design points for specific terms to be included in leaves of the statistical fan. A practitioner led approach to building hierarchical saturated models that are identifiable is developed. This approach is compared to the method of model building based on Gröbner bases. The main results are illustrated by examples. △ Less

Submitted 11 September, 2024; originally announced September 2024.

Comments: 15 pages, no figures

MSC Class: 62K99; 62B15; 62R01; 05B30

Electronic State Population Dynamics upon Ultrafast Strong Field Ionization and Fragmentation of Molecular Nitrogen

Authors: Carlo Kleine, Marc-Oliver Winghart, Zhuang-Yan Zhang, Maria Richter, Maria Ekimova, Sebastian Eckert, Marc J. J. Vrakking, Erik T. J. Nibbering, Arnaud Rouzee, Edward R. Grant

Abstract: Air-lasing from single ionized N$_2^+$ molecules induced by laser filamentation in air has been intensively investigated and the mechanisms responsible for lasing are currently highly debated. We use ultrafast nitrogen K-edge spectroscopy to follow the strong field ionization and fragmentation dynamics of N$_2$ upon interaction with an ultrashort 800 nm laser pulse. Using probe pulses generated by… ▽ More Air-lasing from single ionized N$_2^+$ molecules induced by laser filamentation in air has been intensively investigated and the mechanisms responsible for lasing are currently highly debated. We use ultrafast nitrogen K-edge spectroscopy to follow the strong field ionization and fragmentation dynamics of N$_2$ upon interaction with an ultrashort 800 nm laser pulse. Using probe pulses generated by extreme high-order harmonic generation, we observe transitions indicative of the formation of the electronic ground X$^2Σ_{g}^{+}$, first excited A$^2Π_u$ and second excited B$^2Σ^+_u$ states of N$_2^+$ on femtosecond time scales, from which we can quantitatively determine the time-dependent electronic state population distribution dynamics of N$_2^+$. Our results show a remarkably low population of the A$^2Π_u$ state, and nearly equal populations of the X$^2Σ_{g}^{+}$ and B$^2Σ^+_u$ states. In addition, we observe fragmentation of N$_2^+$ into N and N$^+$ on a time scale of several tens of picoseconds that we assign to significant collisional dynamics in the plasma, resulting in dissociative excitation of N$_2^+$. △ Less

Submitted 10 September, 2024; originally announced September 2024.

Journal ref: Phys. Rev. Lett. 129 (2022) 123002

X-ray spectral performance of the Sony IMX290 CMOS sensor near Fano limit after a per-pixel gain calibration

Authors: Benjamin Schneider, Gregory Prigozhin, Richard F. Foster, Marshall W. Bautz, Hope Fu, Catherine E. Grant, Sarah Heine, Jill Juneau, Beverly LaMarr, Olivier Limousin, Nathan Lourie, Andrew Malonis, Eric D. Miller

Abstract: The advent of back-illuminated complementary metal-oxide-semiconductor (CMOS) sensors and their well-known advantages over charge-coupled devices (CCDs) make them an attractive technology for future X-ray missions. However, numerous challenges remain, including improving their depletion depth and identifying effective methods to calculate per-pixel gain conversion. We have tested a commercial Sony… ▽ More The advent of back-illuminated complementary metal-oxide-semiconductor (CMOS) sensors and their well-known advantages over charge-coupled devices (CCDs) make them an attractive technology for future X-ray missions. However, numerous challenges remain, including improving their depletion depth and identifying effective methods to calculate per-pixel gain conversion. We have tested a commercial Sony IMX290LLR CMOS sensor under X-ray light using an $^{55}$Fe radioactive source and collected X-ray photons for $\sim$15 consecutive days under stable conditions at regulated temperatures of 21°C and 26°C. At each temperature, the data set contained enough X-ray photons to produce one spectrum per pixel consisting only of single-pixel events. We determined the gain dispersion of its 2.1 million pixels using the peak fitting and the Energy Calibration by Correlation (ECC) methods. We measured a gain dispersion of 0.4\% at both temperatures and demonstrated the advantage of the ECC method in the case of spectra with low statistics. The energy resolution at 5.9 keV after the per-pixel gain correction is improved by $\gtrsim$10 eV for single-pixel and all event spectra, with single-pixel event energy resolution reaching $123.6\pm 0.2$ eV, close to the Fano limit of silicon sensors at room temperature. Finally, our long data acquisition demonstrated the excellent stability of the detector over more than 30 days under a flux of $10^4$ photons per second. △ Less

Submitted 9 September, 2024; originally announced September 2024.

Comments: 15 pages, 7 figures, accepted for publication in JATIS

Unveiling the Cosmic Chemistry: Revisiting the Mass-Metallicity Relation with JWST/NIRSpec at 4 < z < 10

Authors: Arnab Sarkar, Priyanka Chakraborty, Mark Vogelsberger, Michael McDonald, Paul Torrey, Alex M. Garcia, Gourav Khullar, Gary J. Ferland, William Forman, Scott Wolk, Benjamin Schneider, Mark Bautz, Eric Miller, Catherine Grant, John ZuHone

Abstract: We present star formation rates (SFR), the mass-metallicity relation (MZR), and the SFR-dependent MZR across redshifts 4 to 10 using 81 star-forming galaxies observed by the JWST NIRSpec employing both low-resolution PRISM and medium-resolution gratings, including galaxies from the JADES GOODS-N and GOODS-S fields, the JWST-PRIMAL Legacy Survey, and additional galaxies from the literature in Abell… ▽ More We present star formation rates (SFR), the mass-metallicity relation (MZR), and the SFR-dependent MZR across redshifts 4 to 10 using 81 star-forming galaxies observed by the JWST NIRSpec employing both low-resolution PRISM and medium-resolution gratings, including galaxies from the JADES GOODS-N and GOODS-S fields, the JWST-PRIMAL Legacy Survey, and additional galaxies from the literature in Abell 2744, SMACS-0723, RXJ2129, BDF, COSMOS, and MACS1149 fields. These galaxies span a 3 dex stellar mass range of $10^7 < M_{\ast}/M_{\odot} < 10^{10}$, with an average SFR of $7.2 \pm 1.2 M_{\odot} {\rm yr}^{-1}$ and an average metallicity of $12+{\rm log(O/H)} = 7.91 \pm 0.08$. Our findings align with previous observations up to $z=8$ for the MZR and indicate no deviation from local universe FMR up to this redshift. Beyond $z=8$, we observe a significant deviation $\sim 0.27$ dex) in FMR, consistent with recent JWST findings. We also integrate CEERS (135 galaxies) and JADES (47 galaxies) samples with our data to study metallicity evolution with redshift in a combined sample of 263 galaxies, revealing a decreasing metallicity trend with a slope of $0.067 \pm 0.013$, consistent with IllustrisTNG and EAGLE, but contradicts with FIRE simulations. We introduce an empirical mass-metallicity-redshift (MZ-$z$ relation): $12+{\rm log(O/H)}=6.29 + 0.237 \times{\rm log}(M_{\ast}/M_{\odot}) - 0.06 \times (1+z)$, which accurately reproduces the observed trends in metallicity with both redshift and stellar mass. This trend underscores the ``Grand Challenge'' in understanding the factors driving high-redshift galactic metallicity trends, such as inflow, outflow, and AGN/stellar feedback -- and emphasizes the need for further investigations with larger samples and enhanced simulations. △ Less

Submitted 13 December, 2024; v1 submitted 15 August, 2024; originally announced August 2024.

Comments: 24 Pages, 9 Figures. Accepted for publication in the Astrophysical Journal

Towards efficient machine-learning-based reduction of the cosmic-ray induced background in X-ray imaging detectors: increasing context awareness

Authors: Artem Poliszczuk, Dan Wilkins, Steven W. Allen, Eric D. Miller, Tanmoy Chattopadhyay, Benjamin Schneider, Julien Eric Darve, Marshall Bautz, Abe Falcone, Richard Foster, Catherine E. Grant, Sven Herrmann, Ralph Kraft, R. Glenn Morris, Paul Nulsen, Peter Orel, Gerrit Schellenberger, Haley R. Stueber

Abstract: Traditional cosmic ray filtering algorithms used in X-ray imaging detectors aboard space telescopes perform event reconstruction based on the properties of activated pixels above a certain energy threshold, within 3x3 or 5x5 pixel sliding windows. This approach can reject up to 98% of the cosmic ray background. However, the remaining unrejected background constitutes a significant impediment to st… ▽ More Traditional cosmic ray filtering algorithms used in X-ray imaging detectors aboard space telescopes perform event reconstruction based on the properties of activated pixels above a certain energy threshold, within 3x3 or 5x5 pixel sliding windows. This approach can reject up to 98% of the cosmic ray background. However, the remaining unrejected background constitutes a significant impediment to studies of low surface brightness objects, which are especially prevalent in the high-redshift universe. The main limitation of the traditional filtering algorithms is their ignorance of the long-range contextual information present in image frames. This becomes particularly problematic when analyzing signals created by secondary particles produced during interactions of cosmic rays with body of the detector. Such signals may look identical to the energy deposition left by X-ray photons, when one considers only the properties within the small sliding window. Additional information is present, however, in the spatial and energy correlations between signals in different parts of the frame, which can be accessed by modern machine learning (ML) techniques. In this work, we continue the development of an ML-based pipeline for cosmic ray background mitigation. Our latest method consist of two stages: first, a frame classification neural network is used to create class activation maps (CAM), localizing all events within the frame; second, after event reconstruction, a random forest classifier, using features obtained from CAMs, is used to separate X-ray and cosmic ray features. The method delivers >40% relative improvement over traditional filtering in background rejection in standard 0.3-10keV energy range, at the expense of only a small (<2%) level of lost X-ray signal. Our method also provides a convenient way to tune the cosmic ray rejection threshold to adapt to a user's specific scientific needs. △ Less

Submitted 23 July, 2024; originally announced July 2024.

Comments: To appear in SPIE Astronomical Telescopes + Instrumentation proceedings 2024

Augmenting astronomical X-ray detectors with AI for enhanced sensitivity and reduced background

Authors: D. R. Wilkins, A. Poliszczuk, B. Schneider, E. D. Miller, S. W. Allen, M. Bautz, T. Chattopadhyay, A. D. Falcone, R. Foster, C. E. Grant, S. Herrmann, R. Kraft, R. G. Morris, P. Nulsen, P. Orel, G. Schellenberger

Abstract: Bringing artificial intelligence (AI) alongside next-generation X-ray imaging detectors, including CCDs and DEPFET sensors, enhances their sensitivity to achieve many of the flagship science cases targeted by future X-ray observatories, based upon low surface brightness and high redshift sources. Machine learning algorithms operating on the raw frame-level data provide enhanced identification of b… ▽ More Bringing artificial intelligence (AI) alongside next-generation X-ray imaging detectors, including CCDs and DEPFET sensors, enhances their sensitivity to achieve many of the flagship science cases targeted by future X-ray observatories, based upon low surface brightness and high redshift sources. Machine learning algorithms operating on the raw frame-level data provide enhanced identification of background vs. astrophysical X-ray events, by considering all of the signals in the context within which they appear within each frame. We have developed prototype machine learning algorithms to identify valid X-ray and cosmic-ray induced background events, trained and tested upon a suite of realistic end-to-end simulations that trace the interaction of cosmic ray particles and their secondaries through the spacecraft and detector. These algorithms demonstrate that AI can reduce the unrejected instrumental background by up to 41.5 per cent compared with traditional filtering methods. Alongside AI algorithms to reduce the instrumental background, next-generation event reconstruction methods, based upon fitting physically-motivated Gaussian models of the charge clouds produced by events within the detector, promise increased accuracy and spectral resolution of the lowest energy photon events. △ Less

Submitted 23 July, 2024; originally announced July 2024.

Comments: Proceedings of the SPIE, Astronomical Telescopes and Instrumentation, Space Telescopes and Instrumentation 2024: Ultraviolet to Gamma Ray

Journal ref: Proc. SPIE, 2024, 13093-65

International Astrophysical Consortium for High-energy Calibration: Summary of the 15th IACHEC Workshop

Authors: K. K. Madsen, V. Burwitz, K. Forster, C. E. Grant, M. Guainazzi, V. Kashyap, H. L. Marshall, E. D. Miller, L. Natalucci, P. P. Plucinsky, Y. Terada

Abstract: In this report, we summarize the activities of the International Astronomical Consortium for High Energy Calibration (IACHEC) from the 15th IACHEC Workshop in Pelham, Germany. Sixty scientists directly involved in the calibration of operational and future high-energy missions gathered for 3.5 days to discuss the status of the cross-calibration between the current international complement of X-ray… ▽ More In this report, we summarize the activities of the International Astronomical Consortium for High Energy Calibration (IACHEC) from the 15th IACHEC Workshop in Pelham, Germany. Sixty scientists directly involved in the calibration of operational and future high-energy missions gathered for 3.5 days to discuss the status of the cross-calibration between the current international complement of X-ray observatories and the possibilities to improve it. This summary consists of reports from the Working Groups with topics ranging across the identification and characterization of standard calibration sources, multi-observatory cross-calibration campaigns, appropriate and new statistical techniques, calibration of instruments and characterization of background, preservation of knowledge, and results for the benefit of the astronomical community. △ Less

Submitted 12 July, 2024; originally announced July 2024.

Comments: 10 pages, 1 figure. arXiv admin note: text overlap with arXiv:2111.01613

Curved detectors for future X-ray astrophysics missions

Authors: Eric D. Miller, James A. Gregory, Marshall W. Bautz, Harry R. Clark, Michael Cooper, Kevan Donlon, Richard F. Foster, Catherine E. Grant, Mallory Jensen, Beverly LaMarr, Renee Lambert, Christopher Leitz, Andrew Malonis, Mo Neak, Gregory Prigozhin, Kevin Ryu, Benjamin Schneider, Keith Warner, Douglas J. Young, William W. Zhang

Abstract: Future X-ray astrophysics missions will survey large areas of the sky with unparalleled sensitivity, enabled by lightweight, high-resolution optics. These optics inherently produce curved focal surfaces with radii as small as 2 m, requiring a large area detector system that closely conforms to the curved focal surface. We have embarked on a project using a curved charge-coupled device (CCD) detect… ▽ More Future X-ray astrophysics missions will survey large areas of the sky with unparalleled sensitivity, enabled by lightweight, high-resolution optics. These optics inherently produce curved focal surfaces with radii as small as 2 m, requiring a large area detector system that closely conforms to the curved focal surface. We have embarked on a project using a curved charge-coupled device (CCD) detector technology developed at MIT Lincoln Laboratory to provide large-format, curved detectors for such missions, improving performance and simplifying design. We present the current status of this work, which aims to curve back-illuminated, large-format (5 cm x 4 cm) CCDs to 2.5-m radius and confirm X-ray performance. We detail the design of fixtures and the curving process, and present intial results on curving bare silicon samples and monitor devices and characterizing the surface geometric accuracy. The tests meet our accuracy requirement of <5 $μ$m RMS surface non-conformance for samples of similar thickness to the functional detectors. We finally show X-ray performance measurements of planar CCDs that will serve as a baseline to evaluate the curved detectors. The detectors exhibit low noise, good charge-transfer efficiency, and excellent, uniform spectroscopic performance, including in the important soft X-ray band. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: 18 pages, 13 figures, submitted to the Proceedings of SPIE, Astronomical Telescopes + Instrumentation 2024

The Advanced CCD Imaging Spectrometer on the Chandra X-ray Observatory: twenty-five years of on-orbit operation

Authors: Catherine E. Grant, Marshall W. Bautz, Paul P. Plucinsky, Peter G. Ford

Abstract: As the Advanced CCD Imaging Spectrometer (ACIS) on the Chandra X-ray Observatory completes a quarter century of on orbit operations, it continues to perform well and produce spectacular scientific results. The response of ACIS has evolved over the lifetime of the observatory due to radiation damage, molecular contamination, changing particle environment, and aging of the spacecraft in general. We… ▽ More As the Advanced CCD Imaging Spectrometer (ACIS) on the Chandra X-ray Observatory completes a quarter century of on orbit operations, it continues to perform well and produce spectacular scientific results. The response of ACIS has evolved over the lifetime of the observatory due to radiation damage, molecular contamination, changing particle environment, and aging of the spacecraft in general. We present highlights from the instrument team's monitoring program and our expectations for the future of ACIS. Performance changes on ACIS continue to be manageable, and do not indicate any limitations on ACIS lifetime. We examine aspects of the design and operation of ACIS that have impacted its long lifetime with lessons learned for future instruments. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: 15 pages, 10 figures, appearing in the proceedings of SPIE Astronomical Telescopes and Instrumentation 2024, Space Telescopes and Instrumentation 2024: Ultraviolet to Gamma Ray, paper 13093-50

Dynamical control in a prethermalized molecular ultracold plasma: Local dissipation drives global relaxation

Authors: Ruoxi Wang, Amin Allahverdian, Smilla Colombini, Nathan Durand-Brousseau, Kevin Marroquın, James Keller, John Sous, Abhinav Prem, Edward Grant

Abstract: Prethermalization occurs as an important phase in the dynamics of many-body systems when strong coupling drives a quasi-equilibrium in a subspace separated from the thermodynamic equilibrium by the restriction of a gap in energy or other conserved quantity. Here, we report the signature of an enduring prethermal regime of arrested relaxation in the molecular ultracold plasma that forms following t… ▽ More Prethermalization occurs as an important phase in the dynamics of many-body systems when strong coupling drives a quasi-equilibrium in a subspace separated from the thermodynamic equilibrium by the restriction of a gap in energy or other conserved quantity. Here, we report the signature of an enduring prethermal regime of arrested relaxation in the molecular ultracold plasma that forms following the avalanche of a state-selected Rydberg gas of nitric oxide. Electron collisions mix orbital angular momentum, scattering Rydberg molecules to states of very high-$\ell$. Spontaneous predissociation purifies this non-penetrating character, creating an extraordinary gap between the plasma states of $n \approx \ell$, with measured $n>200$ and penetrating states of $\ell = 0, ~1$ and 2. Evolution to a statistically equilibrated state of N and O atoms cannot occur without Rydberg electron penetration, and this gap blocks relaxation for a millisecond or more. Evolving through the critical phase, electrons that balance the NO$^+$ charge behave as though localized in the prethermal phase and play an ineffective role in bridging this gap. However, the application of a weak radiofrequency (RF) field promotes a dramatic degree of relaxation owing to electron collisions. On an entirely different scale, exciting a quantum-state transition in an exceedingly small fraction of the molecules in the prethermalized ensemble acts with even greater effect to drive the entire system toward equilibrium. We ascribe this to dissipative character added to a small fraction of the states in the prethermally localized ensemble. Using the Lindblad master equation, we illustrate qualitatively similar dynamics for a toy model of an open quantum system that consists of a localized set of spins on which dissipation acts locally at a single site. △ Less

Submitted 6 July, 2024; v1 submitted 12 June, 2024; originally announced June 2024.

Advancing Precision Particle Background Estimation for Future X-ray Missions: Correlated Variability between AMS and Chandra/XMM-Newton

Authors: Arnab Sarkar, Catherine E. Grant, Eric D. Miller, Mark Bautz, Benjamin Schneider, Rick F. Foster, Gerrit Schellenberger, Steven Allen, Ralph P. Kraft, Dan Wilkins, Abe Falcone, Andrew Ptak

Abstract: Galactic cosmic ray (GCR) particles have a significant impact on the particle-induced background of X-ray observatories, and their flux exhibits substantial temporal variability, potentially influencing background levels. In this study, we present one-day binned high-energy reject rates derived from the Chandra-ACIS and XMM-Newton EPIC-pn instruments, serving as proxies for GCR particle flux. We s… ▽ More Galactic cosmic ray (GCR) particles have a significant impact on the particle-induced background of X-ray observatories, and their flux exhibits substantial temporal variability, potentially influencing background levels. In this study, we present one-day binned high-energy reject rates derived from the Chandra-ACIS and XMM-Newton EPIC-pn instruments, serving as proxies for GCR particle flux. We systematically analyze the ACIS and EPIC-pn reject rates and compare them with the AMS proton flux. Our analysis initially reveals robust correlations between the AMS proton flux and the ACIS/EPIC-pn reject rates when binned over 27-day intervals. However, a closer examination reveals substantial fluctuations within each 27-day bin, indicating shorter-term variability. Upon daily binning, we observe finer. temporal structures in the datasets, demonstrating the presence of recurrent variations with periods of $\sim$ 25 days and 23 days in ACIS and EPIC-pn reject rates, respectively, spanning the years 2014 to 2018. Notably, during the 2016--2017 period, we additionally detect periodicities of $\sim$13.5 days and 9 days in the ACIS and EPIC-pn reject rates, respectively. Intriguingly, we observe a time lag of $\sim$ 6 days between the AMS proton flux and the ACIS/EPIC-pn reject rates during the second half of 2016. This time lag is not visible before 2016 and aftern2017. The underlying physical mechanisms responsible for this time lag remain a subject of ongoing investigation. △ Less

Submitted 10 May, 2024; originally announced May 2024.

Comments: 16 pages, 8 figures, accepted for publication in ApJ

FetalDiffusion: Pose-Controllable 3D Fetal MRI Synthesis with Conditional Diffusion Model

Authors: Molin Zhang, Polina Golland, Patricia Ellen Grant, Elfar Adalsteinsson

Abstract: The quality of fetal MRI is significantly affected by unpredictable and substantial fetal motion, leading to the introduction of artifacts even when fast acquisition sequences are employed. The development of 3D real-time fetal pose estimation approaches on volumetric EPI fetal MRI opens up a promising avenue for fetal motion monitoring and prediction. Challenges arise in fetal pose estimation due… ▽ More The quality of fetal MRI is significantly affected by unpredictable and substantial fetal motion, leading to the introduction of artifacts even when fast acquisition sequences are employed. The development of 3D real-time fetal pose estimation approaches on volumetric EPI fetal MRI opens up a promising avenue for fetal motion monitoring and prediction. Challenges arise in fetal pose estimation due to limited number of real scanned fetal MR training images, hindering model generalization when the acquired fetal MRI lacks adequate pose. In this study, we introduce FetalDiffusion, a novel approach utilizing a conditional diffusion model to generate 3D synthetic fetal MRI with controllable pose. Additionally, an auxiliary pose-level loss is adopted to enhance model performance. Our work demonstrates the success of this proposed model by producing high-quality synthetic fetal MRI images with accurate and recognizable fetal poses, comparing favorably with in-vivo real fetal MRI. Furthermore, we show that the integration of synthetic fetal MR images enhances the fetal pose estimation model's performance, particularly when the number of available real scanned data is limited resulting in 15.4% increase in PCK and 50.2% reduced in mean error. All experiments are done on a single 32GB V100 GPU. Our method holds promise for improving real-time tracking models, thereby addressing fetal motion issues more effectively. △ Less

Submitted 29 March, 2024; originally announced April 2024.

Comments: 8 pages, 3 figures, 2 tables, submitted to MICCAI 2024, code available if accepted

On the Particle Acceleration Mechanisms in a Double Radio Relic Galaxy Cluster, Abell 1240

Authors: Arnab Sarkar, Felipe Andrade-Santos, Reinout J. van Weeren, Ralph P. Kraft, Duy N. Hoang, Timothy W. Shimwell, Paul Nulsen, William Forman, Scott Randall, Yuanyuan Su, Priyanka Chakraborty, Christine Jones, Eric Miller, Mark Bautz, Catherine E. Grant

Abstract: We present a 368 ks deep Chandra observation of Abell~1240, a binary merging galaxy cluster at a redshift of 0.195 with two Brightest Cluster Galaxies (BCGs) may have passed each other 0.3 Gyr ago. Building upon previous investigations involving GMRT, VLA, and LOFAR data, our study focuses on two prominent extended radio relics at the north-west (NW) and south-east (SE) of the cluster core. By lev… ▽ More We present a 368 ks deep Chandra observation of Abell~1240, a binary merging galaxy cluster at a redshift of 0.195 with two Brightest Cluster Galaxies (BCGs) may have passed each other 0.3 Gyr ago. Building upon previous investigations involving GMRT, VLA, and LOFAR data, our study focuses on two prominent extended radio relics at the north-west (NW) and south-east (SE) of the cluster core. By leveraging the high-resolution Chandra imaging, we have identified two distinct surface brightness edges at $\sim$ 1 Mpc and 1.2 Mpc NW and SE of the cluster center, respectively, coinciding with the outer edges of both relics. Our temperature measurements hint the edges to be shock front edges. The Mach numbers, derived from the gas density jumps, yield $\cal{M}_{\rm SE}$ = 1.49$^{+0.22}_{-0.24}$ for the South Eastern shock and $\cal{M}_{\rm NW}$ = 1.41$^{+0.17}_{-0.19}$ for the North Western shock. Our estimated Mach numbers are remarkably smaller compared to those derived from radio observations ($\cal{M}_{\rm SE}$ = 2.3 and $\cal{M}_{\rm NW}$ = 2.4), highlighting the prevalence of a re-acceleration scenario over direct acceleration of electrons from the thermal pool. Furthermore, we compare the observed temperature profiles across both shocks with that of predictions from collisional vs. collisionless models. Both shocks favor the Coulomb collisional model, but we could not rule out a purely collisionless model due to pre-shock temperature uncertainties. △ Less

Submitted 12 January, 2024; v1 submitted 3 January, 2024; originally announced January 2024.

Comments: Accepted for publication in The Astrophysical Journal

SE(3)-Equivariant and Noise-Invariant 3D Rigid Motion Tracking in Brain MRI

Authors: Benjamin Billot, Neel Dey, Daniel Moyer, Malte Hoffmann, Esra Abaci Turk, Borjan Gagoski, Ellen Grant, Polina Golland

Abstract: Rigid motion tracking is paramount in many medical imaging applications where movements need to be detected, corrected, or accounted for. Modern strategies rely on convolutional neural networks (CNN) and pose this problem as rigid registration. Yet, CNNs do not exploit natural symmetries in this task, as they are equivariant to translations (their outputs shift with their inputs) but not to rotati… ▽ More Rigid motion tracking is paramount in many medical imaging applications where movements need to be detected, corrected, or accounted for. Modern strategies rely on convolutional neural networks (CNN) and pose this problem as rigid registration. Yet, CNNs do not exploit natural symmetries in this task, as they are equivariant to translations (their outputs shift with their inputs) but not to rotations. Here we propose EquiTrack, the first method that uses recent steerable SE(3)-equivariant CNNs (E-CNN) for motion tracking. While steerable E-CNNs can extract corresponding features across different poses, testing them on noisy medical images reveals that they do not have enough learning capacity to learn noise invariance. Thus, we introduce a hybrid architecture that pairs a denoiser with an E-CNN to decouple the processing of anatomically irrelevant intensity features from the extraction of equivariant spatial features. Rigid transforms are then estimated in closed-form. EquiTrack outperforms state-of-the-art learning and optimisation methods for motion tracking in adult brain MRI and fetal MRI time series. Our code is available at https://github.com/BBillot/EquiTrack. △ Less

Submitted 12 June, 2024; v1 submitted 20 December, 2023; originally announced December 2023.

Comments: Published at IEEE transactions on Medical Imaging

Shape-aware Segmentation of the Placenta in BOLD Fetal MRI Time Series

Authors: S. Mazdak Abulnaga, Neel Dey, Sean I. Young, Eileen Pan, Katherine I. Hobgood, Clinton J. Wang, P. Ellen Grant, Esra Abaci Turk, Polina Golland

Abstract: Blood oxygen level dependent (BOLD) MRI time series with maternal hyperoxia can assess placental oxygenation and function. Measuring precise BOLD changes in the placenta requires accurate temporal placental segmentation and is confounded by fetal and maternal motion, contractions, and hyperoxia-induced intensity changes. Current BOLD placenta segmentation methods warp a manually annotated subject-… ▽ More Blood oxygen level dependent (BOLD) MRI time series with maternal hyperoxia can assess placental oxygenation and function. Measuring precise BOLD changes in the placenta requires accurate temporal placental segmentation and is confounded by fetal and maternal motion, contractions, and hyperoxia-induced intensity changes. Current BOLD placenta segmentation methods warp a manually annotated subject-specific template to the entire time series. However, as the placenta is a thin, elongated, and highly non-rigid organ subject to large deformations and obfuscated edges, existing work cannot accurately segment the placental shape, especially near boundaries. In this work, we propose a machine learning segmentation framework for placental BOLD MRI and apply it to segmenting each volume in a time series. We use a placental-boundary weighted loss formulation and perform a comprehensive evaluation across several popular segmentation objectives. Our model is trained and tested on a cohort of 91 subjects containing healthy fetuses, fetuses with fetal growth restriction, and mothers with high BMI. Biomedically, our model performs reliably in segmenting volumes in both normoxic and hyperoxic points in the BOLD time series. We further find that boundary-weighting increases placental segmentation performance by 8.3% and 6.0% Dice coefficient for the cross-entropy and signed distance transform objectives, respectively. Our code and trained model is available at https://github.com/mabulnaga/automatic-placenta-segmentation. △ Less

Submitted 8 December, 2023; originally announced December 2023.

Comments: Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2023:017. arXiv admin note: substantial text overlap with arXiv:2208.02895

Journal ref: Machine.Learning.for.Biomedical.Imaging. 2 (2023)

Bayes in the age of intelligent machines

Authors: Thomas L. Griffiths, Jian-Qiao Zhu, Erin Grant, R. Thomas McCoy

Abstract: The success of methods based on artificial neural networks in creating intelligent machines seems like it might pose a challenge to explanations of human cognition in terms of Bayesian inference. We argue that this is not the case, and that in fact these systems offer new opportunities for Bayesian modeling. Specifically, we argue that Bayesian models of cognition and artificial neural networks li… ▽ More The success of methods based on artificial neural networks in creating intelligent machines seems like it might pose a challenge to explanations of human cognition in terms of Bayesian inference. We argue that this is not the case, and that in fact these systems offer new opportunities for Bayesian modeling. Specifically, we argue that Bayesian models of cognition and artificial neural networks lie at different levels of analysis and are complementary modeling approaches, together offering a way to understand human cognition that spans these levels. We also argue that the same perspective can be applied to intelligent machines, where a Bayesian approach may be uniquely valuable in understanding the behavior of large, opaque artificial neural networks that are trained on proprietary data. △ Less

Submitted 16 November, 2023; originally announced November 2023.

The Transient Nature of Emergent In-Context Learning in Transformers

Authors: Aaditya K. Singh, Stephanie C. Y. Chan, Ted Moskovitz, Erin Grant, Andrew M. Saxe, Felix Hill

Abstract: Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g. through the lens of mechanistic interpretability, Bayesian inference, or by examining the distributional properties of training data. However, in each of these cases, ICL is t… ▽ More Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g. through the lens of mechanistic interpretability, Bayesian inference, or by examining the distributional properties of training data. However, in each of these cases, ICL is treated largely as a persistent phenomenon; namely, once ICL emerges, it is assumed to persist asymptotically. Here, we show that the emergence of ICL during transformer training is, in fact, often transient. We train transformers on synthetic data designed so that both ICL and in-weights learning (IWL) strategies can lead to correct predictions. We find that ICL first emerges, then disappears and gives way to IWL, all while the training loss decreases, indicating an asymptotic preference for IWL. The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to "overtrain" transformers when seeking compact, cheaper-to-run models. We find that L2 regularization may offer a path to more persistent ICL that removes the need for early stopping based on ICL-style validation tasks. Finally, we present initial evidence that ICL transience may be caused by competition between ICL and IWL circuits. △ Less

Submitted 11 December, 2023; v1 submitted 14 November, 2023; originally announced November 2023.

Comments: 19 pages, 16 figures

Dynamic Neural Fields for Learning Atlases of 4D Fetal MRI Time-series

Authors: Zeen Chi, Zhongxiao Cong, Clinton J. Wang, Yingcheng Liu, Esra Abaci Turk, P. Ellen Grant, S. Mazdak Abulnaga, Polina Golland, Neel Dey

Abstract: We present a method for fast biomedical image atlas construction using neural fields. Atlases are key to biomedical image analysis tasks, yet conventional and deep network estimation methods remain time-intensive. In this preliminary work, we frame subject-specific atlas building as learning a neural field of deformable spatiotemporal observations. We apply our method to learning subject-specific… ▽ More We present a method for fast biomedical image atlas construction using neural fields. Atlases are key to biomedical image analysis tasks, yet conventional and deep network estimation methods remain time-intensive. In this preliminary work, we frame subject-specific atlas building as learning a neural field of deformable spatiotemporal observations. We apply our method to learning subject-specific atlases and motion stabilization of dynamic BOLD MRI time-series of fetuses in utero. Our method yields high-quality atlases of fetal BOLD time-series with $\sim$5-7$\times$ faster convergence compared to existing work. While our method slightly underperforms well-tuned baselines in terms of anatomical overlap, it estimates templates significantly faster, thus enabling rapid processing and stabilization of large databases of 4D dynamic MRI acquisitions. Code is available at https://github.com/Kidrauh/neural-atlasing △ Less

Submitted 6 November, 2023; originally announced November 2023.

Comments: 6 pages, 2 figures. Accepted by Medical Imaging Meets NeurIPS 2023

Overview of the Advanced X-ray Imaging Satellite (AXIS)

Authors: Christopher S. Reynolds, Erin A. Kara, Richard F. Mushotzky, Andrew Ptak, Michael J. Koss, Brian J. Williams, Steven W. Allen, Franz E. Bauer, Marshall Bautz, Arash Bodaghee, Kevin B. Burdge, Nico Cappelluti, Brad Cenko, George Chartas, Kai-Wing Chan, Lía Corrales, Tansu Daylan, Abraham D. Falcone, Adi Foord, Catherine E. Grant, Mélanie Habouzit, Daryl Haggard, Sven Herrmann, Edmund Hodges-Kluck, Oleg Kargaltsev , et al. (18 additional authors not shown)

Abstract: The Advanced X-ray Imaging Satellite (AXIS) is a Probe-class concept that will build on the legacy of the Chandra X-ray Observatory by providing low-background, arcsecond-resolution imaging in the 0.3-10 keV band across a 450 arcminute$^2$ field of view, with an order of magnitude improvement in sensitivity. AXIS utilizes breakthroughs in the construction of lightweight segmented X-ray optics usin… ▽ More The Advanced X-ray Imaging Satellite (AXIS) is a Probe-class concept that will build on the legacy of the Chandra X-ray Observatory by providing low-background, arcsecond-resolution imaging in the 0.3-10 keV band across a 450 arcminute$^2$ field of view, with an order of magnitude improvement in sensitivity. AXIS utilizes breakthroughs in the construction of lightweight segmented X-ray optics using single-crystal silicon, and developments in the fabrication of large-format, small-pixel, high readout rate CCD detectors with good spectral resolution, allowing a robust and cost-effective design. Further, AXIS will be responsive to target-of-opportunity alerts and, with onboard transient detection, will be a powerful facility for studying the time-varying X-ray universe, following on from the legacy of the Neil Gehrels (Swift) X-ray observatory that revolutionized studies of the transient X-ray Universe. In this paper, we present an overview of AXIS, highlighting the prime science objectives driving the AXIS concept and how the observatory design will achieve these objectives. △ Less

Submitted 1 November, 2023; originally announced November 2023.

Comments: Published in Proceedings of SPIE Optics & Photonics 2023, San Diego

Getting aligned on representational alignment

Authors: Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Christopher J. Cueva, Erin Grant, Iris Groen, Jascha Achterberg, Joshua B. Tenenbaum, Katherine M. Collins, Katherine L. Hermann, Kerem Oktar, Klaus Greff, Martin N. Hebart, Nathan Cloos, Nikolaus Kriegeskorte, Nori Jacoby, Qiuyi Zhang, Raja Marjieh, Robert Geirhos, Sherol Chen, Simon Kornblith , et al. (8 additional authors not shown)

Abstract: Biological and artificial information processing systems form representations of the world that they can use to categorize, reason, plan, navigate, and make decisions. How can we measure the similarity between the representations formed by these diverse systems? Do similarities in representations then translate into similar behavior? If so, then how can a system's representations be modified to be… ▽ More Biological and artificial information processing systems form representations of the world that they can use to categorize, reason, plan, navigate, and make decisions. How can we measure the similarity between the representations formed by these diverse systems? Do similarities in representations then translate into similar behavior? If so, then how can a system's representations be modified to better match those of another system? These questions pertaining to the study of representational alignment are at the heart of some of the most promising research areas in contemporary cognitive science, neuroscience, and machine learning. In this Perspective, we survey the exciting recent developments in representational alignment research in the fields of cognitive science, neuroscience, and machine learning. Despite their overlapping interests, there is limited knowledge transfer between these fields, so work in one field ends up duplicated in another, and useful innovations are not shared effectively. To improve communication, we propose a unifying framework that can serve as a common language for research on representational alignment, and map several streams of existing work across fields within our framework. We also lay out open problems in representational alignment where progress can benefit all three of these fields. We hope that this paper will catalyze cross-disciplinary collaboration and accelerate progress for all communities studying and developing information processing systems. △ Less

Submitted 26 November, 2024; v1 submitted 18 October, 2023; originally announced October 2023.

Comments: 51 pages; Working paper (changes to be made in upcoming revisions)

Consistency Regularization Improves Placenta Segmentation in Fetal EPI MRI Time Series

Authors: Yingcheng Liu, Neerav Karani, Neel Dey, S. Mazdak Abulnaga, Junshen Xu, P. Ellen Grant, Esra Abaci Turk, Polina Golland

Abstract: The placenta plays a crucial role in fetal development. Automated 3D placenta segmentation from fetal EPI MRI holds promise for advancing prenatal care. This paper proposes an effective semi-supervised learning method for improving placenta segmentation in fetal EPI MRI time series. We employ consistency regularization loss that promotes consistency under spatial transformation of the same image a… ▽ More The placenta plays a crucial role in fetal development. Automated 3D placenta segmentation from fetal EPI MRI holds promise for advancing prenatal care. This paper proposes an effective semi-supervised learning method for improving placenta segmentation in fetal EPI MRI time series. We employ consistency regularization loss that promotes consistency under spatial transformation of the same image and temporal consistency across nearby images in a time series. The experimental results show that the method improves the overall segmentation accuracy and provides better performance for outliers and hard samples. The evaluation also indicates that our method improves the temporal coherency of the prediction, which could lead to more accurate computation of temporal placental biomarkers. This work contributes to the study of the placenta and prenatal clinical decision-making. Code is available at https://github.com/firstmover/cr-seg. △ Less

Submitted 15 October, 2023; v1 submitted 5 October, 2023; originally announced October 2023.

Statistical physics, Bayesian inference and neural information processing

Authors: Erin Grant, Sandra Nestler, Berfin Şimşek, Sara Solla

Abstract: Lecture notes from the course given by Professor Sara A. Solla at the Les Houches summer school on "Statistical physics of Machine Learning". The notes discuss neural information processing through the lens of Statistical Physics. Contents include Bayesian inference and its connection to a Gibbs description of learning and generalization, Generalized Linear Models as a controlled alternative to ba… ▽ More Lecture notes from the course given by Professor Sara A. Solla at the Les Houches summer school on "Statistical physics of Machine Learning". The notes discuss neural information processing through the lens of Statistical Physics. Contents include Bayesian inference and its connection to a Gibbs description of learning and generalization, Generalized Linear Models as a controlled alternative to backpropagation through time, and linear and non-linear techniques for dimensionality reduction. △ Less

Submitted 29 September, 2023; originally announced September 2023.

Comments: These are notes from the lecture of Sara Solla given at the summer school "Statistical Physics & Machine Learning", that took place in Les Houches School of Physics in France from 4th to 29th July 2022. The school was organized by Florent Krzakala and Lenka Zdeborová from EPFL

The high-speed X-ray camera on AXIS

Authors: Eric D. Miller, Marshall W. Bautz, Catherine E. Grant, Richard F. Foster, Beverly LaMarr, Andrew Malonis, Gregory Prigozhin, Benjamin Schneider, Christopher Leitz, Sven Herrmann, Steven W. Allen, Tanmoy Chattopadhyay, Peter Orel, R. Glenn Morris, Haley Stueber, Abraham D. Falcone, Andrew Ptak, Christopher Reynolds

Abstract: AXIS is a Probe-class mission concept that will provide high-throughput, high-spatial-resolution X-ray spectral imaging, enabling transformative studies of high-energy astrophysical phenomena. To take advantage of the advanced optics and avoid photon pile-up, the AXIS focal plane requires detectors with readout rates at least 20 times faster than previous soft X-ray imaging spectrometers flying ab… ▽ More AXIS is a Probe-class mission concept that will provide high-throughput, high-spatial-resolution X-ray spectral imaging, enabling transformative studies of high-energy astrophysical phenomena. To take advantage of the advanced optics and avoid photon pile-up, the AXIS focal plane requires detectors with readout rates at least 20 times faster than previous soft X-ray imaging spectrometers flying aboard missions such as Chandra and Suzaku, while retaining the low noise, excellent spectral performance, and low power requirements of those instruments. We present the design of the AXIS high-speed X-ray camera, which baselines large-format MIT Lincoln Laboratory CCDs employing low-noise pJFET output amplifiers and a single-layer polysilicon gate structure that allows fast, low-power clocking. These detectors are combined with an integrated high-speed, low-noise ASIC readout chip from Stanford University that provides better performance than conventional discrete solutions at a fraction of their power consumption and footprint. Our complementary front-end electronics concept employs state of the art digital video waveform capture and advanced signal processing to deliver low noise at high speed. We review the current performance of this technology, highlighting recent improvements on prototype devices that achieve excellent noise characteristics at the required readout rate. We present measurements of the CCD spectral response across the AXIS energy band, augmenting lab measurements with detector simulations that help us understand sources of charge loss and evaluate the quality of the CCD backside passivation technique. We show that our technology is on a path that will meet our requirements and enable AXIS to achieve world-class science. △ Less

Submitted 1 September, 2023; originally announced September 2023.

Comments: 17 pages, 11 figures, submitted to Proceedings of SPIE Optics + Photonics 2023

Terahertz imaging through emissivity control

Authors: Michal Mrnka, Harry Penketh, Ian R. Hooper, Sonal Saxena, Nicholas E. Grant, John D. Murphy, David B. Phillips, Euan Hendry

Abstract: Adoption of terahertz technologies is hindered by the lack of cost-effective THz sources. Here we demonstrate a fundamentally new way to generate and control THz radiation, via spatio-temporal emissivity modulation. By patterning the optical photoexcitation of a surface-passivated silicon wafer, we locally control the free-electron density, and thereby pattern the wafer's emissivity in the THz par… ▽ More Adoption of terahertz technologies is hindered by the lack of cost-effective THz sources. Here we demonstrate a fundamentally new way to generate and control THz radiation, via spatio-temporal emissivity modulation. By patterning the optical photoexcitation of a surface-passivated silicon wafer, we locally control the free-electron density, and thereby pattern the wafer's emissivity in the THz part of the electromagnetic spectrum. We show how this unconventional source of controllable THz radiation enables a new form of incoherent computational THz imaging. We use it to image various concealed objects, demonstrating this scheme has the penetrating capability of state-of-the-art THz imaging approaches, without the requirement of femto-second pulsed laser sources. Furthermore, the incoherent nature of thermal radiation also ensures the obtained images are free of interference artifacts. Our spatio-temporal emissivity control paves the way towards a new family of long-wavelength structured illumination, imaging and spectroscopy systems. △ Less

Submitted 23 August, 2023; originally announced August 2023.

AnyStar: Domain randomized universal star-convex 3D instance segmentation

Authors: Neel Dey, S. Mazdak Abulnaga, Benjamin Billot, Esra Abaci Turk, P. Ellen Grant, Adrian V. Dalca, Polina Golland

Abstract: Star-convex shapes arise across bio-microscopy and radiology in the form of nuclei, nodules, metastases, and other units. Existing instance segmentation networks for such structures train on densely labeled instances for each dataset, which requires substantial and often impractical manual annotation effort. Further, significant reengineering or finetuning is needed when presented with new dataset… ▽ More Star-convex shapes arise across bio-microscopy and radiology in the form of nuclei, nodules, metastases, and other units. Existing instance segmentation networks for such structures train on densely labeled instances for each dataset, which requires substantial and often impractical manual annotation effort. Further, significant reengineering or finetuning is needed when presented with new datasets and imaging modalities due to changes in contrast, shape, orientation, resolution, and density. We present AnyStar, a domain-randomized generative model that simulates synthetic training data of blob-like objects with randomized appearance, environments, and imaging physics to train general-purpose star-convex instance segmentation networks. As a result, networks trained using our generative model do not require annotated images from unseen datasets. A single network trained on our synthesized data accurately 3D segments C. elegans and P. dumerilii nuclei in fluorescence microscopy, mouse cortical nuclei in micro-CT, zebrafish brain nuclei in EM, and placental cotyledons in human fetal MRI, all without any retraining, finetuning, transfer learning, or domain adaptation. Code is available at https://github.com/neel-dey/AnyStar. △ Less

Submitted 13 July, 2023; originally announced July 2023.

Comments: Code available at https://github.com/neel-dey/AnyStar

Zero-DeepSub: Zero-Shot Deep Subspace Reconstruction for Rapid Multiparametric Quantitative MRI Using 3D-QALAS

Authors: Yohan Jun, Yamin Arefeen, Jaejin Cho, Shohei Fujita, Xiaoqing Wang, P. Ellen Grant, Borjan Gagoski, Camilo Jaimes, Michael S. Gee, Berkin Bilgic

Abstract: Purpose: To develop and evaluate methods for 1) reconstructing 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) time-series images using a low-rank subspace method, which enables accurate and rapid T1 and T2 mapping, and 2) improving the fidelity of subspace QALAS by combining scan-specific deep-learning-based reconstruction and subspace… ▽ More Purpose: To develop and evaluate methods for 1) reconstructing 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) time-series images using a low-rank subspace method, which enables accurate and rapid T1 and T2 mapping, and 2) improving the fidelity of subspace QALAS by combining scan-specific deep-learning-based reconstruction and subspace modeling. Methods: A low-rank subspace method for 3D-QALAS (i.e., subspace QALAS) and zero-shot deep-learning subspace method (i.e., Zero-DeepSub) were proposed for rapid and high fidelity T1 and T2 mapping and time-resolved imaging using 3D-QALAS. Using an ISMRM/NIST system phantom, the accuracy and reproducibility of the T1 and T2 maps estimated using the proposed methods were evaluated by comparing them with reference techniques. The reconstruction performance of the proposed subspace QALAS using Zero-DeepSub was evaluated in vivo and compared with conventional QALAS at high reduction factors of up to 9-fold. Results: Phantom experiments showed that subspace QALAS had good linearity with respect to the reference methods while reducing biases and improving precision compared to conventional QALAS, especially for T2 maps. Moreover, in vivo results demonstrated that subspace QALAS had better g-factor maps and could reduce voxel blurring, noise, and artifacts compared to conventional QALAS and showed robust performance at up to 9-fold acceleration with Zero-DeepSub, which enabled whole-brain T1, T2, and PD mapping at 1 mm isotropic resolution within 2 min of scan time. Conclusion: The proposed subspace QALAS along with Zero-DeepSub enabled high fidelity and rapid whole-brain multiparametric quantification and time-resolved imaging. △ Less

Submitted 23 January, 2024; v1 submitted 3 July, 2023; originally announced July 2023.

Comments: 24 figures, 4 tables

Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets

Authors: Sheng He, Rina Bao, Jingpeng Li, Jeffrey Stout, Atle Bjornerud, P. Ellen Grant, Yangming Ou

Abstract: Background: The segment-anything model (SAM), introduced in April 2023, shows promise as a benchmark model and a universal solution to segment various natural images. It comes without previously-required re-training or fine-tuning specific to each new dataset. Purpose: To test SAM's accuracy in various medical image segmentation tasks and investigate potential factors that may affect its accurac… ▽ More Background: The segment-anything model (SAM), introduced in April 2023, shows promise as a benchmark model and a universal solution to segment various natural images. It comes without previously-required re-training or fine-tuning specific to each new dataset. Purpose: To test SAM's accuracy in various medical image segmentation tasks and investigate potential factors that may affect its accuracy in medical images. Methods: SAM was tested on 12 public medical image segmentation datasets involving 7,451 subjects. The accuracy was measured by the Dice overlap between the algorithm-segmented and ground-truth masks. SAM was compared with five state-of-the-art algorithms specifically designed for medical image segmentation tasks. Associations of SAM's accuracy with six factors were computed, independently and jointly, including segmentation difficulties as measured by segmentation ability score and by Dice overlap in U-Net, image dimension, size of the target region, image modality, and contrast. Results: The Dice overlaps from SAM were significantly lower than the five medical-image-based algorithms in all 12 medical image segmentation datasets, by a margin of 0.1-0.5 and even 0.6-0.7 Dice. SAM-Semantic was significantly associated with medical image segmentation difficulty and the image modality, and SAM-Point and SAM-Box were significantly associated with image segmentation difficulty, image dimension, target region size, and target-vs-background contrast. All these 3 variations of SAM were more accurate in 2D medical images, larger target region sizes, easier cases with a higher Segmentation Ability score and higher U-Net Dice, and higher foreground-background contrast. △ Less

Submitted 5 May, 2023; v1 submitted 18 April, 2023; originally announced April 2023.

Comments: Technical Report

U-Netmer: U-Net meets Transformer for medical image segmentation

Authors: Sheng He, Rina Bao, P. Ellen Grant, Yangming Ou

Abstract: The combination of the U-Net based deep learning models and Transformer is a new trend for medical image segmentation. U-Net can extract the detailed local semantic and texture information and Transformer can learn the long-rang dependencies among pixels in the input image. However, directly adapting the Transformer for segmentation has ``token-flatten" problem (flattens the local patches into 1D… ▽ More The combination of the U-Net based deep learning models and Transformer is a new trend for medical image segmentation. U-Net can extract the detailed local semantic and texture information and Transformer can learn the long-rang dependencies among pixels in the input image. However, directly adapting the Transformer for segmentation has ``token-flatten" problem (flattens the local patches into 1D tokens which losses the interaction among pixels within local patches) and ``scale-sensitivity" problem (uses a fixed scale to split the input image into local patches). Compared to directly combining U-Net and Transformer, we propose a new global-local fashion combination of U-Net and Transformer, named U-Netmer, to solve the two problems. The proposed U-Netmer splits an input image into local patches. The global-context information among local patches is learnt by the self-attention mechanism in Transformer and U-Net segments each local patch instead of flattening into tokens to solve the `token-flatten" problem. The U-Netmer can segment the input image with different patch sizes with the identical structure and the same parameter. Thus, the U-Netmer can be trained with different patch sizes to solve the ``scale-sensitivity" problem. We conduct extensive experiments in 7 public datasets on 7 organs (brain, heart, breast, lung, polyp, pancreas and prostate) and 4 imaging modalities (MRI, CT, ultrasound, and endoscopy) to show that the proposed U-Netmer can be generally applied to improve accuracy of medical image segmentation. These experimental results show that U-Netmer provides state-of-the-art performance compared to baselines and other models. In addition, the discrepancy among the outputs of U-Netmer with different scales is linearly correlated to the segmentation accuracy which can be considered as a confidence score to rank test images by difficulty without ground-truth. △ Less

Submitted 3 April, 2023; originally announced April 2023.

Comments: 10 pages, 5 figures, under review

SSL-QALAS: Self-Supervised Learning for Rapid Multiparameter Estimation in Quantitative MRI Using 3D-QALAS

Authors: Yohan Jun, Jaejin Cho, Xiaoqing Wang, Michael Gee, P. Ellen Grant, Berkin Bilgic, Borjan Gagoski

Abstract: Purpose: To develop and evaluate a method for rapid estimation of multiparametric T1, T2, proton density (PD), and inversion efficiency (IE) maps from 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) measurements using self-supervised learning (SSL) without the need for an external dictionary. Methods: A SSL-based QALAS mapping method (SS… ▽ More Purpose: To develop and evaluate a method for rapid estimation of multiparametric T1, T2, proton density (PD), and inversion efficiency (IE) maps from 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) measurements using self-supervised learning (SSL) without the need for an external dictionary. Methods: A SSL-based QALAS mapping method (SSL-QALAS) was developed for rapid and dictionary-free estimation of multiparametric maps from 3D-QALAS measurements. The accuracy of the reconstructed quantitative maps using dictionary matching and SSL-QALAS was evaluated by comparing the estimated T1 and T2 values with those obtained from the reference methods on an ISMRM/NIST phantom. The SSL-QALAS and the dictionary matching methods were also compared in vivo, and generalizability was evaluated by comparing the scan-specific, pre-trained, and transfer learning models. Results: Phantom experiments showed that both the dictionary matching and SSL-QALAS methods produced T1 and T2 estimates that had a strong linear agreement with the reference values in the ISMRM/NIST phantom. Further, SSL-QALAS showed similar performance with dictionary matching in reconstructing the T1, T2, PD, and IE maps on in vivo data. Rapid reconstruction of multiparametric maps was enabled by inferring the data using a pre-trained SSL-QALAS model within 10 s. Fast scan-specific tuning was also demonstrated by fine-tuning the pre-trained model with the target subject's data within 15 min. Conclusion: The proposed SSL-QALAS method enabled rapid reconstruction of multiparametric maps from 3D-QALAS measurements without an external dictionary or labeled ground-truth training data. △ Less

Submitted 23 January, 2024; v1 submitted 27 February, 2023; originally announced February 2023.

Comments: 18 figures, 4 tables

Segmentation Ability Map: Interpret deep features for medical image segmentation

Authors: Sheng He, Yanfang Feng, P. Ellen Grant, Yangming Ou

Abstract: Deep convolutional neural networks (CNNs) have been widely used for medical image segmentation. In most studies, only the output layer is exploited to compute the final segmentation results and the hidden representations of the deep learned features have not been well understood. In this paper, we propose a prototype segmentation (ProtoSeg) method to compute a binary segmentation map based on deep… ▽ More Deep convolutional neural networks (CNNs) have been widely used for medical image segmentation. In most studies, only the output layer is exploited to compute the final segmentation results and the hidden representations of the deep learned features have not been well understood. In this paper, we propose a prototype segmentation (ProtoSeg) method to compute a binary segmentation map based on deep features. We measure the segmentation abilities of the features by computing the Dice between the feature segmentation map and ground-truth, named as the segmentation ability score (SA score for short). The corresponding SA score can quantify the segmentation abilities of deep features in different layers and units to understand the deep neural networks for segmentation. In addition, our method can provide a mean SA score which can give a performance estimation of the output on the test images without ground-truth. Finally, we use the proposed ProtoSeg method to compute the segmentation map directly on input images to further understand the segmentation ability of each input image. Results are presented on segmenting tumors in brain MRI, lesions in skin images, COVID-related abnormality in CT images, prostate segmentation in abdominal MRI, and pancreatic mass segmentation in CT images. Our method can provide new insights for interpreting and explainable AI systems for medical image segmentation. Our code is available on: \url{https://github.com/shengfly/ProtoSeg}. △ Less

Submitted 18 December, 2022; originally announced December 2022.

Journal ref: Medical Image Analysis, 2023

Method for in-solution, high-throughput T1 relaxometry using fluorescent nanodiamonds

Authors: Erin. S. Grant, Mina Barzegar Amiri Olia, Ella. P. Walsh, Liam T. Hall, Gawain McColl, David A. Simpson

Abstract: Fluorescent nanodiamonds (FNDs) have been exploited as sensitive quantum probes for nanoscale chemical and biological sensing applications, with the majority of demonstrations to date relying on the detection of single FNDs. This places significant limits on the measurement time, throughput and statistical significance of a measured result as there is usually marked inhomogeneity within FND sample… ▽ More Fluorescent nanodiamonds (FNDs) have been exploited as sensitive quantum probes for nanoscale chemical and biological sensing applications, with the majority of demonstrations to date relying on the detection of single FNDs. This places significant limits on the measurement time, throughput and statistical significance of a measured result as there is usually marked inhomogeneity within FND samples. Here we have developed a measurement platform that can report the T1 spin relaxation time from a large ensemble of FNDs in solution. We first describe a refined sensing protocol for this modality and then use it to identify the optimal FND size for the detection of paramagnetic targets. Our approach is simple to set up, robust and can be used for rapid material characterisation or a variety of in-situ quantum sensing applications. △ Less

Submitted 27 November, 2022; originally announced November 2022.

Comments: 8 pages, 3 figures

Time-efficient, High Resolution 3T Whole Brain Quantitative Relaxometry using 3D-QALAS with Wave-CAIPI Readouts

Authors: Jaejin Cho, Borjan Gagoski, Tae Hyung Kim, Fuyixue Wang, Daniel Nico Splitthoff, Wei-Ching Lo, Wei Liu, Daniel Polak, Stephen Cauley, Kawin Setsompop, P. Ellen Grant, Berkin Bilgic

Abstract: Purpose: Volumetric, high-resolution, quantitative mapping of brain tissue relaxation properties is hindered by long acquisition times and signal-to-noise (SNR) challenges. This study, for the first time, combines the time-efficient wave-CAIPI readouts into the 3D-quantification using an interleaved Look-Locker acquisition sequence with a T2 preparation pulse (3D-QALAS) acquisition scheme, enablin… ▽ More Purpose: Volumetric, high-resolution, quantitative mapping of brain tissue relaxation properties is hindered by long acquisition times and signal-to-noise (SNR) challenges. This study, for the first time, combines the time-efficient wave-CAIPI readouts into the 3D-quantification using an interleaved Look-Locker acquisition sequence with a T2 preparation pulse (3D-QALAS) acquisition scheme, enabling full brain quantitative T1, T2 and proton density (PD) maps at 1.15 mm3 isotropic voxels in only 3 minutes. Methods: Wave-CAIPI readouts were embedded in the standard 3D-QALAS encoding scheme, enabling full brain quantitative parameter maps (T1, T2, and PD) at acceleration factors of R=3x2 with minimum SNR loss due to g-factor penalties. The quantitative parameter maps were estimated using a dictionary-based mapping algorithm incorporating inversion efficiency and B1 field inhomogeneity. The quantitative maps using the accelerated protocol were quantitatively compared against those obtained from conventional 3D-QALAS sequence using GRAPPA acceleration of R=2 in the ISMRM NIST phantom, and ten healthy volunteers. Results: When tested in both the ISMRM/NIST phantom and ten healthy volunteers, the quantitative maps using the accelerated protocol showed excellent agreement against those obtained from conventional 3D-QALAS at RGRAPPA=2. Conclusion: 3D-QALAS enhanced with wave-CAIPI readouts enables time-efficient, full brain quantitative T1, T2, and PD mapping at 1.15 mm3 in 3 minutes at R=3x2 acceleration. When tested on the NIST phantom and ten healthy volunteers, the quantitative maps obtained from the accelerated wave-CAIPI 3D-QALAS protocol showed very similar values to those obtained from the standard 3D-QALAS (R=2) protocol, alluding to the robustness and reliability of the proposed methods. △ Less

Submitted 27 January, 2023; v1 submitted 8 November, 2022; originally announced November 2022.

Reducing the background in X-ray imaging detectors via machine learning

Authors: D. R. Wilkins, S. W. Allen, E. D. Miller, M. Bautz, T. Chattopadhyay, R. Foster, C. E. Grant, S. Hermann, R. Kraft, R. G. Morris, P. Nulsen, G. Schellenberger

Abstract: The sensitivity of astronomical X-ray detectors is limited by the instrumental background. The background is especially important when observing low surface brightness sources that are critical for many of the science cases targeted by future X-ray observatories, including Athena and future US-led flagship or probe-class X-ray missions. Above 2keV, the background is dominated by signals induced by… ▽ More The sensitivity of astronomical X-ray detectors is limited by the instrumental background. The background is especially important when observing low surface brightness sources that are critical for many of the science cases targeted by future X-ray observatories, including Athena and future US-led flagship or probe-class X-ray missions. Above 2keV, the background is dominated by signals induced by cosmic rays interacting with the spacecraft and detector. We develop novel machine learning algorithms to identify events in next-generation X-ray imaging detectors and to predict the probability that an event is induced by a cosmic ray vs. an astrophysical X-ray photon, enabling enhanced filtering of the cosmic ray-induced background. We find that by learning the typical correlations between the secondary events that arise from a single primary, machine learning algorithms are able to successfully identify cosmic ray-induced background events that are missed by traditional filtering methods employed on current-generation X-ray missions, reducing the unrejected background by as much as 30 per cent. △ Less

Submitted 16 August, 2022; originally announced August 2022.

Comments: Proceedings of the SPIE, Astronomical Telescopes and Instrumentation, Space Telescopes and Instrumentation 2022: Ultraviolet to Gamma Ray

Journal ref: Proc. SPIE, 2022, 12181, 155

Understanding the effects of charge diffusion in next-generation soft X-ray imagers

Authors: Eric D. Miller, Gregory Y. Prigozhin, Beverly J. LaMarr, Marshall W. Bautz, Richard F. Foster, Catherine E. Grant, Craig S. Lage, Christopher Leitz, Andrew Malonis

Abstract: To take advantage of high-resolution optics sensitive to a broad energy range, future X-ray imaging instruments will require thick detectors with small pixels. This pixel aspect ratio affects spectral response in the soft X-ray band, vital for many science goals, as charge produced by the photon interaction near the entrance window diffuses across multiple pixels by the time it is collected, and i… ▽ More To take advantage of high-resolution optics sensitive to a broad energy range, future X-ray imaging instruments will require thick detectors with small pixels. This pixel aspect ratio affects spectral response in the soft X-ray band, vital for many science goals, as charge produced by the photon interaction near the entrance window diffuses across multiple pixels by the time it is collected, and is potentially lost below the imposed noise threshold. In an effort to understand these subtle but significant effects and inform the design and requirements of future detectors, we present simulations of charge diffusion using a variety of detector characteristics and operational settings, assessing spectral response at a range of X-ray energies. We validate the simulations by comparing the performance to that of real CCD detectors tested in the lab and deployed in space, spanning a range of thickness, pixel size, and other characteristics. The simulations show that while larger pixels, higher bias voltage, and optimal backside passivation improve performance, reducing the readout noise has a dominant effect in all cases. We finally show how high-pixel-aspect-ratio devices present challenges for measuring the backside passivation performance due to the magnitude of other processes that degrade spectral response, and present a method for utilizing the simulations to qualitatively assess this performance. Since compelling science requirements often compete technically with each other (high spatial resolution, soft X-ray response, hard X-ray response), these results can be used to find the proper balance for a future high-spatial-resolution X-ray instrument. △ Less

Submitted 15 August, 2022; originally announced August 2022.

Comments: 11 pages, 7 figures, submitted to Proceedings of SPIE Astronomical Telescopes + Instrumentation 2022

Gaussian Process Surrogate Models for Neural Networks

Authors: Michael Y. Li, Erin Grant, Thomas L. Griffiths

Abstract: Not being able to understand and predict the behavior of deep learning systems makes it hard to decide what architecture and algorithm to use for a given problem. In science and engineering, modeling is a methodology used to understand complex systems whose internal processes are opaque. Modeling replaces a complex system with a simpler, more interpretable surrogate. Drawing inspiration from this,… ▽ More Not being able to understand and predict the behavior of deep learning systems makes it hard to decide what architecture and algorithm to use for a given problem. In science and engineering, modeling is a methodology used to understand complex systems whose internal processes are opaque. Modeling replaces a complex system with a simpler, more interpretable surrogate. Drawing inspiration from this, we construct a class of surrogate models for neural networks using Gaussian processes. Rather than deriving kernels for infinite neural networks, we learn kernels empirically from the naturalistic behavior of finite neural networks. We demonstrate our approach captures existing phenomena related to the spectral bias of neural networks, and then show that our surrogate models can be used to solve practical problems such as identifying which points most influence the behavior of specific neural networks and predicting which architectures and algorithms will generalize well for specific datasets. △ Less

Submitted 14 September, 2023; v1 submitted 11 August, 2022; originally announced August 2022.

Comments: Proceedings of UAI 2023

Automatic Segmentation of the Placenta in BOLD MRI Time Series

Authors: S. Mazdak Abulnaga, Sean I. Young, Katherine Hobgood, Eileen Pan, Clinton J. Wang, P. Ellen Grant, Esra Abaci Turk, Polina Golland

Abstract: Blood oxygen level dependent (BOLD) MRI with maternal hyperoxia can assess oxygen transport within the placenta and has emerged as a promising tool to study placental function. Measuring signal changes over time requires segmenting the placenta in each volume of the time series. Due to the large number of volumes in the BOLD time series, existing studies rely on registration to map all volumes to… ▽ More Blood oxygen level dependent (BOLD) MRI with maternal hyperoxia can assess oxygen transport within the placenta and has emerged as a promising tool to study placental function. Measuring signal changes over time requires segmenting the placenta in each volume of the time series. Due to the large number of volumes in the BOLD time series, existing studies rely on registration to map all volumes to a manually segmented template. As the placenta can undergo large deformation due to fetal motion, maternal motion, and contractions, this approach often results in a large number of discarded volumes, where the registration approach fails. In this work, we propose a machine learning model based on a U-Net neural network architecture to automatically segment the placenta in BOLD MRI and apply it to segmenting each volume in a time series. We use a boundary-weighted loss function to accurately capture the placental shape. Our model is trained and tested on a cohort of 91 subjects containing healthy fetuses, fetuses with fetal growth restriction, and mothers with high BMI. We achieve a Dice score of 0.83+/-0.04 when matching with ground truth labels and our model performs reliably in segmenting volumes in both normoxic and hyperoxic points in the BOLD time series. Our code and trained model are available at https://github.com/mabulnaga/automatic-placenta-segmentation. △ Less

Submitted 4 August, 2022; originally announced August 2022.

Comments: Accepted at MICCAI PIPPI 2022

Towards precision particle background estimation for future X-ray missions: correlated variability between Chandra ACIS and AMS

Authors: Catherine E. Grant, Eric D. Miller, Marshall W. Bautz, Richard Foster, Ralph P. Kraft, Steven Allen, David N. Burrows

Abstract: A science goal of many future X-ray observatories is mapping the cosmic web through deep exposures of faint diffuse sources. Such observations require low background and the best possible knowledge of the remaining unrejected background. The dominant contribution to the background above 1-2 keV is from Galactic Cosmic Ray protons. Their flux and spectrum are modulated by the solar cycle but also b… ▽ More A science goal of many future X-ray observatories is mapping the cosmic web through deep exposures of faint diffuse sources. Such observations require low background and the best possible knowledge of the remaining unrejected background. The dominant contribution to the background above 1-2 keV is from Galactic Cosmic Ray protons. Their flux and spectrum are modulated by the solar cycle but also by solar activity on shorter timescales. Understanding this variability may prove crucial to reducing background uncertainty for ESA's Athena X-ray Observatory and other missions with large collecting area. We examine of the variability of the particle background as measured by ACIS on the Chandra X-ray Observatory and compare that variability to that measured by the Alpha Magnetic Spectrometer (AMS), a precision particle detector on the ISS. We show that cosmic ray proton variability measured by AMS is well matched to the ACIS background and can be used to estimate proton energies responsible for the background. We discuss how this can inform future missions. △ Less

Submitted 1 August, 2022; originally announced August 2022.

Comments: 11 pages, 8 figures, submitted to Proceedings of SPIE Astronomical Telescopes + Instrumentation 2022

SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI

Authors: Junshen Xu, Daniel Moyer, P. Ellen Grant, Polina Golland, Juan Eugenio Iglesias, Elfar Adalsteinsson

Abstract: Volumetric reconstruction of fetal brains from multiple stacks of MR slices, acquired in the presence of almost unpredictable and often severe subject motion, is a challenging task that is highly sensitive to the initialization of slice-to-volume transformations. We propose a novel slice-to-volume registration method using Transformers trained on synthetically transformed data, which model multipl… ▽ More Volumetric reconstruction of fetal brains from multiple stacks of MR slices, acquired in the presence of almost unpredictable and often severe subject motion, is a challenging task that is highly sensitive to the initialization of slice-to-volume transformations. We propose a novel slice-to-volume registration method using Transformers trained on synthetically transformed data, which model multiple stacks of MR slices as a sequence. With the attention mechanism, our model automatically detects the relevance between slices and predicts the transformation of one slice using information from other slices. We also estimate the underlying 3D volume to assist slice-to-volume registration and update the volume and transformations alternately to improve accuracy. Results on synthetic data show that our method achieves lower registration error and better reconstruction quality compared with existing state-of-the-art methods. Experiments with real-world MRI data are also performed to demonstrate the ability of the proposed model to improve the quality of 3D reconstruction under severe fetal motion. △ Less

Submitted 21 June, 2022; originally announced June 2022.

Comments: Accepted by MICCAI 2022