-
A Deep Generative Model of Neonatal Cortical Surface Development
Authors:
Abdulah Fawaz,
Logan Z. Williams,
A. David Edwards,
Emma Robinson
Abstract:
The neonatal cortical surface is known to be affected by preterm birth, and the subsequent changes to cortical organisation have been associated with poorer neurodevelopmental outcomes. Deep Generative models have the potential to lead to clinically interpretable models of disease, but developing these on the cortical surface is challenging since established techniques for learning convolutional f…
▽ More
The neonatal cortical surface is known to be affected by preterm birth, and the subsequent changes to cortical organisation have been associated with poorer neurodevelopmental outcomes. Deep Generative models have the potential to lead to clinically interpretable models of disease, but developing these on the cortical surface is challenging since established techniques for learning convolutional filters are inappropriate on non-flat topologies. To close this gap, we implement a surface-based CycleGAN using mixture model CNNs (MoNet) to translate sphericalised neonatal cortical surface features (curvature and T1w/T2w cortical myelin) between different stages of cortical maturity. Results show our method is able to reliably predict changes in individual patterns of cortical organisation at later stages of gestation, validated by comparison to longitudinal data; and translate appearance between preterm and term gestation (> 37 weeks gestation), validated through comparison with a trained term/preterm classifier. Simulated differences in cortical maturation are consistent with observations in the literature.
△ Less
Submitted 22 June, 2022; v1 submitted 15 June, 2022;
originally announced June 2022.
-
Surface Vision Transformers: Flexible Attention-Based Modelling of Biomedical Surfaces
Authors:
Simon Dahan,
Hao Xu,
Logan Z. J. Williams,
Abdulah Fawaz,
Chunhui Yang,
Timothy S. Coalson,
Michelle C. Williams,
David E. Newby,
A. David Edwards,
Matthew F. Glasser,
Alistair A. Young,
Daniel Rueckert,
Emma C. Robinson
Abstract:
Recent state-of-the-art performances of Vision Transformers (ViT) in computer vision tasks demonstrate that a general-purpose architecture, which implements long-range self-attention, could replace the local feature learning operations of convolutional neural networks. In this paper, we extend ViTs to surfaces by reformulating the task of surface learning as a sequence-to-sequence learning problem…
▽ More
Recent state-of-the-art performances of Vision Transformers (ViT) in computer vision tasks demonstrate that a general-purpose architecture, which implements long-range self-attention, could replace the local feature learning operations of convolutional neural networks. In this paper, we extend ViTs to surfaces by reformulating the task of surface learning as a sequence-to-sequence learning problem, by proposing patching mechanisms for general surface meshes. Sequences of patches are then processed by a transformer encoder and used for classification or regression. We validate our method on a range of different biomedical surface domains and tasks: brain age prediction in the developing Human Connectome Project (dHCP), fluid intelligence prediction in the Human Connectome Project (HCP), and coronary artery calcium score classification using surfaces from the Scottish Computed Tomography of the Heart (SCOT-HEART) dataset, and investigate the impact of pretraining and data augmentation on model performance. Results suggest that Surface Vision Transformers (SiT) demonstrate consistent improvement over geometric deep learning methods for brain age and fluid intelligence prediction and achieve comparable performance on calcium score classification to standard metrics used in clinical practice. Furthermore, analysis of transformer attention maps offers clear and individualised predictions of the features driving each task. Code is available on Github: https://github.com/metrics-lab/surface-vision-transformers
△ Less
Submitted 7 April, 2022;
originally announced April 2022.
-
Surface Vision Transformers: Attention-Based Modelling applied to Cortical Analysis
Authors:
Simon Dahan,
Abdulah Fawaz,
Logan Z. J. Williams,
Chunhui Yang,
Timothy S. Coalson,
Matthew F. Glasser,
A. David Edwards,
Daniel Rueckert,
Emma C. Robinson
Abstract:
The extension of convolutional neural networks (CNNs) to non-Euclidean geometries has led to multiple frameworks for studying manifolds. Many of those methods have shown design limitations resulting in poor modelling of long-range associations, as the generalisation of convolutions to irregular surfaces is non-trivial. Motivated by the success of attention-modelling in computer vision, we translat…
▽ More
The extension of convolutional neural networks (CNNs) to non-Euclidean geometries has led to multiple frameworks for studying manifolds. Many of those methods have shown design limitations resulting in poor modelling of long-range associations, as the generalisation of convolutions to irregular surfaces is non-trivial. Motivated by the success of attention-modelling in computer vision, we translate convolution-free vision transformer approaches to surface data, to introduce a domain-agnostic architecture to study any surface data projected onto a spherical manifold. Here, surface patching is achieved by representing spherical data as a sequence of triangular patches, extracted from a subdivided icosphere. A transformer model encodes the sequence of patches via successive multi-head self-attention layers while preserving the sequence resolution. We validate the performance of the proposed Surface Vision Transformer (SiT) on the task of phenotype regression from cortical surface metrics derived from the Developing Human Connectome Project (dHCP). Experiments show that the SiT generally outperforms surface CNNs, while performing comparably on registered and unregistered data. Analysis of transformer attention maps offers strong potential to characterise subtle cognitive developmental patterns.
△ Less
Submitted 30 March, 2022;
originally announced March 2022.
-
Characterising brain network topologies: a dynamic analysis approach using heat kernels
Authors:
A. W. Chung,
M. D. Schirmer,
M. L. Krishna,
G. Ball,
P. Aljabar,
A. D. Edwards,
G. Montana
Abstract:
Network theory provides a principled abstraction of the human brain: reducing a complex system into a simpler representation from which to investigate brain organisation. Recent advancement in the neuroimaging field are towards representing brain connectivity as a dynamic process in order to gain a deeper understanding of how the brain is organised for information transport. In this paper we propo…
▽ More
Network theory provides a principled abstraction of the human brain: reducing a complex system into a simpler representation from which to investigate brain organisation. Recent advancement in the neuroimaging field are towards representing brain connectivity as a dynamic process in order to gain a deeper understanding of how the brain is organised for information transport. In this paper we propose a network modelling approach based on the heat kernel to capture the process of heat diffusion in complex networks. By applying the heat kernel to structural brain networks, we define new features which quantify change in energy flow. Identifying suitable features which can classify networks between cohorts is useful towards understanding the effect of disease on brain architecture. We demonstrate the discriminative power of heat kernel features in both synthetic and clinical preterm data. By generating an extensive range of synthetic networks with varying density and randomisation, we investigate how heat flows in the networks in relation to changes in network topology. We demonstrate that our proposed features provide a metric of network efficiency and may be indicative of organisational principles commonly associated with, for example, small-world architecture. In addition, we show the potential of these features to characterise and classify between network topologies. We further demonstrate our methodology in a clinical setting by applying it to a large cohort of preterm babies scanned at term equivalent age from which diffusion networks were computed. We show that our heat kernel features are able to successfully predict motor function measured at two years of age (sensitivity, specificity, F-score, accuracy = 75.0, 82.5, 78.6, 82.3%, respectively.
△ Less
Submitted 22 March, 2016;
originally announced March 2016.
