AI-driven 3D Spatial Transcriptomics
Authors:
Cristina Almagro-Pérez,
Andrew H. Song,
Luca Weishaupt,
Ahrong Kim,
Guillaume Jaume,
Drew F. K. Williamson,
Konstantin Hemker,
Ming Y. Lu,
Kritika Singh,
Bowen Chen,
Long Phi Le,
Alexander S. Baras,
Sizun Jiang,
Ali Bashashati,
Jonathan T. C. Liu,
Faisal Mahmood
Abstract:
A comprehensive three-dimensional (3D) map of tissue architecture and gene expression is crucial for illuminating the complexity and heterogeneity of tissues across diverse biomedical applications. However, most spatial transcriptomics (ST) approaches remain limited to two-dimensional (2D) sections of tissue. Although current 3D ST methods hold promise, they typically require extensive tissue sect…
▽ More
A comprehensive three-dimensional (3D) map of tissue architecture and gene expression is crucial for illuminating the complexity and heterogeneity of tissues across diverse biomedical applications. However, most spatial transcriptomics (ST) approaches remain limited to two-dimensional (2D) sections of tissue. Although current 3D ST methods hold promise, they typically require extensive tissue sectioning, are complex, are not compatible with non-destructive 3D tissue imaging technologies, and often lack scalability. Here, we present VOlumetrically Resolved Transcriptomics EXpression (VORTEX), an AI framework that leverages 3D tissue morphology and minimal 2D ST to predict volumetric 3D ST. By pretraining on diverse 3D morphology-transcriptomic pairs from heterogeneous tissue samples and then fine-tuning on minimal 2D ST data from a specific volume of interest, VORTEX learns both generic tissue-related and sample-specific morphological correlates of gene expression. This approach enables dense, high-throughput, and fast 3D ST, scaling seamlessly to large tissue volumes far beyond the reach of existing 3D ST techniques. By offering a cost-effective and minimally destructive route to obtaining volumetric molecular insights, we anticipate that VORTEX will accelerate biomarker discovery and our understanding of morphomolecular associations and cell states in complex tissues. Interactive 3D ST volumes can be viewed at https://vortex-demo.github.io/
△ Less
Submitted 24 February, 2025;
originally announced February 2025.
Multimodal Whole Slide Foundation Model for Pathology
Authors:
Tong Ding,
Sophia J. Wagner,
Andrew H. Song,
Richard J. Chen,
Ming Y. Lu,
Andrew Zhang,
Anurag J. Vaidya,
Guillaume Jaume,
Muhammad Shaban,
Ahrong Kim,
Drew F. K. Williamson,
Bowen Chen,
Cristina Almagro-Perez,
Paul Doucet,
Sharifa Sahai,
Chengkuan Chen,
Daisuke Komura,
Akihiro Kawabe,
Shumpei Ishikawa,
Georg Gerber,
Tingying Peng,
Long Phi Le,
Faisal Mahmood
Abstract:
The field of computational pathology has been transformed with recent advances in foundation models that encode histopathology region-of-interests (ROIs) into versatile and transferable feature representations via self-supervised learning (SSL). However, translating these advancements to address complex clinical challenges at the patient and slide level remains constrained by limited clinical data…
▽ More
The field of computational pathology has been transformed with recent advances in foundation models that encode histopathology region-of-interests (ROIs) into versatile and transferable feature representations via self-supervised learning (SSL). However, translating these advancements to address complex clinical challenges at the patient and slide level remains constrained by limited clinical data in disease-specific cohorts, especially for rare clinical conditions. We propose TITAN, a multimodal whole slide foundation model pretrained using 335,645 WSIs via visual self-supervised learning and vision-language alignment with corresponding pathology reports and 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology. Without any finetuning or requiring clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports that generalize to resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis. We evaluate TITAN on diverse clinical tasks and find that TITAN outperforms both ROI and slide foundation models across machine learning settings such as linear probing, few-shot and zero-shot classification, rare cancer retrieval and cross-modal retrieval, and pathology report generation.
△ Less
Submitted 29 November, 2024;
originally announced November 2024.
