Embed-Search-Align: DNA Sequence Alignment using Transformer Models
Authors:
Pavan Holur,
K. C. Enevoldsen,
Shreyas Rajesh,
Lajoyce Mboning,
Thalia Georgiou,
Louis-S. Bouchard,
Matteo Pellegrini,
Vwani Roychowdhury
Abstract:
DNA sequence alignment involves assigning short DNA reads to the most probable locations on an extensive reference genome. This process is crucial for various genomic analyses, including variant calling, transcriptomics, and epigenomics. Conventional methods, refined over decades, tackle this challenge in 2 steps: genome indexing followed by efficient search to locate likely positions for given re…
▽ More
DNA sequence alignment involves assigning short DNA reads to the most probable locations on an extensive reference genome. This process is crucial for various genomic analyses, including variant calling, transcriptomics, and epigenomics. Conventional methods, refined over decades, tackle this challenge in 2 steps: genome indexing followed by efficient search to locate likely positions for given reads. Building on the success of Large Language Models in encoding text into embeddings, where the distance metric captures semantic similarity, recent efforts have explored whether the same Transformer architecture can produce embeddings for DNA sequences. Such models have shown early promise in classifying short DNA sequences, such as detecting coding/non-coding regions, and enhancer, promoter sequences. However, performance at sequence classification tasks does not translate to sequence alignment, where it is necessary to search across the genome to align each read, a significantly longer-range task. We bridge this gap by framing the Sequence Alignment task for Transformer models as an "Embed-Search-Align" task. In this framework, a novel Reference-Free DNA Embedding model generates embeddings of reads and reference fragments, which are projected into a shared vector space where the read-fragment distance is used as a surrogate for alignment. Technical contributions include: (1) Contrastive loss for self-supervised training of DNA sequence representations, facilitating rich reference-free, sequence-level embeddings, and (2) a DNA vector store to enable search across fragments on a global scale. DNA-ESA is 99% accurate when aligning 250-length reads onto a human genome (3gb), rivaling conventional methods such as Bowtie and BWA-Mem. DNA-ESA exceeds the performance of 6 Transformer model baselines such as Nucleotide Transformer, Hyena-DNA, and shows task transfer across chromosomes and species.
△ Less
Submitted 5 December, 2024; v1 submitted 20 September, 2023;
originally announced September 2023.
Challenges and perspectives in computational deconvolution of genomics data
Authors:
Lana X. Garmire,
Yijun Li,
Qianhui Huang,
Chuan Xu,
Sarah Teichmann,
Naftali Kaminski,
Matteo Pellegrini,
Quan Nguyen,
Andrew E. Teschendorff
Abstract:
Deciphering cell type heterogeneity is crucial for systematically understanding tissue homeostasis and its dysregulation in diseases. Computational deconvolution is an efficient approach estimating cell type abundances from a variety of omics data. Despite significant methodological progress in computational deconvolution in recent years, challenges are still outstanding. Here we enlist four signi…
▽ More
Deciphering cell type heterogeneity is crucial for systematically understanding tissue homeostasis and its dysregulation in diseases. Computational deconvolution is an efficient approach estimating cell type abundances from a variety of omics data. Despite significant methodological progress in computational deconvolution in recent years, challenges are still outstanding. Here we enlist four significant challenges related to computational deconvolution, from the quality of the reference data, generation of ground truth data, limitations of computational methodologies, and benchmarking design and implementation. Finally, we make recommendations on reference data generation, new directions of computational methodologies and strategies to promote rigorous benchmarking.
△ Less
Submitted 2 September, 2023; v1 submitted 21 November, 2022;
originally announced November 2022.
Position Dependent and Independent Evolutionary Models Based on Empirical Amino Acid Substitution Matrices
Authors:
B. Barbiellini,
Alexandra Portnova,
Anna Chetoukhina,
Chia-Hsin Lu,
Matteo Pellegrini
Abstract:
Evolutionary models measure the probability of amino acid substitutions occurring over different evolutionary distances. We examine various evolutionary models based on empirically derived amino acid substitution matrices. The models are constructed using the PAM and BLOSUM amino acid substitution matrices. We rescale these matrices by raising them to powers to model substitution patterns that a…
▽ More
Evolutionary models measure the probability of amino acid substitutions occurring over different evolutionary distances. We examine various evolutionary models based on empirically derived amino acid substitution matrices. The models are constructed using the PAM and BLOSUM amino acid substitution matrices. We rescale these matrices by raising them to powers to model substitution patterns that account for different evolutionary distances. We also examine models that account for the dissimilarity of substitution rates along a protein sequence. We compare the models by computing the likelihood of each model across different alignments. We also present a specific example to illustrate the subtle differences in the estimation of evolutionary distance computed using the different models.
△ Less
Submitted 19 May, 2004;
originally announced May 2004.
