Showing 1–6 of 6 results for author: Pop, M

Contributions of the Petabyte Scale Sequence Search Codeathon toward efforts to scale sequence-based searches on SRA

Authors: Priyanka Ghosh, Kjiersten Fagnan, Ryan Connor, Ravinder Pannu, Travis J. Wheeler, Mihai Pop, C. Titus Brown, Tessa Pierce-Ward, Rob Patro, Jacquelyn S. Michaelis, Thomas L. Madden, Christiam Camacho, Olaitan I. Awe, Arianna I. Krinos, René KM Xavier, Rodrigo Ortega Polo, Jack W. Roddy, Adelaide Rhodes, Alexander Sweeten, Adrian Viehweger, Bariş Ekim, Harihara Subrahmaniam Muralidharan, Amatur Rahman, Vinícius W. Salazar, Andrew Tritt , et al. (13 additional authors not shown)

Abstract: The volume of biological data being generated by the scientific community is growing exponentially, reflecting technological advances and research activities. The National Institutes of Health's (NIH) Sequence Read Archive (SRA), which is maintained by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), is a rapidly growing public database that resea… ▽ More The volume of biological data being generated by the scientific community is growing exponentially, reflecting technological advances and research activities. The National Institutes of Health's (NIH) Sequence Read Archive (SRA), which is maintained by the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), is a rapidly growing public database that researchers use to drive scientific discovery across all domains of life. This increase in available data has great promise for pushing scientific discovery but also introduces new challenges that scientific communities need to address. As genomic datasets have grown in scale and diversity, a parade of new methods and associated software have been developed to address the challenges posed by this growth. These methodological advances are vital for maximally leveraging the power of next-generation sequencing (NGS) technologies. With the goal of laying a foundation for evaluation of methods for petabyte-scale sequence search, the Department of Energy (DOE) Office of Biological and Environmental Research (BER), the NIH Office of Data Science Strategy (ODSS), and NCBI held a virtual codeathon 'Petabyte Scale Sequence Search: Metagenomics Benchmarking Codeathon' on September 27 - Oct 1 2021, to evaluate emerging solutions in petabyte scale sequence search. The codeathon attracted experts from national laboratories, research institutions, and universities across the world to (a) develop benchmarking approaches to address challenges in conducting large-scale analyses of metagenomic data (which comprises approximately 20% of SRA), (b) identify potential applications that benefit from SRA-wide searches and the tools required to execute the search, and (c) produce community resources i.e. a public facing repository with information to rebuild and reproduce the problems addressed by each team challenge. △ Less

Submitted 9 May, 2025; originally announced May 2025.

Graph-based variant discovery reveals novel dynamics in the human microbiome

Authors: Harihara Subrahmaniam Muralidharan, Jacquelyn S Michaelis, Jay Ghurye, Todd Treangen, Sergey Koren, Marcus Fedarko, Mihai Pop

Abstract: Sequence differences between the strains of bacteria comprising host-associated and environmental microbiota may play a role in community assembly and influence the resilience of microbial communities to disturbances. Tools for characterizing strain-level variation within microbial communities, however, are limited in scope, focusing on just single nucleotide polymorphisms, or relying on reference… ▽ More Sequence differences between the strains of bacteria comprising host-associated and environmental microbiota may play a role in community assembly and influence the resilience of microbial communities to disturbances. Tools for characterizing strain-level variation within microbial communities, however, are limited in scope, focusing on just single nucleotide polymorphisms, or relying on reference-based analyses that miss complex functional and structural variants. Here, we demonstrate the power of assembly graph analysis to detect and characterize structural variants in almost 1,000 metagenomes generated as part of the Human Microbiome Project. We identify over nine million variants comprising insertion/deletion events, repeat copy-number changes, and mobile elements such as plasmids. We highlight some of the potential functional roles of these genomic changes. Our analysis revealed striking differences in the rate of variation across body sites, highlighting niche-specific mechanisms of bacterial adaptation. The structural variants we detect also include potentially novel prophage integration events, highlighting the potential use of graph-based analyses for phage discovery. △ Less

Submitted 3 March, 2024; originally announced March 2024.

MetaCompass: Reference-guided Assembly of Metagenomes

Authors: Tu Luan, Victoria Cepeda, Bo Liu, Zac Bowen, Ujjwal Ayyangar, Mathieu Almeida, Christopher M. Hill, Sergey Koren, Todd J. Treangen, Adam Porter, Mihai Pop

Abstract: Metagenomic studies have primarily relied on de novo assembly for reconstructing genes and genomes from microbial mixtures. While reference-guided approaches have been employed in the assembly of single organisms, they have not been used in a metagenomic context. Here we describe the first effective approach for reference-guided metagenomic assembly that can complement and improve upon de novo met… ▽ More Metagenomic studies have primarily relied on de novo assembly for reconstructing genes and genomes from microbial mixtures. While reference-guided approaches have been employed in the assembly of single organisms, they have not been used in a metagenomic context. Here we describe the first effective approach for reference-guided metagenomic assembly that can complement and improve upon de novo metagenomic assembly methods for certain organisms. Such approaches will be increasingly useful as more genomes are sequenced and made publicly available. △ Less

Submitted 3 March, 2024; originally announced March 2024.

Comments: 18 pages, 6 figures, 3 tables, one supplementary material

Robust Parameter Estimation for Biological Systems: A Study on the Dynamics of Microbial Communities

Authors: Matthias Chung, Justin Krueger, Mihai Pop

Abstract: Interest in the study of in-host microbial communities has increased in recent years due to our improved understanding of the communities' significant role in host health. As a result, the ability to model these communities using differential equations, for example, and analyze the results has become increasingly relevant. The size of the models and limitations in data collection among many other… ▽ More Interest in the study of in-host microbial communities has increased in recent years due to our improved understanding of the communities' significant role in host health. As a result, the ability to model these communities using differential equations, for example, and analyze the results has become increasingly relevant. The size of the models and limitations in data collection among many other considerations require that we develop new parameter estimation methods to address the challenges that arise when using traditional parameter estimation methods for models of these in-host microbial communities. In this work, we present the challenges that appear when applying traditional parameter estimation techniques to differential equation models of microbial communities, and we provide an original, alternative method to those techniques. We show the derivation of our method and how our method avoids the limitations of traditional techniques while including additional benefits. We also provide simulation studies to demonstrate our method's viability, the application of our method to a model of intestinal microbial communities to demonstrate the insights that can be gained from our method, and sample code to give readers the opportunity to apply our method to their own research. △ Less

Submitted 23 September, 2015; originally announced September 2015.

MSC Class: 65L09; 92B05; 65K05

SparseAssembler2: Sparse k-mer Graph for Memory Efficient Genome Assembly

Authors: Chengxi Ye, Charles H. Cannon, Zhanshan Sam Ma, Douglas W. Yu, Mihai Pop

Abstract: The formal version of our work has been published in BMC Bioinformatics and can be found here: http://www.biomedcentral.com/1471-2105/13/S6/S1 Motivation: To tackle the problem of huge memory usage associated with de Bruijn graph-based algorithms, upon which some of the most widely used de novo genome assemblers have been built, we released SparseAssembler1. SparseAssembler1 can save as much as 90… ▽ More The formal version of our work has been published in BMC Bioinformatics and can be found here: http://www.biomedcentral.com/1471-2105/13/S6/S1 Motivation: To tackle the problem of huge memory usage associated with de Bruijn graph-based algorithms, upon which some of the most widely used de novo genome assemblers have been built, we released SparseAssembler1. SparseAssembler1 can save as much as 90% memory consumption in comparison with the state-of-art assemblers, but it requires rounds of denoising to accurately assemble genomes. In this paper, we introduce a new general model for genome assembly that uses only sparse k-mers. The new model replaces the idea of the de Bruijn graph from the beginning, and achieves similar memory efficiency and much better robustness compared with our previous SparseAssembler1. Results: We demonstrate that the decomposition of reads of all overlapping k-mers, which is used in existing de Bruijn graph genome assemblers, is overly cautious. We introduce a sparse k-mer graph structure for saving sparse k-mers, which greatly reduces memory space requirements necessary for de novo genome assembly. In contrast with the de Bruijn graph approach, we devise a simple but powerful strategy, i.e., finding links between the k-mers in the genome and traversing following the links, which can be done by saving only a few k-mers. To implement the strategy, we need to only select some k-mers that may not even be overlapping ones, and build the links between these k-mers indicated by the reads. We can traverse through this sparse k-mer graph to build the contigs, and ultimately complete the genome assembly. Since the new sparse k-mers graph shares almost all advantages of de Bruijn graph, we are able to adapt a Dijkstra-like breadth-first search algorithm to circumvent sequencing errors and resolve polymorphisms. △ Less

Submitted 9 January, 2013; v1 submitted 17 August, 2011; originally announced August 2011.

Comments: Corresponding authors: Zhanshan (Sam) Ma, [email protected]; Mihai Pop, [email protected] || Availability: Programs in both Windows and Linux are available at: https://sites.google.com/site/sparseassembler/

SparseAssembler: de novo Assembly with the Sparse de Bruijn Graph

Authors: Chengxi Ye, Zhanshan Sam Ma, Charles H. Cannon, Mihai Pop, Douglas W. Yu

Abstract: de Bruijn graph-based algorithms are one of the two most widely used approaches for de novo genome assembly. A major limitation of this approach is the large computational memory space requirement to construct the de Bruijn graph, which scales with k-mer length and total diversity (N) of unique k-mers in the genome expressed in base pairs or roughly (2k+8)N bits. This limitation is particularly im… ▽ More de Bruijn graph-based algorithms are one of the two most widely used approaches for de novo genome assembly. A major limitation of this approach is the large computational memory space requirement to construct the de Bruijn graph, which scales with k-mer length and total diversity (N) of unique k-mers in the genome expressed in base pairs or roughly (2k+8)N bits. This limitation is particularly important with large-scale genome analysis and for sequencing centers that simultaneously process multiple genomes. We present a sparse de Bruijn graph structure, based on which we developed SparseAssembler that greatly reduces memory space requirements. The structure also allows us to introduce a novel method for the removal of substitution errors introduced during sequencing. The sparse de Bruijn graph structure skips g intermediate k-mers, therefore reducing the theoretical memory space requirement to ~(2k/g+8)N. We have found that a practical value of g=16 consumes approximately 10% of the memory required by standard de Bruijn graph-based algorithms but yields comparable results. A high error rate could potentially derail the SparseAssembler. Therefore, we developed a sparse de Bruijn graph-based denoising algorithm that can remove more than 99% of substitution errors from datasets with a \leq 2% error rate. Given that substitution error rates for the current generation of sequencers is lower than 1%, our denoising procedure is sufficiently effective to safeguard the performance of our algorithm. Finally, we also introduce a novel Dijkstra-like breadth-first search algorithm for the sparse de Bruijn graph structure to circumvent residual errors and resolve polymorphisms. △ Less

Submitted 14 June, 2011; originally announced June 2011.

Comments: Corresponding author: Douglas W. Yu, [email protected]