-
Large Language Models for Bioinformatics
Authors:
Wei Ruan,
Yanjun Lyu,
Jing Zhang,
Jiazhang Cai,
Peng Shu,
Yang Ge,
Yao Lu,
Shang Gao,
Yue Wang,
Peilong Wang,
Lin Zhao,
Tao Wang,
Yufang Liu,
Luyang Fang,
Ziyu Liu,
Zhengliang Liu,
Yiwei Li,
Zihao Wu,
Junhao Chen,
Hanqi Jiang,
Yi Pan,
Zhenyuan Yang,
Jingyuan Chen,
Shizhe Liang,
Wei Zhang
, et al. (30 additional authors not shown)
Abstract:
With the rapid advancements in large language model (LLM) technology and the emergence of bioinformatics-specific language models (BioLMs), there is a growing need for a comprehensive analysis of the current landscape, computational characteristics, and diverse applications. This survey aims to address this need by providing a thorough review of BioLMs, focusing on their evolution, classification,…
▽ More
With the rapid advancements in large language model (LLM) technology and the emergence of bioinformatics-specific language models (BioLMs), there is a growing need for a comprehensive analysis of the current landscape, computational characteristics, and diverse applications. This survey aims to address this need by providing a thorough review of BioLMs, focusing on their evolution, classification, and distinguishing features, alongside a detailed examination of training methodologies, datasets, and evaluation frameworks. We explore the wide-ranging applications of BioLMs in critical areas such as disease diagnosis, drug discovery, and vaccine development, highlighting their impact and transformative potential in bioinformatics. We identify key challenges and limitations inherent in BioLMs, including data privacy and security concerns, interpretability issues, biases in training data and model outputs, and domain adaptation complexities. Finally, we highlight emerging trends and future directions, offering valuable insights to guide researchers and clinicians toward advancing BioLMs for increasingly sophisticated biological and clinical applications.
△ Less
Submitted 9 January, 2025;
originally announced January 2025.
-
Hierarchical Working Memory and a New Magic Number
Authors:
Weishun Zhong,
Mikhail Katkov,
Misha Tsodyks
Abstract:
The extremely limited working memory span, typically around four items, contrasts sharply with our everyday experience of processing much larger streams of sensory information concurrently. This disparity suggests that working memory can organize information into compact representations such as chunks, yet the underlying neural mechanisms remain largely unknown. Here, we propose a recurrent neural…
▽ More
The extremely limited working memory span, typically around four items, contrasts sharply with our everyday experience of processing much larger streams of sensory information concurrently. This disparity suggests that working memory can organize information into compact representations such as chunks, yet the underlying neural mechanisms remain largely unknown. Here, we propose a recurrent neural network model for chunking within the framework of the synaptic theory of working memory. We showed that by selectively suppressing groups of stimuli, the network can maintain and retrieve the stimuli in chunks, hence exceeding the basic capacity. Moreover, we show that our model can dynamically construct hierarchical representations within working memory through hierarchical chunking. A consequence of this proposed mechanism is a new limit on the number of items that can be stored and subsequently retrieved from working memory, depending only on the basic working memory capacity when chunking is not invoked. Predictions from our model were confirmed by analyzing single-unit responses in epileptic patients and memory experiments with verbal material. Our work provides a novel conceptual and analytical framework for understanding the on-the-fly organization of information in the brain that is crucial for cognition.
△ Less
Submitted 14 August, 2024;
originally announced August 2024.
-
Data navigation on the ENCODE portal
Authors:
Meenakshi S. Kagda,
Bonita Lam,
Casey Litton,
Corinn Small,
Cricket A. Sloan,
Emma Spragins,
Forrest Tanaka,
Ian Whaling,
Idan Gabdank,
Ingrid Youngworth,
J. Seth Strattan,
Jason Hilton,
Jennifer Jou,
Jessica Au,
Jin-Wook Lee,
Kalina Andreeva,
Keenan Graham,
Khine Lin,
Matt Simison,
Otto Jolanki,
Paul Sud,
Pedro Assis,
Philip Adenekan,
Eric Douglas,
Mingjie Li
, et al. (9 additional authors not shown)
Abstract:
Spanning two decades, the Encyclopaedia of DNA Elements (ENCODE) is a collaborative research project that aims to identify all the functional elements in the human and mouse genomes. To best serve the scientific community, all data generated by the consortium is shared through a web-portal (https://www.encodeproject.org/) with no access restrictions. The fourth and final phase of the project added…
▽ More
Spanning two decades, the Encyclopaedia of DNA Elements (ENCODE) is a collaborative research project that aims to identify all the functional elements in the human and mouse genomes. To best serve the scientific community, all data generated by the consortium is shared through a web-portal (https://www.encodeproject.org/) with no access restrictions. The fourth and final phase of the project added a diverse set of new samples (including those associated with human disease), and a wide range of new assays aimed at detection, characterization and validation of functional genomic elements. The ENCODE data portal hosts results from over 23,000 functional genomics experiments, over 800 functional elements characterization experiments (including in vivo transgenic enhancer assays, reporter assays and CRISPR screens) along with over 60,000 results of computational and integrative analyses (including imputations, predictions and genome annotations). The ENCODE Data Coordination Center (DCC) is responsible for development and maintenance of the data portal, along with the implementation and utilisation of the ENCODE uniform processing pipelines to generate uniformly processed data. Here we report recent updates to the data portal. Specifically, we have completely redesigned the home page, improved search interface, added several new pages to highlight collections of biologically related data (deeply profiled cell lines, immune cells, Alzheimer's Disease, RNA-Protein interactions, degron matrix and a matrix of experiments organised by human donors), added single-cell experiments, and enhanced the cart interface for visualisation and download of user-selected datasets.
△ Less
Submitted 4 May, 2023; v1 submitted 27 April, 2023;
originally announced May 2023.
-
A theory of learning with constrained weight-distribution
Authors:
Weishun Zhong,
Ben Sorscher,
Daniel D Lee,
Haim Sompolinsky
Abstract:
A central question in computational neuroscience is how structure determines function in neural networks. The emerging high-quality large-scale connectomic datasets raise the question of what general functional principles can be gleaned from structural information such as the distribution of excitatory/inhibitory synapse types and the distribution of synaptic weights. Motivated by this question, w…
▽ More
A central question in computational neuroscience is how structure determines function in neural networks. The emerging high-quality large-scale connectomic datasets raise the question of what general functional principles can be gleaned from structural information such as the distribution of excitatory/inhibitory synapse types and the distribution of synaptic weights. Motivated by this question, we developed a statistical mechanical theory of learning in neural networks that incorporates structural information as constraints. We derived an analytical solution for the memory capacity of the perceptron, a basic feedforward model of supervised learning, with constraint on the distribution of its weights. Our theory predicts that the reduction in capacity due to the constrained weight-distribution is related to the Wasserstein distance between the imposed distribution and that of the standard normal distribution. To test the theoretical predictions, we use optimal transport theory and information geometry to develop an SGD-based algorithm to find weights that simultaneously learn the input-output task and satisfy the distribution constraint. We show that training in our algorithm can be interpreted as geodesic flows in the Wasserstein space of probability distributions. We further developed a statistical mechanical theory for teacher-student perceptron rule learning and ask for the best way for the student to incorporate prior knowledge of the rule. Our theory shows that it is beneficial for the learner to adopt different prior weight distributions during learning, and shows that distribution-constrained learning outperforms unconstrained and sign-constrained learning. Our theory and algorithm provide novel strategies for incorporating prior knowledge about weights into learning, and reveal a powerful connection between structure and function in neural networks.
△ Less
Submitted 24 October, 2022; v1 submitted 13 June, 2022;
originally announced June 2022.
-
Non-equilibrium statistical mechanics of continuous attractors
Authors:
Weishun Zhong,
Zhiyue Lu,
David J Schwab,
Arvind Murugan
Abstract:
Continuous attractors have been used to understand recent neuroscience experiments where persistent activity patterns encode internal representations of external attributes like head direction or spatial location. However, the conditions under which the emergent bump of neural activity in such networks can be manipulated by space and time-dependent external sensory or motor signals are not underst…
▽ More
Continuous attractors have been used to understand recent neuroscience experiments where persistent activity patterns encode internal representations of external attributes like head direction or spatial location. However, the conditions under which the emergent bump of neural activity in such networks can be manipulated by space and time-dependent external sensory or motor signals are not understood. Here, we find fundamental limits on how rapidly internal representations encoded along continuous attractors can be updated by an external signal. We apply these results to place cell networks to derive a velocity-dependent non-equilibrium memory capacity in neural networks.
△ Less
Submitted 30 December, 2018; v1 submitted 28 September, 2018;
originally announced September 2018.
-
Associative pattern recognition through macro-molecular self-assembly
Authors:
Weishun Zhong,
David J. Schwab,
Arvind Murugan
Abstract:
We show that macro-molecular self-assembly can recognize and classify high-dimensional patterns in the concentrations of $N$ distinct molecular species. Similar to associative neural networks, the recognition here leverages dynamical attractors to recognize and reconstruct partially corrupted patterns. Traditional parameters of pattern recognition theory, such as sparsity, fidelity, and capacity a…
▽ More
We show that macro-molecular self-assembly can recognize and classify high-dimensional patterns in the concentrations of $N$ distinct molecular species. Similar to associative neural networks, the recognition here leverages dynamical attractors to recognize and reconstruct partially corrupted patterns. Traditional parameters of pattern recognition theory, such as sparsity, fidelity, and capacity are related to physical parameters, such as nucleation barriers, interaction range, and non-equilibrium assembly forces. Notably, we find that self-assembly bears greater similarity to continuous attractor neural networks, such as place cell networks that store spatial memories, rather than discrete memory networks. This relationship suggests that features and trade-offs seen here are not tied to details of self-assembly or neural network models but are instead intrinsic to associative pattern recognition carried out through short-ranged interactions.
△ Less
Submitted 24 February, 2017; v1 submitted 6 January, 2017;
originally announced January 2017.
-
Spatiotemporal Noise Triggering Infiltrative Tumor Growth under Immune Surveillance
Authors:
Wei-Rong Zhong,
Yuan-Zhi Shao,
Li Li,
Feng-Hua Wang,
Zhen-Hui He
Abstract:
A spatiotemporal noise is assumed to reflect the environmental fluctuation in a spatially extended tumor system. We introduce firstly the structure factor to reveal the invasive tumor growth quantitatively. The homogenous environment can lead to an expansive growth of the tumor cells, while the inhomogenous environment underlies an infiltrative growth. The different responses of above two cases…
▽ More
A spatiotemporal noise is assumed to reflect the environmental fluctuation in a spatially extended tumor system. We introduce firstly the structure factor to reveal the invasive tumor growth quantitatively. The homogenous environment can lead to an expansive growth of the tumor cells, while the inhomogenous environment underlies an infiltrative growth. The different responses of above two cases are separated by a characteristic critical intensity of the spatiotemporal noise. Theoretical and numerical results present a close annotation to the clinical images.
△ Less
Submitted 2 January, 2008;
originally announced January 2008.
-
Noise Correlation Induced Synchronization in a Mutualism Ecosystem
Authors:
Wei-Rong Zhong,
Yuan-Zhi Shao,
Zhen-Hui He,
Meng-Jie Bie,
Dan Huang
Abstract:
Understanding the cause of the synchronization of population evolution is an important issue for ecological improvement. Here we present a Lotka-Volterra-type model driven by two correlated environmental noises and show, via theoretical analysis and direct simulation, that noise correlation can induce a synchronization of the mutualists. The time series of mutual species exhibit a chaotic-like f…
▽ More
Understanding the cause of the synchronization of population evolution is an important issue for ecological improvement. Here we present a Lotka-Volterra-type model driven by two correlated environmental noises and show, via theoretical analysis and direct simulation, that noise correlation can induce a synchronization of the mutualists. The time series of mutual species exhibit a chaotic-like fluctuation, which is independent to the noise correlation, however, the chaotic fluctuation of mutual species ratio decreases with the noise correlation. A quantitative parameter defined for characterizing chaotic fluctuation provides a good approach to measure when the complete synchronization happens.
△ Less
Submitted 21 January, 2007;
originally announced January 2007.
-
Pure multiplicative stochastic resonance of anti-tumor model with seasonal modulability
Authors:
Wei-Rong Zhong,
Yuan-Zhi Shao,
Zhen-Hui He
Abstract:
The effects of pure multiplicative noise on stochastic resonance in an anti-tumor system modulated by a seasonal external field are investigated by using theoretical analyses of the generalized potential and numerical simulations. For optimally selected values of the multiplicative noise intensity quasi-symmetry of two potential minima and stochastic resonance are observed. Theoretical results a…
▽ More
The effects of pure multiplicative noise on stochastic resonance in an anti-tumor system modulated by a seasonal external field are investigated by using theoretical analyses of the generalized potential and numerical simulations. For optimally selected values of the multiplicative noise intensity quasi-symmetry of two potential minima and stochastic resonance are observed. Theoretical results and numerical simulations are in good quantitative agreement.
△ Less
Submitted 11 January, 2006;
originally announced January 2006.
-
Sensitive predators and endurant preys in an ecosystem driven by correlated noises
Authors:
Wei-Rong Zhong,
Yuan-Zhi Shao,
Zhen-Hui He
Abstract:
We investigate the Volterra ecosystem driven by correlated noises. The competition of the predators induces an increasing in population density of the predators. The competition of the preys, however, leads the predators to decay. The predators may have better stability under strong correlated noises. The predators undergo a sensitivity to a random environment, whereas the preys exhibit a surpri…
▽ More
We investigate the Volterra ecosystem driven by correlated noises. The competition of the predators induces an increasing in population density of the predators. The competition of the preys, however, leads the predators to decay. The predators may have better stability under strong correlated noises. The predators undergo a sensitivity to a random environment, whereas the preys exhibit a surprising endurance to this stochasticity.
△ Less
Submitted 17 August, 2005;
originally announced August 2005.
