Showing 1–12 of 12 results for author: Deng, S

Decoding fairness: a reinforcement learning perspective

Authors: Guozhong Zheng, Jiqiang Zhang, Xin Ou, Shengfeng Deng, Li Chen

Abstract: Behavioral experiments on the ultimatum game (UG) reveal that we humans prefer fair acts, which contradicts the prediction made in orthodox Economics. Existing explanations, however, are mostly attributed to exogenous factors within the imitation learning framework. Here, we adopt the reinforcement learning paradigm, where individuals make their moves aiming to maximize their accumulated rewards.… ▽ More Behavioral experiments on the ultimatum game (UG) reveal that we humans prefer fair acts, which contradicts the prediction made in orthodox Economics. Existing explanations, however, are mostly attributed to exogenous factors within the imitation learning framework. Here, we adopt the reinforcement learning paradigm, where individuals make their moves aiming to maximize their accumulated rewards. Specifically, we apply Q-learning to UG, where each player is assigned two Q-tables to guide decisions for the roles of proposer and responder. In a two-player scenario, fairness emerges prominently when both experiences and future rewards are appreciated. In particular, the probability of successful deals increases with higher offers, which aligns with observations in behavioral experiments. Our mechanism analysis reveals that the system undergoes two phases, eventually stabilizing into fair or rational strategies. These results are robust when the rotating role assignment is replaced by a random or fixed manner, or the scenario is extended to a latticed population. Our findings thus conclude that the endogenous factor is sufficient to explain the emergence of fairness, exogenous factors are not needed. △ Less

Submitted 19 December, 2024; originally announced December 2024.

Comments: 12 pages, 13 figures. Comments are appreciated

Evolution of cooperation in the public goods game with Q-learning

Authors: Guozhong Zheng, Jiqiang Zhang, Shengfeng Deng, Weiran Cai, Li Chen

Abstract: Recent paradigm shifts from imitation learning to reinforcement learning (RL) is shown to be productive in understanding human behaviors. In the RL paradigm, individuals search for optimal strategies through interaction with the environment to make decisions. This implies that gathering, processing, and utilizing information from their surroundings are crucial. However, existing studies typically… ▽ More Recent paradigm shifts from imitation learning to reinforcement learning (RL) is shown to be productive in understanding human behaviors. In the RL paradigm, individuals search for optimal strategies through interaction with the environment to make decisions. This implies that gathering, processing, and utilizing information from their surroundings are crucial. However, existing studies typically study pairwise games such as the prisoners' dilemma and employ a self-regarding setup, where individuals play against one opponent based solely on their own strategies, neglecting the environmental information. In this work, we investigate the evolution of cooperation with the multiplayer game -- the public goods game using the Q-learning algorithm by leveraging the environmental information. Specifically, the decision-making of players is based upon the cooperation information in their neighborhood. Our results show that cooperation is more likely to emerge compared to the case of imitation learning by using Fermi rule. Of particular interest is the observation of an anomalous non-monotonic dependence which is revealed when voluntary participation is further introduced. The analysis of the Q-table explains the mechanisms behind the cooperation evolution. Our findings indicate the fundamental role of environment information in the RL paradigm to understand the evolution of cooperation, and human behaviors in general. △ Less

Submitted 29 July, 2024; originally announced July 2024.

Comments: 16 pages, 12 figures, comments are appreciated

Journal ref: Chaos, Solitons & Fractals 188, 115568 (2024)

Evolution of cooperation with Q-learning: the impact of information perception

Authors: Guozhong Zheng, Zhenwei Ding, Jiqiang Zhang, Shengfeng Deng, Weiran Cai, Li Chen

Abstract: The inherent complexity of human beings manifests in a remarkable diversity of responses to intricate environments, enabling us to approach problems from varied perspectives. However, in the study of cooperation, existing research within the reinforcement learning framework often assumes that individuals have access to identical information when making decisions, which contrasts with the reality t… ▽ More The inherent complexity of human beings manifests in a remarkable diversity of responses to intricate environments, enabling us to approach problems from varied perspectives. However, in the study of cooperation, existing research within the reinforcement learning framework often assumes that individuals have access to identical information when making decisions, which contrasts with the reality that individuals frequently perceive information differently. In this study, we employ the Q-learning algorithm to explore the impact of information perception on the evolution of cooperation in a two-person Prisoner's Dilemma game. We demonstrate that the evolutionary processes differ significantly across three distinct information perception scenarios, highlighting the critical role of information structure in the emergence of cooperation. Notably, the asymmetric information scenario reveals a complex dynamical process, including the emergence, breakdown, and reconstruction of cooperation, mirroring psychological shifts observed in human behavior. Our findings underscore the importance of information structure in fostering cooperation, offering new insights into the establishment of stable cooperative relationships among humans. △ Less

Submitted 18 February, 2025; v1 submitted 28 July, 2024; originally announced July 2024.

Comments: 13pages, 13figures, comments are appreciated

Network Pharmacology, Molecular Docking, and MR Analysis: Targets and Mechanisms of Gegen Qinlian Decoction for Helicobacter pylori

Authors: Ruotong. Lu, Xiaozhe. Huang, Sihuan. Deng, Haikun. Du

Abstract: Objective: The study explored therapeutic targets and mechanisms of Gegen Qinlian Decoction for Helicobacter pylori infection and related gastric cancer using network pharmacology, molecular docking, and Mendelian randomization. Methods: Medicinal components of Gegen Qinlian Decoction were extracted from TCMSP and HERB databases. Disease treatment targets were sourced from DisGeNET and PubChem.… ▽ More Objective: The study explored therapeutic targets and mechanisms of Gegen Qinlian Decoction for Helicobacter pylori infection and related gastric cancer using network pharmacology, molecular docking, and Mendelian randomization. Methods: Medicinal components of Gegen Qinlian Decoction were extracted from TCMSP and HERB databases. Disease treatment targets were sourced from DisGeNET and PubChem. Interaction networks were constructed via the STRING database and visualized using Cytoscape 3.9.1. Enrichment analysis of intersected targets was performed using DAVID and Metascapes. Molecular docking employed Autodock Tools 1.5.6 and PyMOL 2.5.2. Mendelian randomization was based on the ukb-b-531 sample from UK Biobank. Results: 146 active components and 248 targets from Gegen Qinlian Decoction were identified. 66 targets overlapped with Helicobacter pylori infection genes. Molecular docking highlighted interactions between primary drug components like quercetin, wogonin, kaempferol, and target genes PTGS1, PTGS2, MAPK14. Mendelian randomization pinpointed genes like IGF2, PIK3CG, GJA1, and PLAU associated with Helicobacter pylori infection. Conclusion: Gegen Qinlian Decoction's active components target Helicobacter pylori infection through diverse targets and pathways, presenting potential research avenues. △ Less

Submitted 26 September, 2023; originally announced September 2023.

Comments: 16pages

Synchronization transitions on connectome graphs with external force

Authors: Géza Ódor, István Papp, Shengfeng Deng, Jeffrey Kelling

Abstract: We investigate the synchronization transition of the Shinomoto-Kuramoto model on networks of the fruit-fly and two large human connectomes. This model contains a force term, thus is capable of describing critical behavior in the presence of external excitation. By numerical solution we determine the crackling noise durations with and without thermal noise and show extended non-universal scaling ta… ▽ More We investigate the synchronization transition of the Shinomoto-Kuramoto model on networks of the fruit-fly and two large human connectomes. This model contains a force term, thus is capable of describing critical behavior in the presence of external excitation. By numerical solution we determine the crackling noise durations with and without thermal noise and show extended non-universal scaling tails characterized by $2< τ_t < 2.8$, in contrast with the Hopf transition of the Kuramoto model, without the force $τ_t=3.1(1)$. Comparing the phase and frequency order parameters we find different transition points and fluctuations peaks as in case of the Kuramoto model. Using the local order parameter values we also determine the Hurst (phase) and $β$ (frequency) exponents and compare them with recent experimental results obtained by fMRI. We show that these exponents, characterizing the auto-correlations are smaller in the excited system than in the resting state and exhibit module dependence. △ Less

Submitted 12 January, 2023; originally announced January 2023.

Comments: 11 pages, 12 figures

Journal ref: Front. Phys. 11 (2023) 1150246

OntoProtein: Protein Pretraining With Gene Ontology Embedding

Authors: Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Jiazhang Lian, Qiang Zhang, Huajun Chen

Abstract: Self-supervised protein language models have proved their effectiveness in learning the proteins representations. With the increasing computational power, current protein language models pre-trained with millions of diverse sequences can advance the parameter scale from million-level to billion-level and achieve remarkable improvement. However, those prevailing approaches rarely consider incorpora… ▽ More Self-supervised protein language models have proved their effectiveness in learning the proteins representations. With the increasing computational power, current protein language models pre-trained with millions of diverse sequences can advance the parameter scale from million-level to billion-level and achieve remarkable improvement. However, those prevailing approaches rarely consider incorporating knowledge graphs (KGs), which can provide rich structured knowledge facts for better protein representations. We argue that informative biology knowledge in KGs can enhance protein representation with external knowledge. In this work, we propose OntoProtein, the first general framework that makes use of structure in GO (Gene Ontology) into protein pre-training models. We construct a novel large-scale knowledge graph that consists of GO and its related proteins, and gene annotation texts or protein sequences describe all nodes in the graph. We propose novel contrastive learning with knowledge-aware negative sampling to jointly optimize the knowledge graph and protein embedding during pre-training. Experimental results show that OntoProtein can surpass state-of-the-art methods with pre-trained protein language models in TAPE benchmark and yield better performance compared with baselines in protein-protein interaction and protein function prediction. Code and datasets are available in https://github.com/zjunlp/OntoProtein. △ Less

Submitted 3 June, 2022; v1 submitted 23 January, 2022; originally announced January 2022.

Comments: Accepted by ICLR 2022

Control Theory Illustrates the Energy Efficiency in the Dynamic Reconfiguration of Functional Connectivity

Authors: Shikuang Deng, Jingwei Li, B. T. Thomas Yeo, Shi Gu

Abstract: The brain's functional connectivity fluctuates over time instead of remaining steady in a stationary mode even during the resting state. This fluctuation establishes the dynamical functional connectivity that transitions in a non-random order between multiple modes. Yet it remains unexplored how the transition facilitates the entire brain network as a dynamical system and what utility this mechani… ▽ More The brain's functional connectivity fluctuates over time instead of remaining steady in a stationary mode even during the resting state. This fluctuation establishes the dynamical functional connectivity that transitions in a non-random order between multiple modes. Yet it remains unexplored how the transition facilitates the entire brain network as a dynamical system and what utility this mechanism for dynamic reconfiguration can bring over the widely used graph theoretical measurements. To address these questions, we propose to conduct an energetic analysis of functional brain networks using resting-state fMRI and behavioral measurements from the Human Connectome Project. Through comparing the state transition energy under distinct adjacent matrices, we justify that dynamic functional connectivity leads to 60% less energy cost to support the resting state dynamics than static connectivity when driving the transition through default mode network. Moreover, we demonstrate that combining graph theoretical measurements and our energy-based control measurements as the feature vector can provide complementary prediction power for the behavioral scores. Our approach integrates statistical inference and dynamical system inspection towards understanding brain networks. △ Less

Submitted 25 March, 2022; v1 submitted 7 January, 2022; originally announced January 2022.

Molecular Contrastive Learning with Chemical Element Knowledge Graph

Authors: Yin Fang, Qiang Zhang, Haihong Yang, Xiang Zhuang, Shumin Deng, Wen Zhang, Ming Qin, Zhuo Chen, Xiaohui Fan, Huajun Chen

Abstract: Molecular representation learning contributes to multiple downstream tasks such as molecular property prediction and drug design. To properly represent molecules, graph contrastive learning is a promising paradigm as it utilizes self-supervision signals and has no requirements for human annotations. However, prior works fail to incorporate fundamental domain knowledge into graph semantics and thus… ▽ More Molecular representation learning contributes to multiple downstream tasks such as molecular property prediction and drug design. To properly represent molecules, graph contrastive learning is a promising paradigm as it utilizes self-supervision signals and has no requirements for human annotations. However, prior works fail to incorporate fundamental domain knowledge into graph semantics and thus ignore the correlations between atoms that have common attributes but are not directly connected by bonds. To address these issues, we construct a Chemical Element Knowledge Graph (KG) to summarize microscopic associations between elements and propose a novel Knowledge-enhanced Contrastive Learning (KCL) framework for molecular representation learning. KCL framework consists of three modules. The first module, knowledge-guided graph augmentation, augments the original molecular graph based on the Chemical Element KG. The second module, knowledge-aware graph representation, extracts molecular representations with a common graph encoder for the original molecular graph and a Knowledge-aware Message Passing Neural Network (KMPNN) to encode complex information in the augmented molecular graph. The final module is a contrastive objective, where we maximize agreement between these two views of molecular graphs. Extensive experiments demonstrated that KCL obtained superior performances against state-of-the-art baselines on eight molecular datasets. Visualization experiments properly interpret what KCL has learned from atoms and attributes in the augmented molecular graphs. Our codes and data are available at https://github.com/ZJU-Fangyin/KCL. △ Less

Submitted 10 March, 2022; v1 submitted 1 December, 2021; originally announced December 2021.

Comments: Accepted in AAAI 2022 Main track

Inference of cell dynamics on perturbation data using adjoint sensitivity

Authors: Weiqi Ji, Bo Yuan, Ciyue Shen, Aviv Regev, Chris Sander, Sili Deng

Abstract: Data-driven dynamic models of cell biology can be used to predict cell response to unseen perturbations. Recent work (CellBox) had demonstrated the derivation of interpretable models with explicit interaction terms, in which the parameters were optimized using machine learning techniques. While the previous work was tested only in a single biological setting, this work aims to extend the range of… ▽ More Data-driven dynamic models of cell biology can be used to predict cell response to unseen perturbations. Recent work (CellBox) had demonstrated the derivation of interpretable models with explicit interaction terms, in which the parameters were optimized using machine learning techniques. While the previous work was tested only in a single biological setting, this work aims to extend the range of applicability of this model inference approach to a diversity of biological systems. Here we adapted CellBox in Julia differential programming and augmented the method with adjoint algorithms, which has recently been used in the context of neural ODEs. We trained the models using simulated data from both abstract and biology-inspired networks, which afford the ability to evaluate the recovery of the ground truth network structure. The resulting accuracy of prediction by these models is high both in terms of low error against data and excellent agreement with the network structure used for the simulated training data. While there is no analogous ground truth for real life biological systems, this work demonstrates the ability to construct and parameterize a considerable diversity of network models with high predictive ability. The expectation is that this kind of procedure can be used on real perturbation-response data to derive models applicable to diverse biological systems. △ Less

Submitted 13 April, 2021; originally announced April 2021.

Comments: Accepted as a workshop paper at ICLR 2021 SimDL Workshop

Social distancing and epidemic resurgence in agent-based Susceptible-Infectious-Recovered models

Authors: Ruslan I. Mukhamadiarov, Shengfeng Deng, Shannon R. Serrao, Priyanka, Riya Nandi, Louie Hong Yao, Uwe C. Täuber

Abstract: Once an epidemic outbreak has been effectively contained through non-pharmaceutical interventions, a safe protocol is required for the subsequent release of social distancing restrictions to prevent a disastrous resurgence of the infection. We report individual-based numerical simulations of stochastic susceptible-infectious-recovered model variants on four distinct spatially organized lattice and… ▽ More Once an epidemic outbreak has been effectively contained through non-pharmaceutical interventions, a safe protocol is required for the subsequent release of social distancing restrictions to prevent a disastrous resurgence of the infection. We report individual-based numerical simulations of stochastic susceptible-infectious-recovered model variants on four distinct spatially organized lattice and network architectures wherein contact and mobility constraints are implemented. We robustly find that the intensity and spatial spread of the epidemic recurrence wave can be limited to a manageable extent provided release of these restrictions is delayed sufficiently (for a duration of at least thrice the time until the peak of the unmitigated outbreak) and long-distance connections are maintained on a low level (limited to less than five percent of the overall connectivity). △ Less

Submitted 20 December, 2020; v1 submitted 3 June, 2020; originally announced June 2020.

Comments: 6 figures in main text, 4 more in supplementary materials; to appear in Scientific Reports

Journal ref: Scientific Reports 11 (2021) 130

Controllability Analysis of Functional Brain Networks

Authors: Shikuang Deng, Shi Gu

Abstract: Network control theory has recently emerged as a promising approach for understanding brain function and dynamics. By operationalizing notions of control theory for brain networks, it offers a fundamental explanation for how brain dynamics may be regulated by structural connectivity. While powerful, the approach does not currently consider other non-structural explanations of brain dynamics. Here… ▽ More Network control theory has recently emerged as a promising approach for understanding brain function and dynamics. By operationalizing notions of control theory for brain networks, it offers a fundamental explanation for how brain dynamics may be regulated by structural connectivity. While powerful, the approach does not currently consider other non-structural explanations of brain dynamics. Here we extend the analysis of network controllability by formalizing the evolution of neural signals as a function of effective inter-regional coupling and pairwise signal covariance. We find that functional controllability characterizes a region's impact on the capacity for the whole system to shift between states, and significantly predicts individual difference in performance on cognitively demanding tasks including those task working memory, language, and emotional intelligence. When comparing measurements from functional and structural controllability, we observed consistent relations between average and modal controllability, supporting prior work. In the same comparison, we also observed distinct relations between controllability and synchronizability, reflecting the additional information obtained from functional signals. Our work suggests that network control theory can serve as a systematic analysis tool to understand the energetics of brain state transitions, associated cognitive processes, and subsequent behaviors. △ Less

Submitted 18 March, 2020; originally announced March 2020.

Autonomous Discovery of Unknown Reaction Pathways from Data by Chemical Reaction Neural Network

Authors: Weiqi Ji, Sili Deng

Abstract: Chemical reactions occur in energy, environmental, biological, and many other natural systems, and the inference of the reaction networks is essential to understand and design the chemical processes in engineering and life sciences. Yet, revealing the reaction pathways for complex systems and processes is still challenging due to the lack of knowledge of the involved species and reactions. Here, w… ▽ More Chemical reactions occur in energy, environmental, biological, and many other natural systems, and the inference of the reaction networks is essential to understand and design the chemical processes in engineering and life sciences. Yet, revealing the reaction pathways for complex systems and processes is still challenging due to the lack of knowledge of the involved species and reactions. Here, we present a neural network approach that autonomously discovers reaction pathways from the time-resolved species concentration data. The proposed Chemical Reaction Neural Network (CRNN), by design, satisfies the fundamental physics laws, including the Law of Mass Action and the Arrhenius Law. Consequently, the CRNN is physically interpretable such that the reaction pathways can be interpreted, and the kinetic parameters can be quantified simultaneously from the weights of the neural network. The inference of the chemical pathways is accomplished by training the CRNN with species concentration data via stochastic gradient descent. We demonstrate the successful implementations and the robustness of the approach in elucidating the chemical reaction pathways of several chemical engineering and biochemical systems. The autonomous inference by the CRNN approach precludes the need for expert knowledge in proposing candidate networks and addresses the curse of dimensionality in complex systems. The physical interpretability also makes the CRNN capable of not only fitting the data for a given system but also developing knowledge of unknown pathways that could be generalized to similar chemical systems. △ Less

Submitted 8 January, 2021; v1 submitted 20 February, 2020; originally announced February 2020.

Journal ref: The Journal of Physical Chemistry A, 2021