Showing 1–11 of 11 results for author: Duan, J

HAD: Hybrid Architecture Distillation Outperforms Teacher in Genomic Sequence Modeling

Authors: Hexiong Yang, Mingrui Chen, Huaibo Huang, Junxian Duan, Jie Cao, Zhen Zhou, Ran He

Abstract: Inspired by the great success of Masked Language Modeling (MLM) in the natural language domain, the paradigm of self-supervised pre-training and fine-tuning has also achieved remarkable progress in the field of DNA sequence modeling. However, previous methods often relied on massive pre-training data or large-scale base models with huge parameters, imposing a significant computational burden. To a… ▽ More Inspired by the great success of Masked Language Modeling (MLM) in the natural language domain, the paradigm of self-supervised pre-training and fine-tuning has also achieved remarkable progress in the field of DNA sequence modeling. However, previous methods often relied on massive pre-training data or large-scale base models with huge parameters, imposing a significant computational burden. To address this, many works attempted to use more compact models to achieve similar outcomes but still fell short by a considerable margin. In this work, we propose a Hybrid Architecture Distillation (HAD) approach, leveraging both distillation and reconstruction tasks for more efficient and effective pre-training. Specifically, we employ the NTv2-500M as the teacher model and devise a grouping masking strategy to align the feature embeddings of visible tokens while concurrently reconstructing the invisible tokens during MLM pre-training. To validate the effectiveness of our proposed method, we conducted comprehensive experiments on the Nucleotide Transformer Benchmark and Genomic Benchmark. Compared to models with similar parameters, our model achieved excellent performance. More surprisingly, it even surpassed the distillation ceiling-teacher model on some sub-tasks, which is more than 500 $\times$ larger. Lastly, we utilize t-SNE for more intuitive visualization, which shows that our model can gain a sophisticated understanding of the intrinsic representation pattern in genomic sequences. △ Less

Submitted 27 May, 2025; originally announced May 2025.

Pathology-genomic fusion via biologically informed cross-modality graph learning for survival analysis

Authors: Zeyu Zhang, Yuanshen Zhao, Jingxian Duan, Yaou Liu, Hairong Zheng, Dong Liang, Zhenyu Zhang, Zhi-Cheng Li

Abstract: The diagnosis and prognosis of cancer are typically based on multi-modal clinical data, including histology images and genomic data, due to the complex pathogenesis and high heterogeneity. Despite the advancements in digital pathology and high-throughput genome sequencing, establishing effective multi-modal fusion models for survival prediction and revealing the potential association between histo… ▽ More The diagnosis and prognosis of cancer are typically based on multi-modal clinical data, including histology images and genomic data, due to the complex pathogenesis and high heterogeneity. Despite the advancements in digital pathology and high-throughput genome sequencing, establishing effective multi-modal fusion models for survival prediction and revealing the potential association between histopathology and transcriptomics remains challenging. In this paper, we propose Pathology-Genome Heterogeneous Graph (PGHG) that integrates whole slide images (WSI) and bulk RNA-Seq expression data with heterogeneous graph neural network for cancer survival analysis. The PGHG consists of biological knowledge-guided representation learning network and pathology-genome heterogeneous graph. The representation learning network utilizes the biological prior knowledge of intra-modal and inter-modal data associations to guide the feature extraction. The node features of each modality are updated through attention-based graph learning strategy. Unimodal features and bi-modal fused features are extracted via attention pooling module and then used for survival prediction. We evaluate the model on low-grade gliomas, glioblastoma, and kidney renal papillary cell carcinoma datasets from the Cancer Genome Atlas (TCGA) and the First Affiliated Hospital of Zhengzhou University (FAHZU). Extensive experimental results demonstrate that the proposed method outperforms both unimodal and other multi-modal fusion models. For demonstrating the model interpretability, we also visualize the attention heatmap of pathological images and utilize integrated gradient algorithm to identify important tissue structure, biological pathways and key genes. △ Less

Submitted 11 April, 2024; originally announced April 2024.

The most probable dynamics of receptor-ligand binding on cell membrane

Authors: Xi Chen, Hui Wang, Jinqiao Duan

Abstract: We devise a method for predicting certain receptor-ligand binding behaviors, based on stochastic dynamical modelling. We consider the dynamics of a receptor binding to a ligand on the cell membrane, where the receptor and ligand perform different motions and are thus modeled by stochastic differential equations with Gaussian noise or non-Gaussian noise. We use neural networks based on Onsager-Mach… ▽ More We devise a method for predicting certain receptor-ligand binding behaviors, based on stochastic dynamical modelling. We consider the dynamics of a receptor binding to a ligand on the cell membrane, where the receptor and ligand perform different motions and are thus modeled by stochastic differential equations with Gaussian noise or non-Gaussian noise. We use neural networks based on Onsager-Machlup function to compute the probability $P_1$ of the unbounded receptor diffusing to the cell membrane. Meanwhile, we compute the probability $P_2$ of extracellular ligand arriving at the cell membrane by solving the associated Fokker-Planck equation. Then, we could predict the most probable binding probability by combining $P_1$ and $P_2$. In this way, we conclude with some indication about where the ligand will most probably encounter the receptor, contributing to better understanding of cell's response to external stimuli and communication with other cells. △ Less

Submitted 15 February, 2023; originally announced February 2023.

Comments: 12 figures

An Onsager-Machlup approach to the most probable transition pathway for a genetic regulatory network

Authors: Jianyu Hu, Xiaoli Chen, Jinqiao Duan

Abstract: We investigate a quantitative network of gene expression dynamics describing the competence development in Bacillus subtilis. First, we introduce an Onsager-Machlup approach to quantify the most probable transition pathway for both excitable and bistable dynamics. Then, we apply a machine learning method to calculate the most probable transition pathway via the Euler-Lagrangian equation. Finally,… ▽ More We investigate a quantitative network of gene expression dynamics describing the competence development in Bacillus subtilis. First, we introduce an Onsager-Machlup approach to quantify the most probable transition pathway for both excitable and bistable dynamics. Then, we apply a machine learning method to calculate the most probable transition pathway via the Euler-Lagrangian equation. Finally, we analyze how the noise intensity affects the transition phenomena. △ Less

Submitted 1 March, 2022; originally announced March 2022.

A Logistic-Harvest Model with Allee Effect under Multiplicative Noise

Authors: Almaz Tesfay, Daniel Tesfay, James Brannan, Jinqiao Duan

Abstract: This work is devoted to the study of a stochastic logistic growth model with and without the Allee effect. Such a model describes the evolution of a population under environmental stochastic fluctuations and is in the form of a stochastic differential equation driven by multiplicative Gaussian noise. With the help of the associated Fokker-Planck equation, we analyze the population extinction proba… ▽ More This work is devoted to the study of a stochastic logistic growth model with and without the Allee effect. Such a model describes the evolution of a population under environmental stochastic fluctuations and is in the form of a stochastic differential equation driven by multiplicative Gaussian noise. With the help of the associated Fokker-Planck equation, we analyze the population extinction probability and the probability of reaching a large population size before reaching a small one. We further study the impact of the harvest rate, noise intensity, and the Allee effect on population evolution. The analysis and numerical experiments show that if the noise intensity and harvest rate are small, the population grows exponentially, and upon reaching the carrying capacity, the population size fluctuates around it. In the stochastic logistic-harvest model without the Allee effect, when noise intensity becomes small (or goes to zero), the stationary probability density becomes more acute and its maximum point approaches one. However, for large noise intensity and harvest rate, the population size fluctuates wildly and does not grow exponentially to the carrying capacity. So as far as biological meanings are concerned, we must catch at small values of noise intensity and harvest rate. Finally, we discuss the biological implications of our results. △ Less

Submitted 4 August, 2020; originally announced August 2020.

Comments: 18 pages, 14 figures

MSC Class: 2020 MSC: -Mathematics Subject Classification: 39A50; 45K05; 65N22

Deep learning for cardiac image segmentation: A review

Authors: Chen Chen, Chen Qin, Huaqi Qiu, Giacomo Tarroni, Jinming Duan, Wenjia Bai, Daniel Rueckert

Abstract: Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and… ▽ More Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research. △ Less

Submitted 9 November, 2019; originally announced November 2019.

Comments: Under review

Most probable dynamics of a genetic regulatory network under stable Lévy noise

Authors: Xiaoli Chen, Fengyan Wu, Jinqiao Duan, Jürgen Kurths, Xiaofan Li

Abstract: Numerous studies have demonstrated the important role of noise in the dynamical behaviour of a complex system. The most probable trajectories of nonlinear systems under the influence of Gaussian noise have recently been studied already. However, there has been only a few works that examine how most probable trajectories in the two-dimensional system (MeKS network) are influenced under non-Gaussian… ▽ More Numerous studies have demonstrated the important role of noise in the dynamical behaviour of a complex system. The most probable trajectories of nonlinear systems under the influence of Gaussian noise have recently been studied already. However, there has been only a few works that examine how most probable trajectories in the two-dimensional system (MeKS network) are influenced under non-Gaussian stable Lévy noise. Therefore, we discuss the most probable trajectories of a two-dimensional model depicting the competence behaviour in B. subtilis under the influence of stable Lévy noise. On the basis of the Fokker-Planck equation, we describe the noise-induced most probable trajectories of the MeKS network from the low ComK protein concentration (vegetative state) to the high ComK protein concentration (competence state) under stable Lévy noise. We demonstrate choices of the non-Gaussianity index $α$ and the noise intensity $ε$ which generate the ComK protein escape from the low concentration to the high concentration. We also reveal the optimal combination of both parameters $α$ and $ε$ making the tipping time shortest. Moreover, we find that different initial concentrations around the low ComK protein concentration evolve to a metastable state, and provide the optimal $α$ and $ε$ such that the distance between the deterministic competence state and the metastable state is smallest. △ Less

Submitted 8 December, 2018; originally announced December 2018.

Comments: 21 pages

Most Probable Evolution Trajectories in a Genetic Regulatory System Excited by Stable Lévy Noise

Authors: Xiujun Cheng, Hui Wang, Xiao Wang, Jinqiao Duan, Xiaofan Li

Abstract: We study the most probable trajectories of the concentration evolution for the transcription factor activator in a genetic regulation system, with non-Gaussian stable Lévy noise in the synthesis reaction rate taking into account. We calculate the most probable trajectory by spatially maximizing the probability density of the system path, i.e., the solution of the associated nonlocal Fokker-Planck… ▽ More We study the most probable trajectories of the concentration evolution for the transcription factor activator in a genetic regulation system, with non-Gaussian stable Lévy noise in the synthesis reaction rate taking into account. We calculate the most probable trajectory by spatially maximizing the probability density of the system path, i.e., the solution of the associated nonlocal Fokker-Planck equation. We especially examine those most probable trajectories from low concentration state to high concentration state (i.e., the likely transcription regime) for certain parameters, in order to gain insights into the transcription processes and the tipping time for the transcription likely to occur. This enables us: (i) to visualize the progress of concentration evolution (i.e., observe whether the system enters the transcription regime within a given time period); (ii) to predict or avoid certain transcriptions via selecting specific noise parameters in particular regions in the parameter space. Moreover, we have found some peculiar or counter-intuitive phenomena in this gene model system, including (a) a smaller noise intensity may trigger the transcription process, while a larger noise intensity can not, under the same asymmetric Lévy noise. This phenomenon does not occur in the case of symmetric Lévy noise; (b) the symmetric Lévy motion always induces transition to high concentration, but certain asymmetric Lévy motions do not trigger the switch to transcription. These findings provide insights for further experimental research, in order to achieve or to avoid specific gene transcriptions, with possible relevance for medical advances. △ Less

Submitted 28 January, 2019; v1 submitted 9 October, 2018; originally announced October 2018.

Likelihood for transcriptions in a genetic regulatory system under asymmetric stable Lévy noise

Authors: Hui Wang, Xiujun Cheng, Jinqiao Duan, Jürgen Kurths, Xiaofan Li

Abstract: This work is devoted to investigating the evolution of concentration in a genetic regulation system, when the synthesis reaction rate is under additive and multiplicative asymmetric stable Lévy fluctuations. By focusing on the impact of skewness (i.e., non-symmetry) in the probability distributions of noise, we find that via examining the mean first exit time (MFET) and the first escape probabilit… ▽ More This work is devoted to investigating the evolution of concentration in a genetic regulation system, when the synthesis reaction rate is under additive and multiplicative asymmetric stable Lévy fluctuations. By focusing on the impact of skewness (i.e., non-symmetry) in the probability distributions of noise, we find that via examining the mean first exit time (MFET) and the first escape probability (FEP), the asymmetric fluctuations, interacting with nonlinearity in the system, lead to peculiar likelihood for transcription. This includes, in the additive noise case, realizing higher likelihood of transcription for larger positive skewness (i.e., asymmetry) index $β$, causing a stochastic bifurcation at the non-Gaussianity index value $α=1$ (i.e., it is a separating point or line for the likelihood for transcription), and achieving a turning point at the threshold value $β\approx -0.5$ (i.e., beyond which the likelihood for transcription suddenly reversed for $α$ values). The stochastic bifurcation and turning point phenomena do not occur in the symmetric noise case ($β=0$). While in the multiplicative noise case, non-Gaussianity index value $α=1$ is a separating point or line for both the mean first exit time (MFET) and the first escape probability (FEP). We also investigate the noise enhanced stability phenomenon. Additionally, we are able to specify the regions in the whole parameter space for the asymmetric noise, in which we attain desired likelihood for transcription. We have conducted a series of numerical experiments in `regulating' the likelihood of gene transcription by tuning asymmetric stable Lévy noise indexes. This work offers insights for possible ways of achieving gene regulation in experimental research. △ Less

Submitted 30 January, 2018; v1 submitted 18 May, 2017; originally announced May 2017.

Comments: 27 pages, 19 figures

MSC Class: 92C05 ACM Class: I.7.2

Lévy noise-induced transitions in gene regulatory networks

Authors: Fengyan Wu, Xiaoli Chen, Yayun Zheng, Jinqiao Duan, Jürgen Kurths, Xiaofan Li

Abstract: Important effects of noise on a one-dimensional gene expression model involving a single gene have recently been discussed. However, few works have been devoted to the transition in two-dimensional models which include the interaction of genes. Therefore, we investigate here, a quantitative two-dimensional model (MeKS network) of gene expression dynamics describing the competence development in th… ▽ More Important effects of noise on a one-dimensional gene expression model involving a single gene have recently been discussed. However, few works have been devoted to the transition in two-dimensional models which include the interaction of genes. Therefore, we investigate here, a quantitative two-dimensional model (MeKS network) of gene expression dynamics describing the competence development in the B. subtilis under the influence of Lévy as well as Brownian motions, where noises can do the B. subtilis a favor in nutrient depletion. To analyze the transitions between the vegetative and the competence regions therein, two deterministic quantities, the mean first exit time (MFET) and the first escape probability (FEP) from a microscopic perspective, as well as their averaged versions from a macroscopic perspective, are applied. The relative contribution factor (RCF), the ratio of non-Gaussian and Gaussian noise strengths, is adopted to implement optimal control in these transitions. Schematic representations indicate that there exists an optimum choice that makes the transition occurring at the highest probability. Additionally, we use a geometric concept, the stochastic basin of attraction, to exhibit a pictorial comprehension about the influence of the Lévy motion on the basin stability of the competence state. △ Less

Submitted 10 May, 2017; originally announced May 2017.

Mean Exit Time and Escape Probability for a Tumor Growth System under Non-Gaussian Noise

Authors: Jian Ren, Chujin Li, Ting Gao, Xingye Kan, Jinqiao Duan

Abstract: Effects of non-Gaussian $α-$stable Lévy noise on the Gompertz tumor growth model are quantified by considering the mean exit time and escape probability of the cancer cell density from inside a safe or benign domain. The mean exit time and escape probability problems are formulated in a differential-integral equation with a fractional Laplacian operator. Numerical simulations are conducted to eval… ▽ More Effects of non-Gaussian $α-$stable Lévy noise on the Gompertz tumor growth model are quantified by considering the mean exit time and escape probability of the cancer cell density from inside a safe or benign domain. The mean exit time and escape probability problems are formulated in a differential-integral equation with a fractional Laplacian operator. Numerical simulations are conducted to evaluate how the mean exit time and escape probability vary or bifurcates when $α$ changes. Some bifurcation phenomena are observed and their impacts are discussed. △ Less

Submitted 28 November, 2011; originally announced November 2011.