-
NeuroTree: Hierarchical Functional Brain Pathway Decoding for Mental Health Disorders
Authors:
Jun-En Ding,
Dongsheng Luo,
Anna Zilverstand,
Kaustubh Kulkarni,
Feng Liu
Abstract:
Mental disorders are among the most widespread diseases globally. Analyzing functional brain networks through functional magnetic resonance imaging (fMRI) is crucial for understanding mental disorder behaviors. Although existing fMRI-based graph neural networks (GNNs) have demonstrated significant potential in brain network feature extraction, they often fail to characterize complex relationships…
▽ More
Mental disorders are among the most widespread diseases globally. Analyzing functional brain networks through functional magnetic resonance imaging (fMRI) is crucial for understanding mental disorder behaviors. Although existing fMRI-based graph neural networks (GNNs) have demonstrated significant potential in brain network feature extraction, they often fail to characterize complex relationships between brain regions and demographic information in mental disorders. To overcome these limitations, we propose a learnable NeuroTree framework that integrates a k-hop AGE-GCN with neural ordinary differential equations (ODEs) and contrastive masked functional connectivity (CMFC) to enhance similarities and dissimilarities of brain region distance. Furthermore, NeuroTree effectively decodes fMRI network features into tree structures, which improves the capture of high-order brain regional pathway features and enables the identification of hierarchical neural behavioral patterns essential for understanding disease-related brain subnetworks. Our empirical evaluations demonstrate that NeuroTree achieves state-of-the-art performance across two distinct mental disorder datasets. It provides valuable insights into age-related deterioration patterns, elucidating their underlying neural mechanisms.
△ Less
Submitted 23 May, 2025; v1 submitted 25 February, 2025;
originally announced February 2025.
-
Vision Language Models Know Law of Conservation without Understanding More-or-Less
Authors:
Dezhi Luo,
Haiyun Lyu,
Qingying Gao,
Haoran Sun,
Yijiang Li,
Hokin Deng
Abstract:
Understanding law of conservation is a critical milestone in human cognitive development considered to be supported by the apprehension of quantitative concepts and the reversibility of operations. To assess whether this critical component of human intelligence has emerged in Vision Language Models, we have curated the ConserveBench, a battery of 365 cognitive experiments across four dimensions of…
▽ More
Understanding law of conservation is a critical milestone in human cognitive development considered to be supported by the apprehension of quantitative concepts and the reversibility of operations. To assess whether this critical component of human intelligence has emerged in Vision Language Models, we have curated the ConserveBench, a battery of 365 cognitive experiments across four dimensions of physical quantities: volume, solid quantity, length, and number. The former two involve transformational tasks which require reversibility understanding. The latter two involve non-transformational tasks which assess quantity understanding. Surprisingly, we find that while Vision Language Models are generally good at transformational tasks, they tend to fail at non-transformational tasks. There is a dissociation between understanding the reversibility of operations and understanding the concept of quantity, which both are believed to be the cornerstones of understanding law of conservation in humans. $\href{https://growing-ai-like-a-child.github.io/}{Website}$
△ Less
Submitted 13 April, 2025; v1 submitted 30 September, 2024;
originally announced October 2024.
-
Probing Mechanical Reasoning in Large Vision Language Models
Authors:
Haoran Sun,
Qingying Gao,
Haiyun Lyu,
Dezhi Luo,
Yijiang Li,
Hokin Deng
Abstract:
Mechanical reasoning is a hallmark of human intelligence, defined by its ubiquitous yet irreplaceable role in human activities ranging from routine tasks to civil engineering. Embedding machines with mechanical reasoning is therefore an important step towards building human-level artificial intelligence. Here, we leveraged 155 cognitive experiments to test the understanding of system stability, ge…
▽ More
Mechanical reasoning is a hallmark of human intelligence, defined by its ubiquitous yet irreplaceable role in human activities ranging from routine tasks to civil engineering. Embedding machines with mechanical reasoning is therefore an important step towards building human-level artificial intelligence. Here, we leveraged 155 cognitive experiments to test the understanding of system stability, gears and pulley systems, leverage principle, inertia and motion, and fluid mechanics in 26 Vision Language Models (VLMs). Results indicate that VLMs consistently perform worse than humans on all domains, while demonstrate significant difficulty in reasoning about gear systems and fluid mechanics. Notably, their performance on these tasks do not improve as number of parameters increase, suggesting that current attention-based architecture may fail to grasp certain underlying mechanisms required for mechanical reasoning, particularly those pertaining to mental simulations.
△ Less
Submitted 13 April, 2025; v1 submitted 30 September, 2024;
originally announced October 2024.
-
DiffFit: Visually-Guided Differentiable Fitting of Molecule Structures to a Cryo-EM Map
Authors:
Deng Luo,
Zainab Alsuwaykit,
Dawar Khan,
Ondřej Strnad,
Tobias Isenberg,
Ivan Viola
Abstract:
We introduce DiffFit, a differentiable algorithm for fitting protein atomistic structures into an experimental reconstructed Cryo-Electron Microscopy (cryo-EM) volume map. In structural biology, this process is necessary to semi-automatically composite large mesoscale models of complex protein assemblies and complete cellular structures that are based on measured cryo-EM data. The current approach…
▽ More
We introduce DiffFit, a differentiable algorithm for fitting protein atomistic structures into an experimental reconstructed Cryo-Electron Microscopy (cryo-EM) volume map. In structural biology, this process is necessary to semi-automatically composite large mesoscale models of complex protein assemblies and complete cellular structures that are based on measured cryo-EM data. The current approaches require manual fitting in three dimensions to start, resulting in approximately aligned structures followed by an automated fine-tuning of the alignment. The DiffFit approach enables domain scientists to fit new structures automatically and visualize the results for inspection and interactive revision. The fitting begins with differentiable three-dimensional (3D) rigid transformations of the protein atom coordinates followed by sampling the density values at the atom coordinates from the target cryo-EM volume. To ensure a meaningful correlation between the sampled densities and the protein structure, we proposed a novel loss function based on a multi-resolution volume-array approach and the exploitation of the negative space. This loss function serves as a critical metric for assessing the fitting quality, ensuring the fitting accuracy and an improved visualization of the results. We assessed the placement quality of DiffFit with several large, realistic datasets and found it to be superior to that of previous methods. We further evaluated our method in two use cases: automating the integration of known composite structures into larger protein complexes and facilitating the fitting of predicted protein domains into volume densities to aid researchers in identifying unknown proteins. We implemented our algorithm as an open-source plugin (github.com/nanovis/DiffFit) in ChimeraX, a leading visualization software in the field. All supplemental materials are available at osf.io/5tx4q.
△ Less
Submitted 8 January, 2025; v1 submitted 3 April, 2024;
originally announced April 2024.
-
SynopSet: Multiscale Visual Abstraction Set for Explanatory Analysis of DNA Nanotechnology Simulations
Authors:
Deng Luo,
Alexandre Kouyoumdjian,
Ondřej Strnad,
Haichao Miao,
Ivan Barišić,
Ivan Viola
Abstract:
We propose a new abstraction set (SynopSet) that has a continuum of visual representations for the explanatory analysis of molecular dynamics simulations (MDS) in the DNA nanotechnology domain. By re-purposing the commonly used progress bar and designing novel visuals, as well as transforming the data from the domain format to a format that better fits the newly designed visuals, we compose this n…
▽ More
We propose a new abstraction set (SynopSet) that has a continuum of visual representations for the explanatory analysis of molecular dynamics simulations (MDS) in the DNA nanotechnology domain. By re-purposing the commonly used progress bar and designing novel visuals, as well as transforming the data from the domain format to a format that better fits the newly designed visuals, we compose this new set of representations. This set is also designed to be capable of showing all spatial and temporal details, and all structural complexity, or abstracting these to various degrees, enabling both the slow playback of the simulation for detailed examinations or very fast playback for an overview that helps to efficiently identify events of interest, as well as several intermediate levels between these two extremes. For any pair of successive representations, we demonstrate smooth, continuous transitions, enabling users to keep track of relevant information from one representation to the next. By providing multiple representations suited to different temporal resolutions and connected by smooth transitions, we enable time-efficient simulation analysis, giving users the opportunity to examine and present important phases in great detail, or leverage abstract representations to go over uneventful phases much faster. Domain experts can thus gain actionable insight about their simulations and communicate it in a much shorter time. Further, the novel representations are more intuitive and also enable researchers unfamiliar with MDS analysis graphs to better understand the simulation results. We assessed the effectiveness of SynopSet on 12 DNA nanostructure simulations together with a domain expert. We have also shown that our set of representations can be systematically located in a visualization space, dubbed SynopSpace.
△ Less
Submitted 18 April, 2022;
originally announced May 2022.
-
Modeling in the Time of COVID-19: Statistical and Rule-based Mesoscale Models
Authors:
Ngan Nguyen,
Ondrej Strnad,
Tobias Klein,
Deng Luo,
Ruwayda Alharbi,
Peter Wonka,
Martina Maritan,
Peter Mindek,
Ludovic Autin,
David S. Goodsell,
Ivan Viola
Abstract:
We present a new technique for rapid modeling and construction of scientifically accurate mesoscale biological models. Resulting 3D models are based on few 2D microscopy scans and the latest knowledge about the biological entity represented as a set of geometric relationships. Our new technique is based on statistical and rule-based modeling approaches that are rapid to author, fast to construct,…
▽ More
We present a new technique for rapid modeling and construction of scientifically accurate mesoscale biological models. Resulting 3D models are based on few 2D microscopy scans and the latest knowledge about the biological entity represented as a set of geometric relationships. Our new technique is based on statistical and rule-based modeling approaches that are rapid to author, fast to construct, and easy to revise. From a few 2D microscopy scans, we learn statistical properties of various structural aspects, such as the outer membrane shape, spatial properties and distribution characteristics of the macromolecular elements on the membrane. This information is utilized in 3D model construction. Once all imaging evidence is incorporated in the model, additional information can be incorporated by interactively defining rules that spatially characterize the rest of the biological entity, such as mutual interactions among macromolecules, their distances and orientations to other structures. These rules are defined through an intuitive 3D interactive visualization and modeling feedback loop. We demonstrate the utility of our approach on a use case of the modeling procedure of the SARS-CoV-2 virus particle ultrastructure. Its first complete atomistic model, which we present here, can steer biological research to new promising directions in fighting spread of the virus.
△ Less
Submitted 1 May, 2020;
originally announced May 2020.
-
Imaging linear and circular polarization features in leaves with complete Mueller matrix polarimetry
Authors:
C. H. Lucas Patty,
David A. Luo,
Frans Snik,
Freek Ariese,
Wybren Jan Buma,
Inge Loes ten Kate,
Rob J. M. van Spanning,
William B. Sparks,
Thomas A. Germer,
Győző Garab,
Michael W. Kudenov
Abstract:
Spectropolarimetry of intact plant leaves allows to probe the molecular architecture of vegetation photosynthesis in a non-invasive and non-destructive way and, as such, can offer a wealth of physiological information. In addition to the molecular signals due to the photosynthetic machinery, the cell structure and its arrangement within a leaf can create and modify polarization signals. Using Muel…
▽ More
Spectropolarimetry of intact plant leaves allows to probe the molecular architecture of vegetation photosynthesis in a non-invasive and non-destructive way and, as such, can offer a wealth of physiological information. In addition to the molecular signals due to the photosynthetic machinery, the cell structure and its arrangement within a leaf can create and modify polarization signals. Using Mueller matrix polarimetry with rotating retarder modulation, we have visualized spatial variations in polarization in transmission around the chlorophyll a absorbance band from 650 nm to 710 nm. We show linear and circular polarization measurements of maple leaves and cultivated maize leaves and discuss the corresponding Mueller matrices and the Mueller matrix decompositions, which show distinct features in diattenuation, polarizance, retardance and depolarization. Importantly, while normal leaf tissue shows a typical split signal with both a negative and a positive peak in the induced fractional circular polarization and circular dichroism, the signals close to the veins only display a negative band. The results are similar to the negative band as reported earlier for single macrodomains. We discuss the possible role of the chloroplast orientation around the veins as a cause of this phenomenon. Systematic artefacts are ruled out as three independent measurements by different instruments gave similar results. These results provide better insight into circular polarization measurements on whole leaves and options for vegetation remote sensing using circular polarization.
△ Less
Submitted 12 March, 2018;
originally announced March 2018.
