-
NeuralPLexer3: Accurate Biomolecular Complex Structure Prediction with Flow Models
Authors:
Zhuoran Qiao,
Feizhi Ding,
Thomas Dresselhaus,
Mia A. Rosenfeld,
Xiaotian Han,
Owen Howell,
Aniketh Iyengar,
Stephen Opalenski,
Anders S. Christensen,
Sai Krishna Sirumalla,
Frederick R. Manby,
Thomas F. Miller III,
Matthew Welborn
Abstract:
Structure determination is essential to a mechanistic understanding of diseases and the development of novel therapeutics. Machine-learning-based structure prediction methods have made significant advancements by computationally predicting protein and bioassembly structures from sequences and molecular topology alone. Despite substantial progress in the field, challenges remain to deliver structur…
▽ More
Structure determination is essential to a mechanistic understanding of diseases and the development of novel therapeutics. Machine-learning-based structure prediction methods have made significant advancements by computationally predicting protein and bioassembly structures from sequences and molecular topology alone. Despite substantial progress in the field, challenges remain to deliver structure prediction models to real-world drug discovery. Here, we present NeuralPLexer3 -- a physics-inspired flow-based generative model that achieves state-of-the-art prediction accuracy on key biomolecular interaction types and improves training and sampling efficiency compared to its predecessors and alternative methodologies. Examined through newly developed benchmarking strategies, NeuralPLexer3 excels in vital areas that are crucial to structure-based drug design, such as physical validity and ligand-induced conformational changes.
△ Less
Submitted 18 December, 2024; v1 submitted 14 December, 2024;
originally announced December 2024.
-
OrbNet Denali: A machine learning potential for biological and organic chemistry with semi-empirical cost and DFT accuracy
Authors:
Anders S. Christensen,
Sai Krishna Sirumalla,
Zhuoran Qiao,
Michael B. O'Connor,
Daniel G. A. Smith,
Feizhi Ding,
Peter J. Bygrave,
Animashree Anandkumar,
Matthew Welborn,
Frederick R. Manby,
Thomas F. Miller III
Abstract:
We present OrbNet Denali, a machine learning model for electronic structure that is designed as a drop-in replacement for ground-state density functional theory (DFT) energy calculations. The model is a message-passing neural network that uses symmetry-adapted atomic orbital features from a low-cost quantum calculation to predict the energy of a molecule. OrbNet Denali is trained on a vast dataset…
▽ More
We present OrbNet Denali, a machine learning model for electronic structure that is designed as a drop-in replacement for ground-state density functional theory (DFT) energy calculations. The model is a message-passing neural network that uses symmetry-adapted atomic orbital features from a low-cost quantum calculation to predict the energy of a molecule. OrbNet Denali is trained on a vast dataset of 2.3 million DFT calculations on molecules and geometries. This dataset covers the most common elements in bio- and organic chemistry (H, Li, B, C, N, O, F, Na, Mg, Si, P, S, Cl, K, Ca, Br, I) as well as charged molecules. OrbNet Denali is demonstrated on several well-established benchmark datasets, and we find that it provides accuracy that is on par with modern DFT methods while offering a speedup of up to three orders of magnitude. For the GMTKN55 benchmark set, OrbNet Denali achieves WTMAD-1 and WTMAD-2 scores of 7.19 and 9.84, on par with modern DFT functionals. For several GMTKN55 subsets, which contain chemical problems that are not present in the training set, OrbNet Denali produces a mean absolute error comparable to those of DFT methods. For the Hutchison conformers benchmark set, OrbNet Denali has a median correlation coefficient of R^2=0.90 compared to the reference DLPNO-CCSD(T) calculation, and R^2=0.97 compared to the method used to generate the training data (wB97X-D3/def2-TZVP), exceeding the performance of any other method with a similar cost. Similarly, the model reaches chemical accuracy for non-covalent interactions in the S66x10 dataset. For torsional profiles, OrbNet Denali reproduces the torsion profiles of wB97X-D3/def2-TZVP with an average MAE of 0.12 kcal/mol for the potential energy surfaces of the diverse fragments in the TorsionNet500 dataset.
△ Less
Submitted 2 July, 2021; v1 submitted 1 July, 2021;
originally announced July 2021.
-
Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry
Authors:
Zhuoran Qiao,
Anders S. Christensen,
Matthew Welborn,
Frederick R. Manby,
Anima Anandkumar,
Thomas F. Miller III
Abstract:
Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials. By developing a physics-inspired equivariant neural network, we introduce a method to learn molecular representations based on the electronic interactions among atomic orbitals. Our method, OrbNet-Equi, leverages efficient tight-binding simul…
▽ More
Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials. By developing a physics-inspired equivariant neural network, we introduce a method to learn molecular representations based on the electronic interactions among atomic orbitals. Our method, OrbNet-Equi, leverages efficient tight-binding simulations and learned mappings to recover high fidelity quantum chemical properties. OrbNet-Equi models a wide spectrum of target properties with an accuracy consistently better than standard machine learning methods and a speed orders of magnitude greater than density functional theory. Despite only using training samples collected from readily available small-molecule libraries, OrbNet-Equi outperforms traditional methods on comprehensive downstream benchmarks that encompass diverse main-group chemical processes. Our method also describes interactions in challenging charge-transfer complexes and open-shell systems. We anticipate that the strategy presented here will help to expand opportunities for studies in chemistry and materials science, where the acquisition of experimental or reference training data is costly.
△ Less
Submitted 1 April, 2022; v1 submitted 30 May, 2021;
originally announced May 2021.
-
Multi-task learning for electronic structure to predict and explore molecular potential energy surfaces
Authors:
Zhuoran Qiao,
Feizhi Ding,
Matthew Welborn,
Peter J. Bygrave,
Daniel G. A. Smith,
Animashree Anandkumar,
Frederick R. Manby,
Thomas F. Miller III
Abstract:
We refine the OrbNet model to accurately predict energy, forces, and other response properties for molecules using a graph neural-network architecture based on features from low-cost approximated quantum operators in the symmetry-adapted atomic orbital basis. The model is end-to-end differentiable due to the derivation of analytic gradients for all electronic structure terms, and is shown to be tr…
▽ More
We refine the OrbNet model to accurately predict energy, forces, and other response properties for molecules using a graph neural-network architecture based on features from low-cost approximated quantum operators in the symmetry-adapted atomic orbital basis. The model is end-to-end differentiable due to the derivation of analytic gradients for all electronic structure terms, and is shown to be transferable across chemical space due to the use of domain-specific features. The learning efficiency is improved by incorporating physically motivated constraints on the electronic structure through multi-task learning. The model outperforms existing methods on energy prediction tasks for the QM9 dataset and for molecular geometry optimizations on conformer datasets, at a computational cost that is thousand-fold or more reduced compared to conventional quantum-chemistry calculations (such as density functional theory) that offer similar accuracy.
△ Less
Submitted 1 December, 2020; v1 submitted 5 November, 2020;
originally announced November 2020.
-
OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features
Authors:
Zhuoran Qiao,
Matthew Welborn,
Animashree Anandkumar,
Frederick R. Manby,
Thomas F. Miller III
Abstract:
We introduce a machine learning method in which energy solutions from the Schrodinger equation are predicted using symmetry adapted atomic orbitals features and a graph neural-network architecture. \textsc{OrbNet} is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that…
▽ More
We introduce a machine learning method in which energy solutions from the Schrodinger equation are predicted using symmetry adapted atomic orbitals features and a graph neural-network architecture. \textsc{OrbNet} is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that are obtained from semi-empirical electronic structure calculations. For applications to datasets of drug-like molecules, including QM7b-T, QM9, GDB-13-T, DrugBank, and the conformer benchmark dataset of Folmsbee and Hutchison, \textsc{OrbNet} predicts energies within chemical accuracy of DFT at a computational cost that is thousand-fold or more reduced.
△ Less
Submitted 18 January, 2022; v1 submitted 15 July, 2020;
originally announced July 2020.
-
Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning
Authors:
Lixue Cheng,
Nikola B. Kovachki,
Matthew Welborn,
Thomas F. Miller III
Abstract:
Machine learning (ML) in the representation of molecular-orbital-based (MOB) features has been shown to be an accurate and transferable approach to the prediction of post-Hartree-Fock correlation energies. Previous applications of MOB-ML employed Gaussian Process Regression (GPR), which provides good prediction accuracy with small training sets; however, the cost of GPR training scales cubically w…
▽ More
Machine learning (ML) in the representation of molecular-orbital-based (MOB) features has been shown to be an accurate and transferable approach to the prediction of post-Hartree-Fock correlation energies. Previous applications of MOB-ML employed Gaussian Process Regression (GPR), which provides good prediction accuracy with small training sets; however, the cost of GPR training scales cubically with the amount of data and becomes a computational bottleneck for large training sets. In the current work, we address this problem by introducing a clustering/regression/classification implementation of MOB-ML. In a first step, regression clustering (RC) is used to partition the training data to best fit an ensemble of linear regression (LR) models; in a second step, each cluster is regressed independently, using either LR or GPR; and in a third step, a random forest classifier (RFC) is trained for the prediction of cluster assignments based on MOB feature values. Upon inspection, RC is found to recapitulate chemically intuitive groupings of the frontier molecular orbitals, and the combined RC/LR/RFC and RC/GPR/RFC implementations of MOB-ML are found to provide good prediction accuracy with greatly reduced wall-clock training times. For a dataset of thermalized geometries of 7211 organic molecules of up to seven heavy atoms, both implementations reach chemical accuracy (1 kcal/mol error) with only 300 training molecules, while providing 35000-fold and 4500-fold reductions in the wall-clock training time, respectively, compared to MOB-ML without clustering. The resulting models are also demonstrated to retain transferability for the prediction of large-molecule energies with only small-molecule training data. Finally, it is shown that capping the number of training datapoints per cluster leads to further improvements in prediction accuracy with negligible increases in wall-clock training time.
△ Less
Submitted 23 October, 2019; v1 submitted 4 September, 2019;
originally announced September 2019.
-
A Universal Density Matrix Functional from Molecular Orbital-Based Machine Learning: Transferability across Organic Molecules
Authors:
Lixue Cheng,
Matthew Welborn,
Anders S. Christensen,
Thomas F. Miller III
Abstract:
We address the degree to which machine learning can be used to accurately and transferably predict post-Hartree-Fock correlation energies. Refined strategies for feature design and selection are presented, and the molecular-orbital-based machine learning (MOB-ML) method is applied to several test systems. Strikingly, for the MP2, CCSD, and CCSD(T) levels of theory, it is shown that the thermally a…
▽ More
We address the degree to which machine learning can be used to accurately and transferably predict post-Hartree-Fock correlation energies. Refined strategies for feature design and selection are presented, and the molecular-orbital-based machine learning (MOB-ML) method is applied to several test systems. Strikingly, for the MP2, CCSD, and CCSD(T) levels of theory, it is shown that the thermally accessible (350 K) potential energy surface for a single water molecule can be described to within 1 millihartree using a model that is trained from only a single reference calculation at a randomized geometry. To explore the breadth of chemical diversity that can be described, MOB-ML is also applied to a new dataset of thermalized (350 K) geometries of 7211 organic models with up to seven heavy atoms. In comparison with the previously reported $Δ$-ML method, MOB-ML is shown to reach chemical accuracy with three-fold fewer training geometries. Finally, a transferability test in which models trained for seven-heavy-atom systems are used to predict energies for thirteen-heavy-atom systems reveals that MOB-ML reaches chemical accuracy with 36-fold fewer training calculations than $Δ$-ML (140 versus 5000 training calculations).
△ Less
Submitted 4 April, 2019; v1 submitted 10 January, 2019;
originally announced January 2019.
-
Even-handed subsystem selection in projection-based embedding
Authors:
Matthew Welborn,
Frederick R. Manby,
Thomas F. Miller III
Abstract:
Projection-based embedding offers a simple framework for embedding correlated wavefunction methods in density functional theory. Partitioning between the correlated wavefunction and density functional subsystems is performed in the space of localized molecular orbitals. However, during a large geometry change--such as a chemical reaction--the nature of these localized molecular orbitals, as well a…
▽ More
Projection-based embedding offers a simple framework for embedding correlated wavefunction methods in density functional theory. Partitioning between the correlated wavefunction and density functional subsystems is performed in the space of localized molecular orbitals. However, during a large geometry change--such as a chemical reaction--the nature of these localized molecular orbitals, as well as their partitioning into the two subsystems, can change dramatically. This can lead to unphysical cusps and even discontinuities in the potential energy surface. In this work, we present an even-handed framework for localized orbital partitioning that ensures consistent subsystems across a set of molecular geometries. We illustrate this problem and the even-handed solution with a simple example of an SN2 reaction. Applications to a nitrogen umbrella flip in a cobalt-based CO2 reduction catalyst and to the binding of CO to Cu clusters are presented. In both cases, we find that even-handed partitioning enables chemically accurate embedding with modestly-sized embedded regions for systems in which previous partitioning strategies are problematic.
△ Less
Submitted 9 September, 2018;
originally announced September 2018.
-
Incremental Embedding: A Density Matrix Embedding Scheme for Molecules
Authors:
Hong-Zhou Ye,
Matthew Welborn,
Nathan D. Ricke,
Troy Van Voorhis
Abstract:
The idea of using fragment embedding to circumvent the high computational scaling of accurate electronic structure methods while retaining high accuracy has been a long-standing goal for quantum chemists. Traditional fragment embedding methods mainly focus on systems composed of weakly correlated parts and are insufficient when division across chemical bonds is unavoidable. Recently, density matri…
▽ More
The idea of using fragment embedding to circumvent the high computational scaling of accurate electronic structure methods while retaining high accuracy has been a long-standing goal for quantum chemists. Traditional fragment embedding methods mainly focus on systems composed of weakly correlated parts and are insufficient when division across chemical bonds is unavoidable. Recently, density matrix embedding theory (DMET) and other methods based on the Schmidt decomposition have emerged as a fresh approach to this problem. Despite their success on model systems, these methods can prove difficult for realistic systems because they rely on either a rigid, non-overlapping partition of the system or a specification of some special sites (i.e. `edge' and `center' sites), neither of which is well-defined in general for real molecules. In this work, we present a new Schmidt decomposition-based embedding scheme called Incremental Embedding that allows the combination of arbitrary overlapping fragments without the knowledge of edge sites. This method forms a convergent hierarchy in the sense that higher accuracy can be obtained by using fragments involving more sites. The computational scaling for the first few levels is lower than that of most correlated wave function methods. We present results for several small molecules in atom-centered Gaussian basis sets and demonstrate that Incremental Embedding converges quickly with fragment size and recovers most static correlation in small basis sets even when truncated at the second lowest level.
△ Less
Submitted 23 July, 2018;
originally announced July 2018.
-
Transferability in Machine Learning for Electronic Structure via the Molecular Orbital Basis
Authors:
Matthew Welborn,
Lixue Cheng,
Thomas F. Miller III
Abstract:
We present a machine learning (ML) method for predicting electronic structure correlation energies using Hartree-Fock input.The total correlation energy is expressed in terms of individual and pair contributions from occupied molecular orbitals, and Gaussian process regression is used to predict these contributions from a feature set that is based on molecular orbital properties, such as Fock, Cou…
▽ More
We present a machine learning (ML) method for predicting electronic structure correlation energies using Hartree-Fock input.The total correlation energy is expressed in terms of individual and pair contributions from occupied molecular orbitals, and Gaussian process regression is used to predict these contributions from a feature set that is based on molecular orbital properties, such as Fock, Coulomb, and exchange matrix elements. With the aim of maximizing transferability across chemical systems and compactness of the feature set, we avoid the usual specification of ML features in terms of atom- or geometry-specific information, such atom/element-types, bond-types, or local molecular structure. ML predictions of MP2 and CCSD energies are presented for a range of systems, demonstrating that the method maintains accuracy while providing transferability both within and across chemical families; this includes predictions for molecules with atom-types and elements that are not included in the training set. The method holds promise both in its current form and as a proof-of-principle for the use of ML in the design of generalized density-matrix functionals.
△ Less
Submitted 26 July, 2018; v1 submitted 31 May, 2018;
originally announced June 2018.
-
Error analysis of free probability approximations to the density of states of disordered systems
Authors:
Jiahao Chen,
Eric Hontz,
Jeremy Moix,
Matthew Welborn,
Troy Van Voorhis,
Alberto Suárez,
Ramis Movassagh,
Alan Edelman
Abstract:
Theoretical studies of localization, anomalous diffusion and ergodicity breaking require solving the electronic structure of disordered systems. We use free probability to approximate the ensemble- averaged density of states without exact diagonalization. We present an error analysis that quantifies the accuracy using a generalized moment expansion, allowing us to distinguish between different app…
▽ More
Theoretical studies of localization, anomalous diffusion and ergodicity breaking require solving the electronic structure of disordered systems. We use free probability to approximate the ensemble- averaged density of states without exact diagonalization. We present an error analysis that quantifies the accuracy using a generalized moment expansion, allowing us to distinguish between different approximations. We identify an approximation that is accurate to the eighth moment across all noise strengths, and contrast this with the perturbation theory and isotropic entanglement theory.
△ Less
Submitted 27 February, 2012;
originally announced February 2012.
