-
Establishing Deep InfoMax as an effective self-supervised learning methodology in materials informatics
Authors:
Michael Moran,
Vladimir V. Gusev,
Michael W. Gaultois,
Dmytro Antypov,
Matthew J. Rosseinsky
Abstract:
The scarcity of property labels remains a key challenge in materials informatics, whereas materials data without property labels are abundant in comparison. By pretraining supervised property prediction models on self-supervised tasks that depend only on the "intrinsic information" available in any Crystallographic Information File (CIF), there is potential to leverage the large amount of crystal…
▽ More
The scarcity of property labels remains a key challenge in materials informatics, whereas materials data without property labels are abundant in comparison. By pretraining supervised property prediction models on self-supervised tasks that depend only on the "intrinsic information" available in any Crystallographic Information File (CIF), there is potential to leverage the large amount of crystal data without property labels to improve property prediction results on small datasets. We apply Deep InfoMax as a self-supervised machine learning framework for materials informatics that explicitly maximises the mutual information between a point set (or graph) representation of a crystal and a vector representation suitable for downstream learning. This allows the pretraining of supervised models on large materials datasets without the need for property labels and without requiring the model to reconstruct the crystal from a representation vector. We investigate the benefits of Deep InfoMax pretraining implemented on the Site-Net architecture to improve the performance of downstream property prediction models with small amounts (<10^3) of data, a situation relevant to experimentally measured materials property databases. Using a property label masking methodology, where we perform self-supervised learning on larger supervised datasets and then train supervised models on a small subset of the labels, we isolate Deep InfoMax pretraining from the effects of distributional shift. We demonstrate performance improvements in the contexts of representation learning and transfer learning on the tasks of band gap and formation energy prediction. Having established the effectiveness of Deep InfoMax pretraining in a controlled environment, our findings provide a foundation for extending the approach to address practical challenges in materials informatics.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
Site-Net: Using global self-attention and real-space supercells to capture long-range interactions in crystal structures
Authors:
Michael Moran,
Michael W. Gaultois,
Vladimir V. Gusev,
Matthew J. Rosseinsky
Abstract:
Site-Net is a transformer architecture that models the periodic crystal structures of inorganic materials as a labelled point set of atoms and relies entirely on global self-attention and geometric information to guide learning. Site-Net processes standard crystallographic information files to generate a large real-space supercell, and the importance of interactions between all atomic sites is fle…
▽ More
Site-Net is a transformer architecture that models the periodic crystal structures of inorganic materials as a labelled point set of atoms and relies entirely on global self-attention and geometric information to guide learning. Site-Net processes standard crystallographic information files to generate a large real-space supercell, and the importance of interactions between all atomic sites is flexibly learned by the model for the prediction task presented. The attention mechanism is probed to reveal Site-Net can learn long-range interactions in crystal structures, and that specific attention heads become specialized to deal with primarily short- or long-range interactions. We perform a preliminary hyperparameter search and train Site-Net using a single graphics processing unit (GPU), and show Site-Net achieves state-of-the-art performance on a standard band gap regression task.
△ Less
Submitted 16 September, 2022;
originally announced September 2022.
-
Double-Helical Tiled Chain Structure of the Twist-Bend Liquid Crystal phase in CB7CB
Authors:
Michael R. Tuchband,
Min Shuai,
Keri A. Graber,
Dong Chen,
Chenhui Zhu,
Leo Radzihovsky,
Arthur Klittnick,
Lee M. Foley,
Alyssa Scarbrough,
Jan H. Porada,
Mark Moran,
Joseph Yelk,
Dmitry Bedrov,
Eva Korblova,
David M. Walba,
Alexander Hexemer,
Joseph E. Maclennan,
Matthew A. Glaser,
Noel A. Clark
Abstract:
The twist-bend nematic liquid crystal phase is a three-dimensional fluid in which achiral bent molecules spontaneously form an orientationally ordered macroscopically chiral heliconical winding of molecular scale pitch, in absence of positional ordering. Here we characterize the structure of the ground state of the twist-bend phase of the bent dimer CB7CB and its mixtures with 5CB over a wide rang…
▽ More
The twist-bend nematic liquid crystal phase is a three-dimensional fluid in which achiral bent molecules spontaneously form an orientationally ordered macroscopically chiral heliconical winding of molecular scale pitch, in absence of positional ordering. Here we characterize the structure of the ground state of the twist-bend phase of the bent dimer CB7CB and its mixtures with 5CB over a wide range of concentrations and temperatures, showing that the contour length along the molecular direction for a single turn of the helix is approximately equal to 2Ď€Rmol, where Rmol is the radius of bend curvature of a single all-trans CB7CB molecule. This relation emerges from a model which simply relates the macroscopic characteristics of the helical structure, which is mostly biaxial twist and has little bend, to the bent molecular shape. This connection comes about through the presence in the fluid of self-assembled oligomer-like correlations of interlocking molecules, arising from the nanosegregation of rigid and flexible molecular subcomponents, forming a brickwork tiling of pairs of molecular strands into a duplex double-helical chain.
△ Less
Submitted 31 March, 2017;
originally announced March 2017.
-
The twist-bend nematic phase of bent mesogenic dimer CB7CB and its mixtures
Authors:
Michael R. Tuchband,
Min Shuai,
Keri A. Graber,
Dong Chen,
Leo Radzihovsky,
Arthur Klittnick,
Lee Foley,
Alyssa Scarbrough,
Jan H. Porada,
Mark Moran,
Eva Korblova,
David M. Walba,
Matthew A. Glaser,
Joseph E. Maclennan,
Noel A. Clark
Abstract:
Binary mixtures of the twist-bend nematic-forming liquid crystal CB7CB with the prototypical rod-like liquid crystal 5CB exhibit a twist-bend nematic phase with properties similar to those reported for neat CB7CB. The mixtures appear homogeneous, with no micron- or nano-scale segregation evident at any concentration. The linear dependence of the phase transition temperature on concentration indica…
▽ More
Binary mixtures of the twist-bend nematic-forming liquid crystal CB7CB with the prototypical rod-like liquid crystal 5CB exhibit a twist-bend nematic phase with properties similar to those reported for neat CB7CB. The mixtures appear homogeneous, with no micron- or nano-scale segregation evident at any concentration. The linear dependence of the phase transition temperature on concentration indicates that these binary mixtures are nearly ideal. However, a decrease in the viscosity with the addition of 5CB allows the characteristic twist-bend stripe textures to relax into a state of uniform birefringence. We confirm the presence of nanoscale modulations of the molecular orientation in the mixtures by freeze-fracture transmission electron microscopy (FFTEM), further evidence of their twist-bend nature. We devise and implement a statistical approach to quantitatively measure the ground state pitch of the twist-bend phase and its mixtures using FFTEM. The addition of 5CB generally shifts the measured ground-state pitch distributions towards larger pitch. Interestingly, the pitch appears to increase discontinuously by ~10 nm at the 50 wt% concentration of 5CB, indicating that the twist-bend phase undergoes a structural transition at higher 5CB concentrations.
△ Less
Submitted 23 November, 2015;
originally announced November 2015.
