Deep Learning for Automated Experimentation in Scanning Transmission Electron Microscopy
Authors:
Sergei V. Kalinin,
Debangshu Mukherjee,
Kevin M. Roccapriore,
Ben Blaiszik,
Ayana Ghosh,
Maxim A. Ziatdinov,
A. Al-Najjar,
Christina Doty,
Sarah Akers,
Nageswara S. Rao,
Joshua C. Agar,
Steven R. Spurgeon
Abstract:
Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centered experiment workflow design and…
▽ More
Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centered experiment workflow design and optimization. Here, we discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects, the requirements for the edge operation, local and cloud data storage, and theory in the loop operations. Specifically, we discuss the relative contributions of human scientists and ML agents in the ideation, orchestration, and execution of experimental workflows and the need to develop universal hyper languages that can apply across multiple platforms. These considerations will collectively inform the operationalization of ML in next-generation experimentation.
△ Less
Submitted 4 April, 2023;
originally announced April 2023.
A roadmap for edge computing enabled automated multidimensional transmission electron microscopy
Authors:
Debangshu Mukherjee,
Kevin M. Roccapriore,
Anees Al-Najjar,
Ayana Ghosh,
Jacob D. Hinkle,
Andrew R. Lupini,
Rama K. Vasudevan,
Sergei V. Kalinin,
Olga S. Ovchinnikova,
Maxim A. Ziatdinov,
Nageswara S. Rao
Abstract:
The advent of modern, high-speed electron detectors has made the collection of multidimensional hyperspectral transmission electron microscopy datasets, such as 4D-STEM, a routine. However, many microscopists find such experiments daunting since such datasets' analysis, collection, long-term storage, and networking remain challenging. Some common issues are the large and unwieldy size of the said…
▽ More
The advent of modern, high-speed electron detectors has made the collection of multidimensional hyperspectral transmission electron microscopy datasets, such as 4D-STEM, a routine. However, many microscopists find such experiments daunting since such datasets' analysis, collection, long-term storage, and networking remain challenging. Some common issues are the large and unwieldy size of the said datasets, often running into several gigabytes, non-standardized data analysis routines, and a lack of clarity about the computing and network resources needed to utilize the electron microscope fully. However, the existing computing and networking bottlenecks introduce significant penalties in each step of these experiments, and thus, real-time analysis-driven automated experimentation for multidimensional TEM is exceptionally challenging. One solution is integrating microscopy with edge computing, where moderately powerful computational hardware performs the preliminary analysis before handing off the heavier computation to HPC systems. In this perspective, we trace the roots of computation in modern electron microscopy, demonstrate deep learning experiments running on an edge system, and discuss the networking requirements for tying together microscopes, edge computers, and HPC systems.
△ Less
Submitted 5 October, 2022;
originally announced October 2022.
