Showing 1–14 of 14 results for author: Steinbach, P

Virtual Pulse Reconstruction Diagnostic for Single-Shot Measurement of Free Electron Laser Radiation Power

Authors: Till Korten, Vladimir Rybnikov, Peter Steinbach, Najmeh Mirian

Abstract: Accurate characterization of radiation pulse profiles is crucial for optimizing beam quality and enhancing experimental outcomes in Free Electron Laser (FEL) research. In this paper, we present a novel approach that employs machine learning techniques for real-time virtual diagnostics of FEL radiation pulses. Our advanced artificial intelligence (AI)-based diagnostic tool utilizes longitudinal pha… ▽ More Accurate characterization of radiation pulse profiles is crucial for optimizing beam quality and enhancing experimental outcomes in Free Electron Laser (FEL) research. In this paper, we present a novel approach that employs machine learning techniques for real-time virtual diagnostics of FEL radiation pulses. Our advanced artificial intelligence (AI)-based diagnostic tool utilizes longitudinal phase space data obtained from the X-band transverse deflecting structure to reconstruct the temporal profile of FEL pulses in real time. Unlike traditional single-shot methods, this AI-driven solution provides a non-invasive, highly efficient alternative for pulse characterization. By leveraging state-of-the-art machine learning models, our method facilitates precise single-shot measurements of FEL pulse power, offering significant advantages for FEL science research. This work outlines the conceptual framework, methodology, and validation results of our virtual diagnostic tool, demonstrating its potential to significantly impact FEL research. △ Less

Submitted 26 November, 2024; originally announced November 2024.

Comments: 9 pages , 5 figures

sbi reloaded: a toolkit for simulation-based inference workflows

Authors: Jan Boelts, Michael Deistler, Manuel Gloeckler, Álvaro Tejero-Cantero, Jan-Matthis Lueckmann, Guy Moss, Peter Steinbach, Thomas Moreau, Fabio Muratore, Julia Linhart, Conor Durkan, Julius Vetter, Benjamin Kurt Miller, Maternus Herold, Abolfazl Ziaeemehr, Matthijs Pals, Theo Gruner, Sebastian Bischoff, Nastya Krouglova, Richard Gao, Janne K. Lappalainen, Bálint Mucsányi, Felix Pei, Auguste Schulz, Zinovia Stefanidi , et al. (8 additional authors not shown)

Abstract: Scientists and engineers use simulators to model empirically observed phenomena. However, tuning the parameters of a simulator to ensure its outputs match observed data presents a significant challenge. Simulation-based inference (SBI) addresses this by enabling Bayesian inference for simulators, identifying parameters that match observed data and align with prior knowledge. Unlike traditional Bay… ▽ More Scientists and engineers use simulators to model empirically observed phenomena. However, tuning the parameters of a simulator to ensure its outputs match observed data presents a significant challenge. Simulation-based inference (SBI) addresses this by enabling Bayesian inference for simulators, identifying parameters that match observed data and align with prior knowledge. Unlike traditional Bayesian inference, SBI only needs access to simulations from the model and does not require evaluations of the likelihood-function. In addition, SBI algorithms do not require gradients through the simulator, allow for massive parallelization of simulations, and can perform inference for different observations without further simulations or training, thereby amortizing inference. Over the past years, we have developed, maintained, and extended $\texttt{sbi}$, a PyTorch-based package that implements Bayesian SBI algorithms based on neural networks. The $\texttt{sbi}$ toolkit implements a wide range of inference methods, neural network architectures, sampling methods, and diagnostic tools. In addition, it provides well-tested default settings but also offers flexibility to fully customize every step of the simulation-based inference workflow. Taken together, the $\texttt{sbi}$ toolkit enables scientists and engineers to apply state-of-the-art SBI methods to black-box simulators, opening up new possibilities for aligning simulations with empirically observed data. △ Less

Submitted 26 November, 2024; originally announced November 2024.

Testing Uncertainty of Large Language Models for Physics Knowledge and Reasoning

Authors: Elizaveta Reganova, Peter Steinbach

Abstract: Large Language Models (LLMs) have gained significant popularity in recent years for their ability to answer questions in various fields. However, these models have a tendency to "hallucinate" their responses, making it challenging to evaluate their performance. A major challenge is determining how to assess the certainty of a model's predictions and how it correlates with accuracy. In this work, w… ▽ More Large Language Models (LLMs) have gained significant popularity in recent years for their ability to answer questions in various fields. However, these models have a tendency to "hallucinate" their responses, making it challenging to evaluate their performance. A major challenge is determining how to assess the certainty of a model's predictions and how it correlates with accuracy. In this work, we introduce an analysis for evaluating the performance of popular open-source LLMs, as well as gpt-3.5 Turbo, on multiple choice physics questionnaires. We focus on the relationship between answer accuracy and variability in topics related to physics. Our findings suggest that most models provide accurate replies in cases where they are certain, but this is by far not a general behavior. The relationship between accuracy and uncertainty exposes a broad horizontal bell-shaped distribution. We report how the asymmetry between accuracy and uncertainty intensifies as the questions demand more logical reasoning of the LLM agent, while the same relationship remains sharp for knowledge retrieval tasks. △ Less

Submitted 18 November, 2024; originally announced November 2024.

Harnessing Machine Learning for Single-Shot Measurement of Free Electron Laser Pulse Power

Authors: Till Korten, Vladimir Rybnikov, Mathias Vogt, Juliane Roensch-Schulenburg, Peter Steinbach, Najmeh Mirian

Abstract: Electron beam accelerators are essential in many scientific and technological fields. Their operation relies heavily on the stability and precision of the electron beam. Traditional diagnostic techniques encounter difficulties in addressing the complex and dynamic nature of electron beams. Particularly in the context of free-electron lasers (FELs), it is fundamentally impossible to measure the las… ▽ More Electron beam accelerators are essential in many scientific and technological fields. Their operation relies heavily on the stability and precision of the electron beam. Traditional diagnostic techniques encounter difficulties in addressing the complex and dynamic nature of electron beams. Particularly in the context of free-electron lasers (FELs), it is fundamentally impossible to measure the lasing-on and lasingoff electron power profiles for a single electron bunch. This is a crucial hurdle in the exact reconstruction of the photon pulse profile. To overcome this hurdle, we developed a machine learning model that predicts the temporal power profile of the electron bunch in the lasing-off regime using machine parameters that can be obtained when lasing is on. The model was statistically validated and showed superior predictions compared to the state-of-the-art batch calibrations. The work we present here is a critical element for a virtual pulse reconstruction diagnostic (VPRD) tool designed to reconstruct the power profile of individual photon pulses without requiring repeated measurements in the lasing-off regime. This promises to significantly enhance the diagnostic capabilities in FELs at large. △ Less

Submitted 15 November, 2024; v1 submitted 14 November, 2024; originally announced November 2024.

Comments: 10 pages, 4 figures, Machine Learning and the Physical Sciences Workshop, NeurIPS 2024 https://neurips.cc/virtual/2024/100009

Uncertainty Estimation in Instance Segmentation with Star-convex Shapes

Authors: Qasim M. K. Siddiqui, Sebastian Starke, Peter Steinbach

Abstract: Instance segmentation has witnessed promising advancements through deep neural network-based algorithms. However, these models often exhibit incorrect predictions with unwarranted confidence levels. Consequently, evaluating prediction uncertainty becomes critical for informed decision-making. Existing methods primarily focus on quantifying uncertainty in classification or regression tasks, lacking… ▽ More Instance segmentation has witnessed promising advancements through deep neural network-based algorithms. However, these models often exhibit incorrect predictions with unwarranted confidence levels. Consequently, evaluating prediction uncertainty becomes critical for informed decision-making. Existing methods primarily focus on quantifying uncertainty in classification or regression tasks, lacking emphasis on instance segmentation. Our research addresses the challenge of estimating spatial certainty associated with the location of instances with star-convex shapes. Two distinct clustering approaches are evaluated which compute spatial and fractional certainty per instance employing samples by the Monte-Carlo Dropout or Deep Ensemble technique. Our study demonstrates that combining spatial and fractional certainty scores yields improved calibrated estimation over individual certainty scores. Notably, our experimental results show that the Deep Ensemble technique alongside our novel radial clustering approach proves to be an effective strategy. Our findings emphasize the significance of evaluating the calibration of estimated certainties for model reliability and decision-making. △ Less

Submitted 19 September, 2023; originally announced September 2023.

Detecting Adversarial Examples in Batches -- a geometrical approach

Authors: Danush Kumar Venkatesh, Peter Steinbach

Abstract: Many deep learning methods have successfully solved complex tasks in computer vision and speech recognition applications. Nonetheless, the robustness of these models has been found to be vulnerable to perturbed inputs or adversarial examples, which are imperceptible to the human eye, but lead the model to erroneous output decisions. In this study, we adapt and introduce two geometric metrics, dens… ▽ More Many deep learning methods have successfully solved complex tasks in computer vision and speech recognition applications. Nonetheless, the robustness of these models has been found to be vulnerable to perturbed inputs or adversarial examples, which are imperceptible to the human eye, but lead the model to erroneous output decisions. In this study, we adapt and introduce two geometric metrics, density and coverage, and evaluate their use in detecting adversarial samples in batches of unseen data. We empirically study these metrics using MNIST and two real-world biomedical datasets from MedMNIST, subjected to two different adversarial attacks. Our experiments show promising results for both metrics to detect adversarial examples. We believe that his work can lay the ground for further study on these metrics' use in deployed machine learning systems to monitor for possible attacks by adversarial examples or related pathologies such as dataset shift. △ Less

Submitted 17 June, 2022; originally announced June 2022.

Comments: Submitted to AdvML workshop at ICML2022

Recommendations on test datasets for evaluating AI solutions in pathology

Authors: André Homeyer, Christian Geißler, Lars Ole Schwen, Falk Zakrzewski, Theodore Evans, Klaus Strohmenger, Max Westphal, Roman David Bülow, Michaela Kargl, Aray Karjauv, Isidre Munné-Bertran, Carl Orge Retzlaff, Adrià Romero-López, Tomasz Sołtysiński, Markus Plass, Rita Carvalho, Peter Steinbach, Yu-Chia Lan, Nassim Bouteldja, David Haber, Mateo Rojas-Carulla, Alireza Vafaei Sadr, Matthias Kraft, Daniel Krüger, Rutger Fick , et al. (5 additional authors not shown)

Abstract: Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recom… ▽ More Artificial intelligence (AI) solutions that automatically extract information from digital histology images have shown great promise for improving pathological diagnosis. Prior to routine use, it is important to evaluate their predictive performance and obtain regulatory approval. This assessment requires appropriate test datasets. However, compiling such datasets is challenging and specific recommendations are missing. A committee of various stakeholders, including commercial AI developers, pathologists, and researchers, discussed key aspects and conducted extensive literature reviews on test datasets in pathology. Here, we summarize the results and derive general recommendations for the collection of test datasets. We address several questions: Which and how many images are needed? How to deal with low-prevalence subsets? How can potential bias be detected? How should datasets be reported? What are the regulatory requirements in different countries? The recommendations are intended to help AI developers demonstrate the utility of their products and to help regulatory agencies and end users verify reported performance measures. Further research is needed to formulate criteria for sufficiently representative test datasets so that AI solutions can operate with less user intervention and better support diagnostic workflows in the future. △ Less

Submitted 21 April, 2022; originally announced April 2022.

Journal ref: Mod Pathol (2022)

Machine Learning State-of-the-Art with Uncertainties

Authors: Peter Steinbach, Felicita Gernhardt, Mahnoor Tanveer, Steve Schmerler, Sebastian Starke

Abstract: With the availability of data, hardware, software ecosystem and relevant skill sets, the machine learning community is undergoing a rapid development with new architectures and approaches appearing at high frequency every year. In this article, we conduct an exemplary image classification study in order to demonstrate how confidence intervals around accuracy measurements can greatly enhance the co… ▽ More With the availability of data, hardware, software ecosystem and relevant skill sets, the machine learning community is undergoing a rapid development with new architectures and approaches appearing at high frequency every year. In this article, we conduct an exemplary image classification study in order to demonstrate how confidence intervals around accuracy measurements can greatly enhance the communication of research results as well as impact the reviewing process. In addition, we explore the hallmarks and limitations of this approximation. We discuss the relevance of this approach reflecting on a spotlight publication of ICLR22. A reproducible workflow is made available as an open-source adjoint to this publication. Based on our discussion, we make suggestions for improving the authoring and reviewing process of machine learning articles. △ Less

Submitted 14 April, 2022; v1 submitted 11 April, 2022; originally announced April 2022.

Comments: 9 pages, 6 figures. Accepted at the ICLR2022 workshop on ML Evaluation Standards. Code to reproduce results can be obtained from https://github.com/psteinb/sota_on_uncertainties.git

Quantification of Bore Path Uncertainty in Borehole Heat Exchanger Arrays

Authors: Philipp Steinbach, Daniel Otto Schulte, Bastian Welsch, Ingo Sass, Jens Lang

Abstract: Borehole heat exchanger arrays have become a common implement for the utilization of thermal energy in the soil. Building these facilities is expensive, especially the drilling of boreholes, into which closed-pipe heat exchangers are inserted. Therefore, cost-reducing drilling methods are common practice, which can produce inaccuracies of varying degree. This brings into question how much these in… ▽ More Borehole heat exchanger arrays have become a common implement for the utilization of thermal energy in the soil. Building these facilities is expensive, especially the drilling of boreholes, into which closed-pipe heat exchangers are inserted. Therefore, cost-reducing drilling methods are common practice, which can produce inaccuracies of varying degree. This brings into question how much these inaccuracies could potentially affect the performance of a planned system. In the presented case study, an uncertainty quantification for seasonally operated borehole heat exchanger arrays is performed to analyze the bore paths' deviations impact. We introduce an adaptive, anisotropic stochastic collocation method, known as the generalized Smolyak algorithm, which was previously unused in this context and apply it to a numerical model of the borehole heat exchanger array. Our results show that the borehole heat exchanger array performance is surprisingly reliable even with potentially severe implementation errors during their construction. This, coupled with the potential uses of the presented method in similar applications gives planners and investors valuable information regarding the viability of borehole heat exchanger arrays in the face of uncertainty. With this paper, we hope to provide a powerful statistical tool to the field of geothermal energy, in which uncertainty quantification methods are still rarely used at this point. The discussed case study represents a jumping-off point for further investigations on the effects of uncertainty on borehole heat exchanger arrays and borehole thermal energy storage systems. △ Less

Submitted 27 February, 2021; originally announced March 2021.

Comments: 31 pages, 13 figures

MSC Class: 65C20; 65C60; 86-08;

VLF transmitters as tools for monitoring the plasmasphere

Authors: David Koronczay, Janos Lichtenberger, Lilla Juhasz, Peter Steinbach, George Hospodarsky

Abstract: Continuous burst mode VLF measurements were recorded on the RBSP/Van Allen Probes satellites and are analyzed to detect pulses from the Russian Alpha (RSDN-20) ground-based navigational system. Based on the wave characteristics of these pulses and on the position of the spacecraft, the signals propagated mostly in ducted mode in the plasmasphere. Knowledge of the propagation path allowed us to car… ▽ More Continuous burst mode VLF measurements were recorded on the RBSP/Van Allen Probes satellites and are analyzed to detect pulses from the Russian Alpha (RSDN-20) ground-based navigational system. Based on the wave characteristics of these pulses and on the position of the spacecraft, the signals propagated mostly in ducted mode in the plasmasphere. Knowledge of the propagation path allowed us to carry out a monochromatic wave propagation inversion to obtain plasmaspheric electron densities. We compared the obtained densities with independent in-situ measurements on the spacecraft. The results show good agreement, validating our inversion process. This contributes to validating the field-aligned density profile model routinely used in the inversion of whistlers detected on the ground. Furthermore, our method can provide electron densities at regimes where no alternative measurements are available on the spacecraft. This raises the possibility of using this method as an additional tool to measure and monitor plasmaspheric electron densities. △ Less

Submitted 1 November, 2018; v1 submitted 4 July, 2018; originally announced July 2018.

Comments: 21 pages, 5 figures, accepted by Journal of Geophysical Research

Journal ref: Journal of Geophysical Research: Space Physics, 123. (2018)

gearshifft - The FFT Benchmark Suite for Heterogeneous Platforms

Authors: Peter Steinbach, Matthias Werner

Abstract: Fast Fourier Transforms (FFTs) are exploited in a wide variety of fields ranging from computer science to natural sciences and engineering. With the rising data production bandwidths of modern FFT applications, judging best which algorithmic tool to apply, can be vital to any scientific endeavor. As tailored FFT implementations exist for an ever increasing variety of high performance computer hard… ▽ More Fast Fourier Transforms (FFTs) are exploited in a wide variety of fields ranging from computer science to natural sciences and engineering. With the rising data production bandwidths of modern FFT applications, judging best which algorithmic tool to apply, can be vital to any scientific endeavor. As tailored FFT implementations exist for an ever increasing variety of high performance computer hardware, choosing the best performing FFT implementation has strong implications for future hardware purchase decisions, for resources FFTs consume and for possibly decisive financial and time savings ahead of the competition. This paper therefor presents gearshifft, which is an open-source and vendor agnostic benchmark suite to process a wide variety of problem sizes and types with state-of-the-art FFT implementations (fftw, clfft and cufft). gearshifft provides a reproducible, unbiased and fair comparison on a wide variety of hardware to explore which FFT variant is best for a given problem size. △ Less

Submitted 11 July, 2017; v1 submitted 2 February, 2017; originally announced February 2017.

Journal ref: High Performance Computing, Theoretical Computer Science and General Issues, Vol. 10266, Springer International Publishing. (ISC High Performance 2017)

An automated workflow for parallel processing of large multiview SPIM recordings

Authors: Christopher Schmied, Peter Steinbach, Tobias Pietzsch, Stephan Preibisch, Pavel Tomancak

Abstract: Multiview light sheet fluorescence microscopy (LSFM) allows to image developing organisms in 3D at unprecedented temporal resolution over long periods of time. The resulting massive amounts of raw image data requires extensive processing interactively via dedicated graphical user interface (GUI) applications. The consecutive processing steps can be easily automated and the individual time points c… ▽ More Multiview light sheet fluorescence microscopy (LSFM) allows to image developing organisms in 3D at unprecedented temporal resolution over long periods of time. The resulting massive amounts of raw image data requires extensive processing interactively via dedicated graphical user interface (GUI) applications. The consecutive processing steps can be easily automated and the individual time points can be processed independently, which lends itself to trivial parallelization on a high performance cluster (HPC). Here we introduce an automated workflow for processing large multiview, multi-channel, multi-illumination time-lapse LSFM data on a single workstation or in parallel on a HPC. The pipeline relies on snakemake to resolve dependencies among consecutive processing steps and can be easily adapted to any cluster environment for processing LSFM data in a fraction of the time required to collect it. △ Less

Submitted 11 August, 2015; v1 submitted 30 July, 2015; originally announced July 2015.

Comments: 13 pages with supplement, LATEX; 1 table, 1 figure, 2 supplementary figures, 2 supplementary lists, 2 supplementary tables; corrected error in results table, results unchanged

How do particle physicists learn the programming concepts they need?

Authors: Stefan Kluth, Maria Grazia Pia, Thomas Schoerner-Sadenius, Peter Steinbach

Abstract: The ability to read, use and develop code efficiently and successfully is a key ingredient in modern particle physics. We report the experience of a training program, identified as "Advanced Programming Concepts", that introduces software concepts, methods and techniques to work effectively on a daily basis in a HEP experiment or other programming intensive fields. This paper illustrates the princ… ▽ More The ability to read, use and develop code efficiently and successfully is a key ingredient in modern particle physics. We report the experience of a training program, identified as "Advanced Programming Concepts", that introduces software concepts, methods and techniques to work effectively on a daily basis in a HEP experiment or other programming intensive fields. This paper illustrates the principles, motivations and methods that shape the "Advanced Computing Concepts" training program, the knowledge base that it conveys, an analysis of the feedback received so far, and the integration of these concepts in the software development process of the experiments as well as its applicability to a wider audience. △ Less

Submitted 18 May, 2015; originally announced May 2015.

Comments: 8 pages, 2 figures, CHEP2015 proceedings

Expected Performance of the ATLAS Experiment - Detector, Trigger and Physics

Authors: The ATLAS Collaboration, G. Aad, E. Abat, B. Abbott, J. Abdallah, A. A. Abdelalim, A. Abdesselam, O. Abdinov, B. Abi, M. Abolins, H. Abramowicz, B. S. Acharya, D. L. Adams, T. N. Addy, C. Adorisio, P. Adragna, T. Adye, J. A. Aguilar-Saavedra, M. Aharrouche, S. P. Ahlen, F. Ahles, A. Ahmad, H. Ahmed, G. Aielli, T. Akdogan , et al. (2587 additional authors not shown)

Abstract: A detailed study is presented of the expected performance of the ATLAS detector. The reconstruction of tracks, leptons, photons, missing energy and jets is investigated, together with the performance of b-tagging and the trigger. The physics potential for a variety of interesting physics processes, within the Standard Model and beyond, is examined. The study comprises a series of notes based on… ▽ More A detailed study is presented of the expected performance of the ATLAS detector. The reconstruction of tracks, leptons, photons, missing energy and jets is investigated, together with the performance of b-tagging and the trigger. The physics potential for a variety of interesting physics processes, within the Standard Model and beyond, is examined. The study comprises a series of notes based on simulations of the detector and physics processes, with particular emphasis given to the data expected from the first years of operation of the LHC at CERN. △ Less

Submitted 14 August, 2009; v1 submitted 28 December, 2008; originally announced January 2009.