Showing 1–17 of 17 results for author: Pivarski, J

The Critical Importance of Software for HEP

Authors: HEP Software Foundation, :, Christina Agapopoulou, Claire Antel, Saptaparna Bhattacharya, Steven Gardiner, Krzysztof L. Genser, James Andrew Gooding, Alexander Held, Michel Hernandez Villanueva, Michel Jouvin, Tommaso Lari, Valeriia Lukashenko, Sudhir Malik, Alexander Moreno Briceño, Stephen Mrenna, Inês Ochoa, Joseph D. Osborn, Jim Pivarski, Alan Price, Eduardo Rodrigues, Richa Sharma, Nicholas Smith, Graeme Andrew Stewart, Anna Zaborowska , et al. (2 additional authors not shown)

Abstract: Particle physics has an ambitious and broad global experimental programme for the coming decades. Large investments in building new facilities are already underway or under consideration. Scaling the present processing power and data storage needs by the foreseen increase in data rates in the next decade for HL-LHC is not sustainable within the current budgets. As a result, a more efficient usage… ▽ More Particle physics has an ambitious and broad global experimental programme for the coming decades. Large investments in building new facilities are already underway or under consideration. Scaling the present processing power and data storage needs by the foreseen increase in data rates in the next decade for HL-LHC is not sustainable within the current budgets. As a result, a more efficient usage of computing resources is required in order to realise the physics potential of future experiments. Software and computing are an integral part of experimental design, trigger and data acquisition, simulation, reconstruction, and analysis, as well as related theoretical predictions. A significant investment in computing and software is therefore critical. Advances in software and computing, including artificial intelligence (AI) and machine learning (ML), will be key for solving these challenges. Making better use of new processing hardware such as graphical processing units (GPUs) or ARM chips is a growing trend. This forms part of a computing solution that makes efficient use of facilities and contributes to the reduction of the environmental footprint of HEP computing. The HEP community already provided a roadmap for software and computing for the last EPPSU, and this paper updates that, with a focus on the most resource critical parts of our data processing chain. △ Less

Submitted 1 April, 2025; originally announced April 2025.

PyHEP.dev 2024 Workshop Summary Report, August 26-30 2024, Aachen, Germany

Authors: Azzah Alshehri, Jan Bürger, Saransh Chopra, Niclas Eich, Jonas Eppelt, Martin Erdmann, Jonas Eschle, Peter Fackeldey, Maté Farkas, Matthew Feickert, Tristan Fillinger, Benjamin Fischer, Stefan Fröse, Lino Oscar Gerlach, Nikolai Hartmann, Alexander Heidelbach, Alexander Held, Marian I Ivanov, Josué Molina, Yaroslav Nikitenko, Ianna Osborne, Vincenzo Eduardo Padulano, Jim Pivarski, Cyrille Praz, Marcel Rieger , et al. (6 additional authors not shown)

Abstract: The second PyHEP.dev workshop, part of the "Python in HEP Developers" series organized by the HEP Software Foundation (HSF), took place in Aachen, Germany, from August 26 to 30, 2024. This gathering brought together nearly 30 Python package developers, maintainers, and power users to engage in informal discussions about current trends in Python, with a primary focus on analysis tools and technique… ▽ More The second PyHEP.dev workshop, part of the "Python in HEP Developers" series organized by the HEP Software Foundation (HSF), took place in Aachen, Germany, from August 26 to 30, 2024. This gathering brought together nearly 30 Python package developers, maintainers, and power users to engage in informal discussions about current trends in Python, with a primary focus on analysis tools and techniques in High Energy Physics (HEP). The workshop agenda encompassed a range of topics, such as defining the scope of HEP data analysis, exploring the Analysis Grand Challenge project, evaluating statistical models and serialization methods, assessing workflow management systems, examining histogramming practices, and investigating distributed processing tools like RDataFrame, Coffea, and Dask. Additionally, the workshop dedicated time to brainstorming the organization of future PyHEP.dev events, upholding the tradition of alternating between Europe and the United States as host locations. This document, prepared by the session conveners in the weeks following the workshop, serves as a summary of the key discussions, salient points, and conclusions that emerged. △ Less

Submitted 17 December, 2024; v1 submitted 2 October, 2024; originally announced October 2024.

Comments: 16 pages, 0 figures; workshop summary

Bridging Worlds: Achieving Language Interoperability between Julia and Python in Scientific Computing

Authors: Ianna Osborne, Jim Pivarski, Jerry Ling

Abstract: In the realm of scientific computing, both Julia and Python have established themselves as powerful tools. Within the context of High Energy Physics (HEP) data analysis, Python has been traditionally favored, yet there exists a compelling case for migrating legacy software to Julia. This article focuses on language interoperability, specifically exploring how Awkward Array data structures can brid… ▽ More In the realm of scientific computing, both Julia and Python have established themselves as powerful tools. Within the context of High Energy Physics (HEP) data analysis, Python has been traditionally favored, yet there exists a compelling case for migrating legacy software to Julia. This article focuses on language interoperability, specifically exploring how Awkward Array data structures can bridge the gap between Julia and Python. The talk offers insights into key considerations such as memory management, data buffer copies, and dependency handling. It delves into the performance enhancements achieved by invoking Julia from Python and vice versa, particularly for intensive array-oriented calculations involving large-scale, though not excessively dimensional, arrays of HEP data. The advantages and challenges inherent in achieving interoperability between Julia and Python in the domain of scientific computing are discussed. △ Less

Submitted 28 April, 2024; originally announced April 2024.

Comments: 8 pages, 1 figure, ACAT2024 workshop

Potential of the Julia programming language for high energy physics computing

Authors: J. Eschle, T. Gal, M. Giordano, P. Gras, B. Hegner, L. Heinrich, U. Hernandez Acosta, S. Kluth, J. Ling, P. Mato, M. Mikhasenko, A. Moreno Briceño, J. Pivarski, K. Samaras-Tsakiris, O. Schulz, G. . A. Stewart, J. Strube, V. Vassilev

Abstract: Research in high energy physics (HEP) requires huge amounts of computing and storage, putting strong constraints on the code speed and resource usage. To meet these requirements, a compiled high-performance language is typically used; while for physicists, who focus on the application when developing the code, better research productivity pleads for a high-level programming language. A popular app… ▽ More Research in high energy physics (HEP) requires huge amounts of computing and storage, putting strong constraints on the code speed and resource usage. To meet these requirements, a compiled high-performance language is typically used; while for physicists, who focus on the application when developing the code, better research productivity pleads for a high-level programming language. A popular approach consists of combining Python, used for the high-level interface, and C++, used for the computing intensive part of the code. A more convenient and efficient approach would be to use a language that provides both high-level programming and high-performance. The Julia programming language, developed at MIT especially to allow the use of a single language in research activities, has followed this path. In this paper the applicability of using the Julia language for HEP research is explored, covering the different aspects that are important for HEP code development: runtime performance, handling of large projects, interface with legacy code, distributed computing, training, and ease of programming. The study shows that the HEP community would benefit from a large scale adoption of this programming language. The HEP-specific foundation libraries that would need to be consolidated are identified △ Less

Submitted 6 October, 2023; v1 submitted 6 June, 2023; originally announced June 2023.

Comments: 32 pages, 5 figures, 4 tables

ACM Class: J.2

Journal ref: Computing. Comput Softw Big Sci 7, 10 (2023)

Awkward to RDataFrame and back

Authors: Ianna Osborne, Jim Pivarski

Abstract: Awkward Arrays and RDataFrame provide two very different ways of performing calculations at scale. By adding the ability to zero-copy convert between them, users get the best of both. It gives users a better flexibility in mixing different packages and languages in their analysis. In Awkward Array version 2, the ak.to_rdataframe function presents a view of an Awkward Array as an RDataFrame source.… ▽ More Awkward Arrays and RDataFrame provide two very different ways of performing calculations at scale. By adding the ability to zero-copy convert between them, users get the best of both. It gives users a better flexibility in mixing different packages and languages in their analysis. In Awkward Array version 2, the ak.to_rdataframe function presents a view of an Awkward Array as an RDataFrame source. This view is generated on demand and the data are not copied. The column readers are generated based on the run-time type of the views. The readers are passed to a generated source derived from ROOT::RDF::RDataSource. The ak.from_rdataframe function converts the selected columns as native Awkward Arrays. The details of the implementation exploiting JIT techniques are discussed. The examples of analysis of data stored in Awkward Arrays via a high-level interface of an RDataFrame are presented. A few examples of the column definition, applying user-defined filters written in C++, and plotting or extracting the columnar data as Awkward Arrays are shown. Current limitations and future plans are discussed. △ Less

Submitted 20 February, 2023; originally announced February 2023.

Comments: 5 pages, 3 figures

An array-oriented Python interface for FastJet

Authors: Aryan Roy, Jim Pivarski, Chad Wells Freer

Abstract: Analysis on HEP data is an iterative process in which the results of one step often inform the next. In an exploratory analysis, it is common to perform one computation on a collection of events, then view the results (often with histograms) to decide what to try next. Awkward Array is a Scikit-HEP Python package that enables data analysis with array-at-a-time operations to implement cuts as slice… ▽ More Analysis on HEP data is an iterative process in which the results of one step often inform the next. In an exploratory analysis, it is common to perform one computation on a collection of events, then view the results (often with histograms) to decide what to try next. Awkward Array is a Scikit-HEP Python package that enables data analysis with array-at-a-time operations to implement cuts as slices, combinatorics as composable functions, etc. However, most C++ HEP libraries, such as FastJet, have an imperative, one-particle-at-a-time interface, which would be inefficient in Python and goes against the grain of the array-at-a-time logic of scientific Python. Therefore, we developed fastjet, a pip-installable Python package that provides FastJet C++ binaries, the classic (particle-at-a-time) Python interface, and the new array-oriented interface for use with Awkward Array. The new interface streamlines interoperability with scientific Python software beyond HEP, such as machine learning. In one case, adopting this library along with other array-oriented tools accelerated HEP analysis code by a factor of 20. It was designed to be easily integrated with libraries in the Scikit-HEP ecosystem, including Uproot (file I/O), hist (histogramming), Vector (Lorentz vectors), and Coffea (high-level glue). We discuss the design of the fastjet Python library, integrating the classic interface with the array oriented interface and with the Vector library for Lorentz vector operations. The new interface was developed as open source. △ Less

Submitted 8 February, 2022; originally announced February 2022.

Comments: 5 pages, 2 figures, submitted to ACAT 2021 proceedings

Journal ref: J. Phys.: Conf. Ser. 2438 012011 (2023)

HL-LHC Computing Review Stage 2, Common Software Projects: Data Science Tools for Analysis

Authors: Jim Pivarski, Eduardo Rodrigues, Kevin Pedro, Oksana Shadura, Benjamin Krikler, Graeme A. Stewart

Abstract: This paper was prepared by the HEP Software Foundation (HSF) PyHEP Working Group as input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which took place in November, 2021. It describes the adoption of Python and data science tools in HEP, discusses the likelihood of future scenarios, and recommendations for action by the HEP community. This paper was prepared by the HEP Software Foundation (HSF) PyHEP Working Group as input to the second phase of the LHCC review of High-Luminosity LHC (HL-LHC) computing, which took place in November, 2021. It describes the adoption of Python and data science tools in HEP, discusses the likelihood of future scenarios, and recommendations for action by the HEP community. △ Less

Submitted 4 February, 2022; originally announced February 2022.

Comments: 25 pages, 7 figures; presented at https://indico.cern.ch/event/1058274/ (LHCC Review of HL-LHC Computing)

Report number: FERMILAB-CONF-22-061-SCD

Learning from the Pandemic: the Future of Meetings in HEP and Beyond

Authors: Mark S. Neubauer, Todd Adams, Jennifer Adelman-McCarthy, Gabriele Benelli, Tulika Bose, David Britton, Pat Burchat, Joel Butler, Timothy A. Cartwright, Tomáš Davídek, Jacques Dumarchez, Peter Elmer, Matthew Feickert, Ben Galewsky, Mandeep Gill, Maciej Gladki, Aman Goel, Jonathan E. Guyer, Bo Jayatilaka, Brendan Kiburg, Benjamin Krikler, David Lange, Claire Lee, Nick Manganelli, Giovanni Marchiori , et al. (14 additional authors not shown)

Abstract: The COVID-19 pandemic has by-and-large prevented in-person meetings since March 2020. While the increasing deployment of effective vaccines around the world is a very positive development, the timeline and pathway to "normality" is uncertain and the "new normal" we will settle into is anyone's guess. Particle physics, like many other scientific fields, has more than a year of experience in holding… ▽ More The COVID-19 pandemic has by-and-large prevented in-person meetings since March 2020. While the increasing deployment of effective vaccines around the world is a very positive development, the timeline and pathway to "normality" is uncertain and the "new normal" we will settle into is anyone's guess. Particle physics, like many other scientific fields, has more than a year of experience in holding virtual meetings, workshops, and conferences. A great deal of experimentation and innovation to explore how to execute these meetings effectively has occurred. Therefore, it is an appropriate time to take stock of what we as a community learned from running virtual meetings and discuss possible strategies for the future. Continuing to develop effective strategies for meetings with a virtual component is likely to be important for reducing the carbon footprint of our research activities, while also enabling greater diversity and inclusion for participation. This report summarizes a virtual two-day workshop on Virtual Meetings held May 5-6, 2021 which brought together experts from both inside and outside of high-energy physics to share their experiences and practices with organizing and executing virtual workshops, and to develop possible strategies for future meetings as we begin to emerge from the COVID-19 pandemic. This report outlines some of the practices and tools that have worked well which we hope will serve as a valuable resource for future virtual meeting organizers in all scientific fields. △ Less

Submitted 29 June, 2021; originally announced June 2021.

Comments: A report from the "Virtual Meetings" IRIS-HEP Blueprint Workshop: https://indico.cern.ch/event/1026363/

Analysis Description Languages for the LHC

Authors: Sezen Sekmen, Philippe Gras, Lindsey Gray, Benjamin Krikler, Jim Pivarski, Harrison B. Prosper, Andrea Rizzi, Gokhan Unel, Gordon Watts

Abstract: An analysis description language is a domain specific language capable of describing the contents of an LHC analysis in a standard and unambiguous way, independent of any computing framework. It is designed for use by anyone with an interest in, and knowledge of, LHC physics, i.e., experimentalists, phenomenologists and other enthusiasts. Adopting analysis description languages would bring numerou… ▽ More An analysis description language is a domain specific language capable of describing the contents of an LHC analysis in a standard and unambiguous way, independent of any computing framework. It is designed for use by anyone with an interest in, and knowledge of, LHC physics, i.e., experimentalists, phenomenologists and other enthusiasts. Adopting analysis description languages would bring numerous benefits for the LHC experimental and phenomenological communities ranging from analysis preservation beyond the lifetimes of experiments or analysis software to facilitating the abstraction, design, visualization, validation, combination, reproduction, interpretation and overall communication of the analysis contents. Here, we introduce the analysis description language concept and summarize the current efforts ongoing to develop such languages and tools to use them in LHC analyses. △ Less

Submitted 3 November, 2020; originally announced November 2020.

Comments: Accepted contribution to the proceedings of The 8th Annual Conference on Large Hadron Collider Physics, LHCP2020, 25-30 May, 2020, online

Journal ref: Proceedings of Science, PoS(LHCP2020)065

HL-LHC Computing Review: Common Tools and Community Software

Authors: HEP Software Foundation, :, Thea Aarrestad, Simone Amoroso, Markus Julian Atkinson, Joshua Bendavid, Tommaso Boccali, Andrea Bocci, Andy Buckley, Matteo Cacciari, Paolo Calafiura, Philippe Canal, Federico Carminati, Taylor Childers, Vitaliano Ciulli, Gloria Corti, Davide Costanzo, Justin Gage Dezoort, Caterina Doglioni, Javier Mauricio Duarte, Agnieszka Dziurda, Peter Elmer, Markus Elsing, V. Daniel Elvira, Giulio Eulisse , et al. (85 additional authors not shown)

Abstract: Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this doc… ▽ More Common and community software packages, such as ROOT, Geant4 and event generators have been a key part of the LHC's success so far and continued development and optimisation will be critical in the future. The challenges are driven by an ambitious physics programme, notably the LHC accelerator upgrade to high-luminosity, HL-LHC, and the corresponding detector upgrades of ATLAS and CMS. In this document we address the issues for software that is used in multiple experiments (usually even more widely than ATLAS and CMS) and maintained by teams of developers who are either not linked to a particular experiment or who contribute to common software within the context of their experiment activity. We also give space to general considerations for future software and projects that tackle upcoming challenges, no matter who writes it, which is an area where community convergence on best practice is extremely useful. △ Less

Submitted 31 August, 2020; originally announced August 2020.

Comments: 40 pages contribution to Snowmass 2021

Report number: HSF-DOC-2020-01

The Scikit HEP Project -- overview and prospects

Authors: Eduardo Rodrigues, Benjamin Krikler, Chris Burr, Dmitri Smirnov, Hans Dembinski, Henry Schreiner, Jaydeep Nandi, Jim Pivarski, Matthew Feickert, Matthieu Marinangeli, Nick Smith, Pratyush Das

Abstract: Scikit-HEP is a community-driven and community-oriented project with the goal of providing an ecosystem for particle physics data analysis in Python. Scikit-HEP is a toolset of approximately twenty packages and a few "affiliated" packages. It expands the typical Python data analysis tools for particle physicists. Each package focuses on a particular topic, and interacts with other packages in the… ▽ More Scikit-HEP is a community-driven and community-oriented project with the goal of providing an ecosystem for particle physics data analysis in Python. Scikit-HEP is a toolset of approximately twenty packages and a few "affiliated" packages. It expands the typical Python data analysis tools for particle physicists. Each package focuses on a particular topic, and interacts with other packages in the toolset, where appropriate. Most of the packages are easy to install in many environments; much work has been done this year to provide binary "wheels" on PyPI and conda-forge packages. The Scikit-HEP project has been gaining interest and momentum, by building a user and developer community engaging collaboration across experiments. Some of the packages are being used by other communities, including the astroparticle physics community. An overview of the overall project and toolset will be presented, as well as a vision for development and sustainability. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Comments: 6 pages, 3 figures, Proceedings of the 24th International Conference on Computing in High Energy and Nuclear Physics (CHEP 2019), Adelaide, Australia, 4-8 November 2019

HEP Software Foundation Community White Paper Working Group -- Data Organization, Management and Access (DOMA)

Authors: Dario Berzano, Riccardo Maria Bianchi, Ian Bird, Brian Bockelman, Simone Campana, Kaushik De, Dirk Duellmann, Peter Elmer, Robert Gardner, Vincent Garonne, Claudio Grandi, Oliver Gutsche, Andrew Hanushevsky, Burt Holzman, Bodhitha Jayatilaka, Ivo Jimenez, Michel Jouvin, Oliver Keeble, Alexei Klimentov, Valentin Kuznetsov, Eric Lancon, Mario Lassnig, Miron Livny, Carlos Maltzahn, Shawn McKee , et al. (13 additional authors not shown)

Abstract: Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess th… ▽ More Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess these changes is also proposed. △ Less

Submitted 30 November, 2018; originally announced December 2018.

Comments: arXiv admin note: text overlap with arXiv:1712.06592

Report number: HSF-CWP-2017-04

HEP Software Foundation Community White Paper Working Group --- Visualization

Authors: Matthew Bellis, Riccardo Maria Bianchi, Sebastien Binet, Ciril Bohak, Benjamin Couturier, Hadrien Grasland, Oliver Gutsche, Sergey Linev, Alex Martyniuk, Thomas McCauley, Edward Moyse, Alja Mrak Tadel, Mark Neubauer, Jeremi Niedziela, Leo Piilonen, Jim Pivarski, Martin Ritter, Tai Sakuma, Matevz Tadel, Barthélémy von Haller, Ilija Vukotic, Ben Waugh

Abstract: In modern High Energy Physics (HEP) experiments visualization of experimental data has a key role in many activities and tasks across the whole data chain: from detector development to monitoring, from event generation to reconstruction of physics objects, from detector simulation to data analysis, and all the way to outreach and education. In this paper, the definition, status, and evolution of d… ▽ More In modern High Energy Physics (HEP) experiments visualization of experimental data has a key role in many activities and tasks across the whole data chain: from detector development to monitoring, from event generation to reconstruction of physics objects, from detector simulation to data analysis, and all the way to outreach and education. In this paper, the definition, status, and evolution of data visualization for HEP experiments will be presented. Suggestions for the upgrade of data visualization tools and techniques in current experiments will be outlined, along with guidelines for future experiments. This paper expands on the summary content published in the HSF \emph{Roadmap} Community White Paper~\cite{HSF-CWP-2017-01} △ Less

Submitted 26 November, 2018; originally announced November 2018.

Report number: HSF-CWP-2017-15

HEP Software Foundation Community White Paper Working Group - Data and Software Preservation to Enable Reuse

Authors: M. D. Hildreth, A. Boehnlein, K. Cranmer, S. Dallmeier, R. Gardner, T. Hacker, L. Heinrich, I. Jimenez, M. Kane, D. S. Katz, T. Malik, C. Maltzahn, M. Neubauer, S. Neubert, Jim Pivarski, E. Sexton, J. Shiers, T. Simko, S. Smith, D. South, A. Verbytskyi, G. Watts, J. Wozniak

Abstract: In this chapter of the High Energy Physics Software Foundation Community Whitepaper, we discuss the current state of infrastructure, best practices, and ongoing developments in the area of data and software preservation in high energy physics. A re-framing of the motivation for preservation to enable re-use is presented. A series of research and development goals in software and other cyberinfrast… ▽ More In this chapter of the High Energy Physics Software Foundation Community Whitepaper, we discuss the current state of infrastructure, best practices, and ongoing developments in the area of data and software preservation in high energy physics. A re-framing of the motivation for preservation to enable re-use is presented. A series of research and development goals in software and other cyberinfrastructure that will aid in the enabling of reuse of particle physics analyses and production software are presented and discussed. △ Less

Submitted 2 October, 2018; originally announced October 2018.

Report number: HSF-CWP-2017-06

Machine Learning in High Energy Physics Community White Paper

Authors: Kim Albertsson, Piero Altoe, Dustin Anderson, John Anderson, Michael Andrews, Juan Pedro Araque Espinosa, Adam Aurisano, Laurent Basara, Adrian Bevan, Wahid Bhimji, Daniele Bonacorsi, Bjorn Burkle, Paolo Calafiura, Mario Campanelli, Louis Capps, Federico Carminati, Stefano Carrazza, Yi-fan Chen, Taylor Childers, Yann Coadou, Elias Coniavitis, Kyle Cranmer, Claire David, Douglas Davis, Andrea De Simone , et al. (103 additional authors not shown)

Abstract: Machine learning has been applied to several problems in particle physics research, beginning with applications to high-level physics analysis in the 1990s and 2000s, followed by an explosion of applications in particle and event identification and reconstruction in the 2010s. In this document we discuss promising future research and development areas for machine learning in particle physics. We d… ▽ More Machine learning has been applied to several problems in particle physics research, beginning with applications to high-level physics analysis in the 1990s and 2000s, followed by an explosion of applications in particle and event identification and reconstruction in the 2010s. In this document we discuss promising future research and development areas for machine learning in particle physics. We detail a roadmap for their implementation, software and hardware resource requirements, collaborative initiatives with the data science community, academia and industry, and training the particle physics community in data science. The main objective of the document is to connect and motivate these areas of research and development with the physics drivers of the High-Luminosity Large Hadron Collider and future neutrino experiments and identify the resource needs for their implementation. Additionally we identify areas where collaboration with external communities will be of great benefit. △ Less

Submitted 16 May, 2019; v1 submitted 8 July, 2018; originally announced July 2018.

Comments: Editors: Sergei Gleyzer, Paul Seyfert and Steven Schramm

HEP Software Foundation Community White Paper Working Group - Data Analysis and Interpretation

Authors: Lothar Bauerdick, Riccardo Maria Bianchi, Brian Bockelman, Nuno Castro, Kyle Cranmer, Peter Elmer, Robert Gardner, Maria Girone, Oliver Gutsche, Benedikt Hegner, José M. Hernández, Bodhitha Jayatilaka, David Lange, Mark S. Neubauer, Daniel S. Katz, Lukasz Kreczko, James Letts, Shawn McKee, Christoph Paus, Kevin Pedro, Jim Pivarski, Martin Ritter, Eduardo Rodrigues, Tai Sakuma, Elizabeth Sexton-Kennedy , et al. (4 additional authors not shown)

Abstract: At the heart of experimental high energy physics (HEP) is the development of facilities and instrumentation that provide sensitivity to new phenomena. Our understanding of nature at its most fundamental level is advanced through the analysis and interpretation of data from sophisticated detectors in HEP experiments. The goal of data analysis systems is to realize the maximum possible scientific po… ▽ More At the heart of experimental high energy physics (HEP) is the development of facilities and instrumentation that provide sensitivity to new phenomena. Our understanding of nature at its most fundamental level is advanced through the analysis and interpretation of data from sophisticated detectors in HEP experiments. The goal of data analysis systems is to realize the maximum possible scientific potential of the data within the constraints of computing and human resources in the least time. To achieve this goal, future analysis systems should empower physicists to access the data with a high level of interactivity, reproducibility and throughput capability. As part of the HEP Software Foundation Community White Paper process, a working group on Data Analysis and Interpretation was formed to assess the challenges and opportunities in HEP data analysis and develop a roadmap for activities in this area over the next decade. In this report, the key findings and recommendations of the Data Analysis and Interpretation Working Group are presented. △ Less

Submitted 9 April, 2018; originally announced April 2018.

Comments: arXiv admin note: text overlap with arXiv:1712.06592

Report number: HSF-CWP-2017-05

A Roadmap for HEP Software and Computing R&D for the 2020s

Authors: Johannes Albrecht, Antonio Augusto Alves Jr, Guilherme Amadio, Giuseppe Andronico, Nguyen Anh-Ky, Laurent Aphecetche, John Apostolakis, Makoto Asai, Luca Atzori, Marian Babik, Giuseppe Bagliesi, Marilena Bandieramonte, Sunanda Banerjee, Martin Barisits, Lothar A. T. Bauerdick, Stefano Belforte, Douglas Benjamin, Catrin Bernius, Wahid Bhimji, Riccardo Maria Bianchi, Ian Bird, Catherine Biscarat, Jakob Blomer, Kenneth Bloom, Tommaso Boccali , et al. (285 additional authors not shown)

Abstract: Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for… ▽ More Particle physics has an ambitious and broad experimental programme for the coming decades. This programme requires large investments in detector hardware, either to build new facilities and experiments, or to upgrade existing ones. Similarly, it requires commensurate investment in the R&D of software to acquire, manage, process, and analyse the shear amounts of data to be recorded. In planning for the HL-LHC in particular, it is critical that all of the collaborating stakeholders agree on the software goals and priorities, and that the efforts complement each other. In this spirit, this white paper describes the R&D activities required to prepare for this software upgrade. △ Less

Submitted 19 December, 2018; v1 submitted 18 December, 2017; originally announced December 2017.

Report number: HSF-CWP-2017-01

Journal ref: Comput Softw Big Sci (2019) 3, 7