-
HEP Software Foundation Community White Paper Working Group -- Data Organization, Management and Access (DOMA)
Authors:
Dario Berzano,
Riccardo Maria Bianchi,
Ian Bird,
Brian Bockelman,
Simone Campana,
Kaushik De,
Dirk Duellmann,
Peter Elmer,
Robert Gardner,
Vincent Garonne,
Claudio Grandi,
Oliver Gutsche,
Andrew Hanushevsky,
Burt Holzman,
Bodhitha Jayatilaka,
Ivo Jimenez,
Michel Jouvin,
Oliver Keeble,
Alexei Klimentov,
Valentin Kuznetsov,
Eric Lancon,
Mario Lassnig,
Miron Livny,
Carlos Maltzahn,
Shawn McKee
, et al. (13 additional authors not shown)
Abstract:
Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess th…
▽ More
Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess these changes is also proposed.
△ Less
Submitted 30 November, 2018;
originally announced December 2018.
-
HEP Software Foundation Community White Paper Working Group - Training, Staffing and Careers
Authors:
HEP Software Foundation,
:,
Dario Berzano,
Riccardo Maria Bianchi,
Peter Elmer,
Sergei V. Gleyzer John Harvey,
Roger Jones,
Michel Jouvin,
Daniel S. Katz,
Sudhir Malik,
Dario Menasce,
Mark Neubauer,
Fernanda Psihas,
Albert Puig Navarro,
Graeme A. Stewart,
Christopher Tunnell,
Justin A. Vasel,
Sean-Jiun Wang
Abstract:
The rapid evolution of technology and the parallel increasing complexity of algorithmic analysis in HEP requires developers to acquire a much larger portfolio of programming skills. Young researchers graduating from universities worldwide currently do not receive adequate preparation in the very diverse fields of modern computing to respond to growing needs of the most advanced experimental challe…
▽ More
The rapid evolution of technology and the parallel increasing complexity of algorithmic analysis in HEP requires developers to acquire a much larger portfolio of programming skills. Young researchers graduating from universities worldwide currently do not receive adequate preparation in the very diverse fields of modern computing to respond to growing needs of the most advanced experimental challenges. There is a growing consensus in the HEP community on the need for training programmes to bring researchers up to date with new software technologies, in particular in the domains of concurrent programming and artificial intelligence. We review some of the initiatives under way for introducing new training programmes and highlight some of the issues that need to be taken into account for these to be successful.
△ Less
Submitted 17 January, 2019; v1 submitted 8 July, 2018;
originally announced July 2018.
-
PROOF as a Service on the Cloud: a Virtual Analysis Facility based on the CernVM ecosystem
Authors:
Dario Berzano,
Jakob Blomer,
Predrag Buncic,
Ioannis Charalampidis,
Gerardo Ganis,
Georgios Lestaris,
René Meusel
Abstract:
PROOF, the Parallel ROOT Facility, is a ROOT-based framework which enables interactive parallelism for event-based tasks on a cluster of computing nodes. Although PROOF can be used simply from within a ROOT session with no additional requirements, deploying and configuring a PROOF cluster used to be not as straightforward. Recently great efforts have been spent to make the provisioning of generic…
▽ More
PROOF, the Parallel ROOT Facility, is a ROOT-based framework which enables interactive parallelism for event-based tasks on a cluster of computing nodes. Although PROOF can be used simply from within a ROOT session with no additional requirements, deploying and configuring a PROOF cluster used to be not as straightforward. Recently great efforts have been spent to make the provisioning of generic PROOF analysis facilities with zero configuration, with the added advantages of positively affecting both stability and scalability, making the deployment operations feasible even for the end user. Since a growing amount of large-scale computing resources are nowadays made available by Cloud providers in a virtualized form, we have developed the Virtual PROOF-based Analysis Facility: a cluster appliance combining the solid CernVM ecosystem and PoD (PROOF on Demand), ready to be deployed on the Cloud and leveraging some peculiar Cloud features such as elasticity. We will show how this approach is effective both for sysadmins, who will have little or no configuration to do to run it on their Clouds, and for the end users, who are ultimately in full control of their PROOF cluster and can even easily restart it by themselves in the unfortunate event of a major failure. We will also show how elasticity leads to a more optimal and uniform usage of Cloud resources.
△ Less
Submitted 19 February, 2014;
originally announced February 2014.
