Using ATLAS@Home to exploit extra CPU from busy grid sites
Authors:
Wenjing Wu,
David Cameron,
Qing Di
Abstract:
Grid computing typically provides most of the data processing resources for large High Energy Physics experiments. However typical grid sites are not fully utilized by regular workloads. In order to increase the CPU utilization of these grid sites, the ATLAS@Home volunteer computing framework can be used as a backfilling mechanism. Results show an extra 15% to 42% of CPU cycles can be exploited by…
▽ More
Grid computing typically provides most of the data processing resources for large High Energy Physics experiments. However typical grid sites are not fully utilized by regular workloads. In order to increase the CPU utilization of these grid sites, the ATLAS@Home volunteer computing framework can be used as a backfilling mechanism. Results show an extra 15% to 42% of CPU cycles can be exploited by backfilling grid sites running regular workloads while the overall CPU utilization can remain over 90%. Backfilling has no impact on the failure rate of the grid jobs, and the impact on the CPU efficiency of grid jobs varies from 1% to 11% depending on the configuration of the site. In addition the throughput of backfill jobs in terms of CPU time per simulated event is the same as for resources dedicated to ATLAS@Home. This approach is sufficiently generic that it can easily be extended to other clusters.
△ Less
Submitted 29 November, 2018;
originally announced November 2018.
Next-Generation EU DataGrid Data Management Services
Authors:
Diana Bosio,
James Casey,
Akos Frohner,
Leanne Guy,
Peter Kunszt,
Erwin Laure,
Sophie Lemaitre,
Levi Lucio,
Heinz Stockinger,
Kurt Stockinger,
William Bell,
David Cameron,
Gavin McCance,
Paul Millar,
Joni Hahkala,
Niklas Karlsson,
Ville Nenonen,
Mika Silander,
Olle Mulmo,
Gian-Luca Volpato,
Giuseppe Andronico
Abstract:
We describe the architecture and initial implementation of the next-generation of Grid Data Management Middleware in the EU DataGrid (EDG) project.
The new architecture stems out of our experience and the users requirements gathered during the two years of running our initial set of Grid Data Management Services. All of our new services are based on the Web Service technology paradigm, very mu…
▽ More
We describe the architecture and initial implementation of the next-generation of Grid Data Management Middleware in the EU DataGrid (EDG) project.
The new architecture stems out of our experience and the users requirements gathered during the two years of running our initial set of Grid Data Management Services. All of our new services are based on the Web Service technology paradigm, very much in line with the emerging Open Grid Services Architecture (OGSA). We have modularized our components and invested a great amount of effort towards a secure, extensible and robust service, starting from the design but also using a streamlined build and testing framework.
Our service components are: Replica Location Service, Replica Metadata Service, Replica Optimization Service, Replica Subscription and high-level replica management. The service security infrastructure is fully GSI-enabled, hence compatible with the existing Globus Toolkit 2-based services; moreover, it allows for fine-grained authorization mechanisms that can be adjusted depending on the service semantics.
△ Less
Submitted 12 June, 2003; v1 submitted 30 May, 2003;
originally announced May 2003.
