Aristotle Cloud Federation: Container Runtimes Technical Report
Authors:
Peter Z. Vaillancourt,
Bennett Wineholt,
Tristan J. Shepherd,
Sara C. Pryor,
Jeffrey Lantz,
Richard Knepper,
Rich Wolski,
Christopher R. Myers,
Ben Trumbore,
Resa Reynolds,
Jodie Sprouse,
David Lifka
Abstract:
A National Science Foundation-sponsored container runtimes investigation was conducted by the Aristotle Cloud Federation to better understand the challenges of selecting and using Docker, Singularity, and X-Containers. The main goal of this investigation was to identify the "pain points" experienced by users when selecting and using containers for scientific research and to share lessons learned.…
▽ More
A National Science Foundation-sponsored container runtimes investigation was conducted by the Aristotle Cloud Federation to better understand the challenges of selecting and using Docker, Singularity, and X-Containers. The main goal of this investigation was to identify the "pain points" experienced by users when selecting and using containers for scientific research and to share lessons learned. Application performance characteristics are included in this report as well as user experiences with Kubernetes and container orchestration on cloud and HPC platforms. Scientists, research computing practitioners, and educators may find value in this report when considering the use and/or deployment of containers or when preparing students to meet the unique challenges of using containers in scientific research.
△ Less
Submitted 24 September, 2021;
originally announced September 2021.
Reproducible and Portable Workflows for Scientific Computing and HPC in the Cloud
Authors:
Peter Vaillancourt,
Bennett Wineholt,
Brandon Barker,
Plato Deliyannis,
Jackie Zheng,
Akshay Suresh,
Adam Brazier,
Rich Knepper,
Rich Wolski
Abstract:
The increasing availability of cloud computing services for science has changed the way scientific code can be developed, deployed, and run. Many modern scientific workflows are capable of running on cloud computing resources. Consequently, there is an increasing interest in the scientific computing community in methods, tools, and implementations that enable moving an application to the cloud and…
▽ More
The increasing availability of cloud computing services for science has changed the way scientific code can be developed, deployed, and run. Many modern scientific workflows are capable of running on cloud computing resources. Consequently, there is an increasing interest in the scientific computing community in methods, tools, and implementations that enable moving an application to the cloud and simplifying the process, and decreasing the time to meaningful scientific results. In this paper, we have applied the concepts of containerization for portability and multi-cloud automated deployment with industry-standard tools to three scientific workflows. We show how our implementations provide reduced complexity to portability of both the applications themselves, and their deployment across private and public clouds. Each application has been packaged in a Docker container with its dependencies and necessary environment setup for production runs. Terraform and Ansible have been used to automate the provisioning of compute resources and the deployment of each scientific application in a Multi-VM cluster. Each application has been deployed on the AWS and Aristotle Cloud Federation platforms. Variation in data management constraints, Multi-VM MPI communication, and embarrassingly parallel instance deployments were all explored and reported on. We thus present a sample of scientific workflows that can be simplified using the tools and our proposed implementation to deploy and run in a variety of cloud environments.
△ Less
Submitted 8 June, 2020;
originally announced June 2020.
