-
Challenges of Translating HPC codes to Workflows for Heterogeneous and Dynamic Environments
Authors:
Fayssal Benkhaldoun,
Christophe Cérin,
Imad Kissami,
Walid Saad
Abstract:
In this paper we would like to share our experience for transforming a parallel code for a Computational Fluid Dynamics (CFD) problem into a parallel version for the RedisDG workflow engine. This system is able to capture heterogeneous and highly dynamic environments, thanks to opportunistic scheduling strategies. We show how to move to the field of "HPC as a Service" in order to use heterogeneous…
▽ More
In this paper we would like to share our experience for transforming a parallel code for a Computational Fluid Dynamics (CFD) problem into a parallel version for the RedisDG workflow engine. This system is able to capture heterogeneous and highly dynamic environments, thanks to opportunistic scheduling strategies. We show how to move to the field of "HPC as a Service" in order to use heterogeneous platforms. We mainly explain, through the CFD use case, how to transform the parallel code and we exhibit challenges to 'unfold' the task graph dynamically in order to improve the overall performance (in a broad sense) of the workflow engine. We discuss in particular of the impact on the workflow engine of such dynamic feature. This paper states that new models for High Performance Computing are possible, under the condition we revisit our mind in the direction of the potential of new paradigms such as cloud, edge computing.
△ Less
Submitted 24 May, 2021;
originally announced May 2021.
-
Towards Parallel CFD computation for the ADAPT framework
Authors:
Imad Kissami,
Christophe Cerin,
Fayssal Benkhaldoun,
Gilles Scarella
Abstract:
In order to run Computational Fluid Dynamics (CFD) codes on large scale infrastructures, parallel computing has to be used because of the computational intensive nature of the problems. In this paper we investigate the ADAPT platform where we couple flow Partial Differential Equations and a Poisson equation. This leads to a linear system which we solve using direct methods. The implementation deal…
▽ More
In order to run Computational Fluid Dynamics (CFD) codes on large scale infrastructures, parallel computing has to be used because of the computational intensive nature of the problems. In this paper we investigate the ADAPT platform where we couple flow Partial Differential Equations and a Poisson equation. This leads to a linear system which we solve using direct methods. The implementation deals with the MUMPS parallel multi-frontal direct solver and mesh partitioning methods using METIS to improve the performance of the framework. We also investigate, in this paper, how the mesh partitioning methods are able to optimize the mesh cell distribution for the ADAPT solver. The experience gained in this paper facilitates the move to the 3D version of ADAPT and the move to a Service Oriented view of ADAPT as future work.
△ Less
Submitted 24 May, 2021;
originally announced May 2021.
-
A Distributed and Approximated Nearest Neighbors Algorithm for an Efficient Large Scale Mean Shift Clustering
Authors:
Gaël Beck,
Tarn Duong,
Mustapha Lebbah,
Hanane Azzag,
Christophe Cérin
Abstract:
In this paper we target the class of modal clustering methods where clusters are defined in terms of the local modes of the probability density function which generates the data. The most well-known modal clustering method is the k-means clustering. Mean Shift clustering is a generalization of the k-means clustering which computes arbitrarily shaped clusters as defined as the basins of attraction…
▽ More
In this paper we target the class of modal clustering methods where clusters are defined in terms of the local modes of the probability density function which generates the data. The most well-known modal clustering method is the k-means clustering. Mean Shift clustering is a generalization of the k-means clustering which computes arbitrarily shaped clusters as defined as the basins of attraction to the local modes created by the density gradient ascent paths. Despite its potential, the Mean Shift approach is a computationally expensive method for unsupervised learning. Thus, we introduce two contributions aiming to provide clustering algorithms with a linear time complexity, as opposed to the quadratic time complexity for the exact Mean Shift clustering. Firstly we propose a scalable procedure to approximate the density gradient ascent. Second, our proposed scalable cluster labeling technique is presented. Both propositions are based on Locality Sensitive Hashing (LSH) to approximate nearest neighbors. These two techniques may be used for moderate sized datasets. Furthermore, we show that using our proposed approximations of the density gradient ascent as a pre-processing step in other clustering methods can also improve dedicated classification metrics. For the latter, a distributed implementation, written for the Spark/Scala ecosystem is proposed. For all these considered clustering methods, we present experimental results illustrating their labeling accuracy and their potential to solve concrete problems.
△ Less
Submitted 11 February, 2019;
originally announced February 2019.
-
Backtracking algorithms for service selection
Authors:
Yanik Ngoko,
Christophe Cérin,
Alfredo Goldman,
Dejan Milojicic
Abstract:
In this paper, we explore the automation of services' compositions. We focus on the service selection problem. In the formulation that we consider, the problem's inputs are constituted by a behavioral composition whose abstract services must be bound to concrete ones. The objective is to find the binding that optimizes the {\it utility} of the composition under some services level agreements. We p…
▽ More
In this paper, we explore the automation of services' compositions. We focus on the service selection problem. In the formulation that we consider, the problem's inputs are constituted by a behavioral composition whose abstract services must be bound to concrete ones. The objective is to find the binding that optimizes the {\it utility} of the composition under some services level agreements. We propose a complete solution. Firstly, we show that the service selection problem can be mapped onto a Constraint Satisfaction Problem (CSP). The benefit of this mapping is that the large know-how in the resolution of the CSP can be used for the service selection problem. Among the existing techniques for solving CSP, we consider the backtracking. Our second contribution is to propose various backtracking-based algorithms for the service selection problem. The proposed variants are inspired by existing heuristics for the CSP. We analyze the runtime gain of our framework over an intuitive resolution based on exhaustive search. Our last contribution is an experimental evaluation in which we demonstrate that there is an effective gain in using backtracking instead of some comparable approaches. The experiments also show that our proposal can be used for finding in real time, optimal solutions on small and medium services' compositions.
△ Less
Submitted 6 February, 2014;
originally announced February 2014.
-
Intégration des intergiciels de grilles de PC dans le nuage SlapOS : le cas de BOINC
Authors:
Christophe Cérin,
Alain Takoudjou,
Nicolas Grenèche
Abstract:
In this article we describe the problems and solutions related to the integration of desktop grid middleware in a cloud, in this case the open source SlapOS cloud. We focus on the issues about recipes that describe the integration and the problem of the confinement of execution. They constitute two aspects of service-oriented architecture and Cloud Computing. These two issues solved with SlapOS ar…
▽ More
In this article we describe the problems and solutions related to the integration of desktop grid middleware in a cloud, in this case the open source SlapOS cloud. We focus on the issues about recipes that describe the integration and the problem of the confinement of execution. They constitute two aspects of service-oriented architecture and Cloud Computing. These two issues solved with SlapOS are not in relation to what is traditionally done in the clouds because we do not rely on virtual machines and, there is no data center (as defined in cloud). Moreover, we show that from the initial deployment model we take into account not only Web applications, B2B applications... but also applications from the field of grids; here desktop grid middleware which is a case study.
△ Less
Submitted 27 November, 2012;
originally announced November 2012.
-
Étude de performance des systèmes de découverte de ressources
Authors:
Heithem Abbes,
Christophe Cérin,
Jean-Christophe Dubacq,
Mohamed Jemni
Abstract:
The Desktop Grid offers solutions to overcome several challenges and to answer increasingly needs of scientific computing. This technology consists mainly in exploiting PC resources, geographically dispersed, to treat time consuming applications and/or important storage capacity requiring applications. However, as resources number increases, the need for scalability, self-organisation, dynamic r…
▽ More
The Desktop Grid offers solutions to overcome several challenges and to answer increasingly needs of scientific computing. This technology consists mainly in exploiting PC resources, geographically dispersed, to treat time consuming applications and/or important storage capacity requiring applications. However, as resources number increases, the need for scalability, self-organisation, dynamic reconfiguration, decentralization and performance becomes more and more essential. In this context, this paper evaluates the scalability and performance of P2P tools for registering and discovering services (Publish/Subscribe systems). Three protocols are used in this purpose: Bonjour, Avahi and Pastry. We have studied the behaviour of these protocols related to two criteria: the elapsed time for registrations services and the needed time to discover new services.
△ Less
Submitted 29 April, 2008;
originally announced April 2008.
-
Performance Analysis of Publish/Subscribe Systems
Authors:
Heithem Abbes,
Christophe Cérin,
Jean-Christophe Dubacq,
Mohamed Jemni
Abstract:
The Desktop Grid offers solutions to overcome several challenges and to answer increasingly needs of scientific computing. Its technology consists mainly in exploiting resources, geographically dispersed, to treat complex applications needing big power of calculation and/or important storage capacity. However, as resources number increases, the need for scalability, self-organisation, dynamic re…
▽ More
The Desktop Grid offers solutions to overcome several challenges and to answer increasingly needs of scientific computing. Its technology consists mainly in exploiting resources, geographically dispersed, to treat complex applications needing big power of calculation and/or important storage capacity. However, as resources number increases, the need for scalability, self-organisation, dynamic reconfigurations, decentralisation and performance becomes more and more essential. Since such properties are exhibited by P2P systems, the convergence of grid computing and P2P computing seems natural. In this context, this paper evaluates the scalability and performance of P2P tools for discovering and registering services. Three protocols are used for this purpose: Bonjour, Avahi and Free-Pastry. We have studied the behaviour of theses protocols related to two criteria: the elapsed time for registrations services and the needed time to discover new services. Our aim is to analyse these results in order to choose the best protocol we can use in order to create a decentralised middleware for desktop grid.
△ Less
Submitted 3 July, 2007;
originally announced July 2007.
-
Methods for Partitioning Data to Improve Parallel Execution Time for Sorting on Heterogeneous Clusters
Authors:
Christophe Cérin,
Jean-Christophe Dubacq,
Jean-Louis Roch,
the SafeScale Collaboration
Abstract:
The aim of the paper is to introduce general techniques in order to optimize the parallel execution time of sorting on a distributed architectures with processors of various speeds. Such an application requires a partitioning step. For uniformly related processors (processors speeds are related by a constant factor), we develop a constant time technique for mastering processor load and execution…
▽ More
The aim of the paper is to introduce general techniques in order to optimize the parallel execution time of sorting on a distributed architectures with processors of various speeds. Such an application requires a partitioning step. For uniformly related processors (processors speeds are related by a constant factor), we develop a constant time technique for mastering processor load and execution time in an heterogeneous environment and also a technique to deal with unknown cost functions. For non uniformly related processors, we use a technique based on dynamic programming. Most of the time, the solutions are in O(p) (p is the number of processors), independent of the problem size n. Consequently, there is a small overhead regarding the problem we deal with but it is inherently limited by the knowing of time complexity of the portion of code following the partitioning.
△ Less
Submitted 10 July, 2006;
originally announced July 2006.
