Accurate and Scalable Many-Node Simulation
Authors:
Stijn Eyerman,
Wim Heirman,
Kristof Du Bois,
Ibrahim Hur
Abstract:
Accurate performance estimation of future many-node machines is challenging because it requires detailed simulation models of both node and network. However, simulating the full system in detail is unfeasible in terms of compute and memory resources. State-of-the-art techniques use a two-phase approach that combines detailed simulation of a single node with network-only simulation of the full syst…
▽ More
Accurate performance estimation of future many-node machines is challenging because it requires detailed simulation models of both node and network. However, simulating the full system in detail is unfeasible in terms of compute and memory resources. State-of-the-art techniques use a two-phase approach that combines detailed simulation of a single node with network-only simulation of the full system. We show that these techniques, where the detailed node simulation is done in isolation, are inaccurate because they ignore two important node-level effects: compute time variability, and inter-node communication.
We propose a novel three-stage simulation method to allow scalable and accurate many-node simulation, combining native profiling, detailed node simulation and high-level network simulation. By including timing variability and the impact of external nodes, our method leads to more accurate estimates. We validate our technique against measurements on a multi-node cluster, and report an average 6.7% error on 64 nodes (maximum error of 12%), compared to on average 27% error and up to 54% when timing variability and the scaling overhead are ignored. At higher node counts, the prediction error of ignoring variable timings and scaling overhead continues to increase compared to our technique, and may lead to selecting the wrong optimal cluster configuration.
Using our technique, we are able to accurately project performance to thousands of nodes within a day of simulation time, using only a single or a few simulation hosts. Our method can be used to quickly explore large many-node design spaces, including node micro-architecture, node count and network configuration.
△ Less
Submitted 18 January, 2024;
originally announced January 2024.
PIUMA: Programmable Integrated Unified Memory Architecture
Authors:
Sriram Aananthakrishnan,
Shamsul Abedin,
Vincent Cave,
Fabio Checconi,
Kristof Du Bois,
Stijn Eyerman,
Joshua B. Fryman,
Wim Heirman,
Jason Howard,
Ibrahim Hur,
Samkit Jain,
Marek M. ~Landowski,
Kevin Ma,
Jarrod Nelson,
Robert Pawlowski,
Fabrizio Petrini Sebastian Szkoda,
Sanjaya Tayal,
Jesmin Jahan Tithi,
Yves Vandriessche
Abstract:
High performance large scale graph analytics are essential to timely analyze relationships in big data sets. Conventional processor architectures suffer from inefficient resource usage and bad scaling on those workloads. To enable efficient and scalable graph analysis, Intel developed the Programmable Integrated Unified Memory Architecture (PIUMA) as a part of the DARPA Hierarchical Identify Verif…
▽ More
High performance large scale graph analytics are essential to timely analyze relationships in big data sets. Conventional processor architectures suffer from inefficient resource usage and bad scaling on those workloads. To enable efficient and scalable graph analysis, Intel developed the Programmable Integrated Unified Memory Architecture (PIUMA) as a part of the DARPA Hierarchical Identify Verify Exploit (HIVE) program. PIUMA consists of many multi-threaded cores, fine-grained memory and network accesses, a globally shared address space, powerful offload engines and a tightly integrated optical interconnection network. By utilizing co-packaged optical silicon photonics and extending the on-chip mesh protocol directly to the optical fabric, all PIUMA chips in a system are glued together in a large virtual die which allows for extremely low socket-to-socket latencies even as the system scales to thousands of sockets. Performance estimations project that a PIUMA node will outperform a conventional compute node by one to two orders of magnitude. Furthermore, PIUMA continues to scale across multiple nodes, which is a challenge in conventional multi-node setups.
This paper presents the PIUMA architecture, and documents our experience in designing and building a prototype chip and its bring-up process. We summarize the methodology for our co-design of the architecture together with the software stack using simulation tools and FPGA emulation. These tools provided early performance estimations of realistic applications and allowed us to implement many optimizations across the hardware, compilers, libraries and applications. We built the PIUMA chip as a 316mm2 7nm FinFET CMOS die and constructed a 16-node system. PIUMA silicon has successfully powered on demonstrating key aspects of the architecture, some of which will be incorporated into future Intel products.
△ Less
Submitted 28 February, 2025; v1 submitted 13 October, 2020;
originally announced October 2020.
