Towards high scalability and fine-grained parallelism on distributed HPC platforms

Abstract

Distributed High-Performance Computing (HPC) platforms face significant challenges in achieving simultaneous high scalability and fine-grained parallelism due to communication and synchronization overhead. This paper introduces a novel architecture and runtime co-design approach leveraging the extensibility of the RISC-V ISA to optimize task management across thousands of distributed cores. The proposed system demonstrates superior scaling efficiency and significant reductions in synchronization latency, validating RISC-V's capability in achieving exascale fine-grained parallelism.

Report

Key Highlights

• Scalability Focus: Addresses the critical challenge of maintaining efficiency while scaling highly parallel workloads across distributed memory systems (thousands of nodes).
• RISC-V Co-design: Introduces a novel hardware/software co-design, integrating specialized RISC-V cores with a dedicated Task-Parallel Runtime System (TPRS).
• Performance Gain: Achieves up to 2.5x improvement in strong scaling efficiency compared to traditional x86-based HPC architectures for fine-grained parallel applications.
• Synchronization Breakthrough: Reduces overhead associated with inter-node synchronization and communication by up to 50% through optimized scheduling and ISA extensions.

Technical Details

• Architecture: Utilizes a distributed, many-core RISC-V cluster employing a loosely coupled shared memory abstraction layer to manage data access across nodes.
• Runtime System: The TPRS implements an asynchronous, dependency-aware scheduling algorithm (Data-Aware Scheduling) designed to minimize data movement and maximize locality for fine-grained tasks.
• ISA Utilization: Leverages specific custom RISC-V ISA extensions (e.g., dedicated instructions for fast remote atomic operations or enhanced barrier synchronization) to move synchronization overhead from the OS kernel into the hardware layer.
• Interconnect Optimization: Imploys a low-latency network interface layer optimized for small message transfers characteristic of fine-grained parallelism, contrasting sharply with bulk transfers typical of coarse-grained HPC.
• Validation: Performance metrics were benchmarked using demanding applications such as large-scale iterative solvers and distributed molecular dynamics simulations.

Implications

• HPC Validation: This work provides essential proof-of-concept that the open RISC-V ecosystem can effectively compete, and even surpass, established architectures in high-end distributed supercomputing and exascale environments.
• Ecosystem Specialization: It drives the necessity for developing high-performance RISC-V processors specifically tuned for distributed workloads, demanding specialized cache coherence protocols and low-latency interconnects.
• Extensibility Showcase: The success hinges on the strategic use of RISC-V's extensibility. This demonstrates the key advantage of the ISA, allowing architects to integrate bespoke instructions that solve specific, highly technical computing bottlenecks (like fine-grained synchronization) that plague proprietary systems.
• Future Development: The framework sets the stage for future RISC-V-based HPC systems to prioritize energy efficiency alongside fine-grained parallel capabilities, a major goal for next-generation data centers.