Full-stack evaluation of Machine Learning inference workloads for RISC-V systems

Abstract

This study conducts a full-stack performance evaluation of diverse machine learning inference workloads on RISC-V architectures using the open-source gem5 architectural simulator. Leveraging an MLIR-based compilation toolchain, the research maps complex deep learning kernels to the hardware and presents detailed benchmarking results. Crucially, the paper also identifies and discusses current limitations within gem5 for RISC-V simulation, offering necessary insights for toolchain and simulator refinement.

Report

Key Highlights

• Presents a full-stack performance evaluation focusing specifically on Machine Learning (ML) inference workloads for RISC-V systems.
• The evaluation utilizes the gem5 open-source architectural simulator, eliminating the need for costly physical prototypes.
• Benchmarking relies on an open-source compilation toolchain built on Multi-Level Intermediate Representation (MLIR).
• The study delivers comprehensive benchmarking results for deep learning inference performance metrics.
• It identifies and outlines specific limitations and deficiencies currently present in gem5 when simulating RISC-V architectures, aiding future development.

Technical Details

• Target Platform: RISC-V architectures.
• Workloads: A wide array of machine learning/deep learning inference workloads, focusing on diverse computational kernels.
• Evaluation Method: Architectural simulation using the open-source tool gem5.
• Compilation Stack: An open-source compilation toolchain leveraging MLIR (Multi-Level Intermediate Representation) is used to map deep learning algorithms efficiently onto the target hardware.
• Objective: Analyze performance metrics and assess the completeness and accuracy of the simulation environment for ML workloads.

Implications

• Accelerating RISC-V ML Deployment: By providing concrete performance data and a validated software stack (MLIR-based toolchain), the study advances RISC-V's readiness for pervasive deep learning and AI applications, especially in edge computing.
• Cost-Effective Architectural Exploration: The reliance on architectural simulators like gem5 allows researchers to explore innovative RISC-V architectural concepts rapidly and cost-effectively, significantly speeding up the design iteration process.
• Improving Open-Source Tools: Highlighting specific current limitations within gem5 for RISC-V simulation directly informs and prioritizes development efforts within the open-source community, leading to more accurate and robust simulation tools for the architecture.
• Validating the Software Stack: The use of an MLIR-based toolchain confirms its efficacy in handling the complex and diverse computational kernels required by modern deep learning algorithms, thereby strengthening the foundational software infrastructure for RISC-V ML development.