IndexMAC: A Custom RISC-V Vector Instruction to Accelerate Structured-Sparse Matrix Multiplications

Abstract

This work introduces IndexMAC, a custom vector index-multiply-accumulate instruction proposed for RISC-V processors, designed to accelerate structured-sparse matrix multiplications (SSMM) critical to modern Machine Learning (ML) applications. IndexMAC enables low-cost indirect reads from the vector register file, substantially improving data locality and reducing memory traffic without significant hardware overhead. Integrated into a decoupled RISC-V vector processor, this instruction demonstrates speedups between 1.80x and 2.14x compared to state-of-the-art vectorized kernels executing sparse CNN layers.

Report

Key Highlights

• Core Innovation: Introduction of IndexMAC, a custom RISC-V vector instruction (Index-Multiply-Accumulate) designed specifically to accelerate Structured-Sparse Matrix Multiplications (SSMM).
• Performance Gain: IndexMAC implementation achieved significant speedups ranging from 1.80x to 2.14x when compared to state-of-the-art vectorized kernels, evaluated using sparse layers of Convolutional Neural Networks (CNNs).
• Hardware Efficiency: The instruction was successfully integrated into a decoupled RISC-V vector processor requiring only negligible additional hardware cost.
• Efficiency Goal: The instruction incorporates the simplicity of structured sparsity into vector execution to reduce unnecessary memory traffic and improve overall performance.

Technical Details

• Instruction Type: IndexMAC is a vector instruction that performs index-multiply-accumulate operations.
• Functionality: Its primary mechanism is enabling low-cost indirect reads directly from the vector register file (VRF).
• Optimization: By performing indirect reads locally, the instruction mitigates the high overhead typically associated with irregular memory access patterns in sparse data handling, thereby increasing data locality.
• Context: The innovation targets the acceleration of ML models (training and inference) whose performance relies heavily on efficient matrix multiplications.

Implications

• Enhanced RISC-V ML Capabilities: IndexMAC demonstrates a successful approach to tailoring the extensible RISC-V architecture for high-performance ML workloads, particularly those leveraging structured sparsity.
• Vector Processor Optimization: The instruction allows vector processors to efficiently handle SSMM, challenging the necessity for completely customized matrix engines for many sparse workloads.
• Industry Relevance: Since structured sparsity is a common technique used to prune complexity in modern ML models, IndexMAC offers a practical, high-efficiency path for deploying sparse models on RISC-V vector hardware.
• Architectural Efficiency: It provides substantial performance improvements (up to 2.14x) while maintaining a low hardware cost, making it an attractive extension for embedded or resource-constrained RISC-V implementations.