AI-PiM—Extending the RISC-V processor with Processing-in-Memory functional units for AI inference at the edge of IoT - Frontiers

Abstract

The AI-PiM project introduces a novel extension to the RISC-V processor architecture by integrating specialized Processing-in-Memory (PiM) functional units. This architectural enhancement is specifically designed to accelerate Artificial Intelligence (AI) inference tasks by minimizing the energy and latency costs associated with data movement. The goal is to provide highly efficient, domain-specific hardware solutions essential for deploying demanding AI workloads at the edge of IoT networks.

Report

Key Highlights

• AI-PiM Architecture: The core innovation is the AI-PiM architecture, which marries the flexibility of the RISC-V processor with high-performance, specialized computing.
• Processing-in-Memory (PiM): The system utilizes PiM functional units, enabling computation to occur directly within or near memory arrays, thereby bypassing the traditional memory wall bottleneck.
• Target Application: The design is optimized specifically for AI inference, addressing the energy constraints and performance requirements typical of edge and IoT devices.
• RISC-V Extensibility: This work demonstrates the practical use of RISC-V’s open ISA for creating custom, domain-specific accelerators (DSA).

Technical Details

• Architectural Modification: The RISC-V processor pipeline is extended via custom functional units (FUs) dedicated to PiM operations. This likely involves leveraging the RISC-V Custom Instruction extension mechanism.
• PiM Implementation: The functional units execute operations directly on data stored in local or nearby memory structures, likely targeting operations critical for neural networks (e.g., highly parallel matrix operations or convolutions).
• Optimization Goal: The focus is maximizing energy efficiency and throughput, which are crucial metrics for battery-powered or passively cooled IoT edge devices.
• System Integration: The custom PiM hardware must be seamlessly integrated with the RISC-V core’s instruction fetch and register file mechanisms to operate efficiently under standard program control.

Implications

• Validation of RISC-V: AI-PiM serves as a strong case study proving the utility and versatility of the RISC-V ecosystem for creating highly specialized silicon solutions tailored for modern computing demands.
• Advancing Edge AI: By overcoming the energy penalty of data movement, this architecture significantly lowers the power budget required for complex AI models, enabling the deployment of deeper, more accurate neural networks directly on constrained IoT devices.
• Future of Computing: This research contributes to the growing trend of heterogeneous computing and hardware acceleration, positioning PiM as a viable, practical approach for achieving performance gains where traditional CPU/GPU architectures struggle due to memory bandwidth limitations.
• Competitive Advantage: Provides a pathway for designers to create highly competitive, application-specific integrated circuits (ASICs) optimized for low-power AI inference.