FPGA-Accelerated RISC-V ISA Extensions for Efficient Neural Network Inference on Edge Devices

Abstract

This paper presents novel FPGA-accelerated RISC-V instruction set architecture (ISA) extensions designed for efficient neural network inference on resource-constrained edge devices. The customized RISC-V core, featuring four domain-specific ISA extensions and integrated accelerators on the Xilinx PYNQ-Z2, achieves substantial performance gains. The complete system demonstrated a 2.14x average latency speedup and a 49.1% energy reduction compared to an ARM Cortex-A9 software baseline across multiple benchmark models.

Report

Key Highlights

• Core Innovation: Development of a custom RISC-V core featuring novel ISA extensions specifically designed for accelerating neural network inference.
• Performance Metrics: Achieved a significant 2.14x average latency speedup and 49.1% energy reduction when benchmarked against an ARM Cortex-A9 software baseline.
• Target Domain: Focuses on improving computational performance and energy efficiency for Edge AI deployment on resource-constrained devices.
• Validation: Performance was validated using physical hardware measurements across four key benchmark models: MobileNet V2, ResNet-18, EfficientNet Lite, and YOLO Tiny.
• Framework: Establishes a reproducible methodology for utilizing ISA-guided FPGA acceleration as a viable alternative to fixed-function ASICs.

Technical Details

• Platform: Implemented and validated on the Xilinx PYNQ-Z2 FPGA platform.
• ISA Extensions: Four novel instruction set extensions were introduced into the custom RISC-V core, targeting critical NN operations:
  • FPGA.VCONV (Vector Convolution)
  • FPGA.GEMM (General Matrix Multiplication)
  • FPGA.RELU (Activation Function)
  • FPGA.CUSTOM (Custom operation)
• Operating Specifications: The hardware implementation successfully closed timing at 50 MHz, reporting a +12.793 ns worst negative slack.
• Resource Utilization:
  • Base RISC-V Core usage: 0.43% LUTs and 11.4% BRAM.
  • Accelerator usage (when active): 38.8% DSPs.
• System Integration: Confirmed functionality of the 64 KB BRAM memory interface and AXI interconnect.

Implications

• RISC-V Ecosystem: This work serves as a strong validation of the RISC-V architecture's primary strength: extensibility. By proving that custom ISA extensions can yield substantial, measurable real-world speedups on commercial FPGAs, it encourages wider adoption for domain-specific accelerators (DSA).
• Edge AI Competitiveness: The demonstrated performance improvement (over 2x speedup and near 50% energy savings) provides a high-efficiency alternative to traditional ARM software implementations, making RISC-V a serious contender in the competitive edge computing market.
• Hardware/Software Co-Design: The use of ISA extensions creates a powerful interface, allowing software/compiler developers to directly leverage the specialized FPGA hardware functionality, effectively bridging the gap between hardware acceleration and software programmability.
• Reproducibility: Establishing a verifiable framework using physical hardware measurements increases trust and facilitates future research and commercial development in RISC-V based heterogenous computing.