Abstract

This work introduces a critical extension to the PULP-NN library optimized for accelerating mixed-precision Quantized Neural Networks (QNNs) on ultra-low-power RISC-V edge devices. Featuring 27 highly optimized kernels supporting 8-bit, 4-bit, and 2-bit permutations, the solution significantly reduces the memory footprint of Deep Neural Networks (DNNs). Benchmarked on an 8-core GAP-8 PULP cluster, the approach demonstrates superior performance, running 21x to 25x faster with 15x to 21x better energy efficiency than an equivalent ARM Cortex M7 microcontroller.

Report

Structured Analysis Report

Key Highlights

• Core Innovation: An extension of the PULP-NN library designed specifically to accelerate mixed-precision Quantized Neural Networks (QNNs).
• Benefit of Mixed Precision: Significantly shrinks the memory footprint of DNNs while maintaining negligible accuracy loss.
• High Performance: Achieves a peak performance of 16 MACs/cycle across 8 RISC-V cores.
• Competitive Advantage: The solution runs 21x to 25x faster and offers 15x to 21x better energy efficiency compared to the STM32H7 (powered by an ARM Cortex M7 processor).
• Target Platform: Parallel Ultra-Low-Power (PULP) clusters utilizing RISC-V based processors.

Technical Details

• Library: Optimized extension of the existing PULP-NN library.
• Kernel Structure: The library comprises 27 distinct kernels, covering every permutation of precision for input feature maps, weights, and output feature maps.
• Supported Precisions: Supports 8-bit, 4-bit, and 2-bit quantization levels.
• Instruction Set Architecture (ISA): Targets the RV32IMCXpulpV2 ISA, which includes custom DSP (Digital Signal Processing) extensions necessary for optimized QNN operations.
• Benchmark Hardware: Benchmarking was conducted on the 8-cores GAP-8 PULP cluster.

Implications

• RISC-V Validation for AI Edge: This work validates the effectiveness of RISC-V clusters, specifically the PULP architecture, for high-performance, ultra-low-power AI inference at the extreme edge. It positions RISC-V as a direct, highly competitive alternative to established microcontrollers like the ARM Cortex M7.
• Leveraging Custom Extensions: The success relies heavily on utilizing the XpulpV2 DSP extensions of the RISC-V ISA. This demonstrates the power of the RISC-V open architecture model, allowing specialized extensions to maximize performance for domain-specific tasks (like mixed-precision AI).
• Advancing QNN Deployment: By providing optimized software supporting complex techniques like mixed-precision quantization, the paper lowers the barrier for deploying complex, memory-constrained neural networks onto resource-limited edge devices, driving efficiency improvements across the tech ecosystem.