A Precision-Scalable RISC-V DNN Processor with On-Device Learning Capability at the Extreme Edge

Abstract

This paper presents a precision-scalable RISC-V DNN processor designed for extreme edge devices, tackling the challenges of efficient inference and enabling on-device learning. The architecture supports diverse fixed-point inference precisions ranging from 2-bit to 16-bit, while integrating enhanced FP16 support specifically for privacy-preserving model updating. Utilizing optimization techniques like multiplier reuse, the processor achieves 1.6x to 14.6x improvement in inference throughput and energy efficiency, and a 16.5x higher FP throughput for learning compared to prior art like XpulpNN.

Report

Structured Report: A Precision-Scalable RISC-V DNN Processor

Key Highlights

• Extreme Edge Focus: The processor is specifically optimized for extreme edge platforms, prioritizing low energy, memory, and computing resources required for in-vehicle smart devices and similar applications.
• Precision Scalability: The design inherently supports a wide range of quantized DNN inference precisions, from highly compressed 2-bit fixed-point up to 16-bit fixed-point.
• On-Device Learning Enabled: Unlike many edge devices that lack the necessary precision, this processor includes robust FP16 (half-precision floating-point) support, crucial for implementing on-device learning and improving model accuracy while preserving data privacy.
• Superior Performance Metrics: Experimental results demonstrated significant gains, showing a $1.6\sim 14.6\times$ improvement in inference throughput and energy efficiency compared to the prior state-of-the-art accelerator, XpulpNN.
• FP Throughput Boost: The processor achieved a $16.5\times$ higher FP throughput specifically for on-device learning tasks.

Technical Details

• Base Architecture: RISC-V Deep Neural Network (DNN) Processor.
• Inference Precision: Variable fixed-point quantization, supporting 2-bit, 4-bit, 8-bit, and 16-bit operations.
• Learning Precision: FP16 (16-bit Floating Point) operations are integrated to handle the gradients and weight updates required for on-device learning.
• Hardware Optimizations: Key hardware methods employed to improve utilization and efficiency include:
  • FP16 multiplier reuse.
  • Multi-precision integer multiplier reuse (to handle varying fixed-point bit-widths efficiently).
  • Balanced mapping of FPGA resources.
• Validation Platform: The processor was benchmarked using the Xilinx ZCU102 FPGA.

Implications

• Advancing RISC-V AI: This development furthers the utility of the open-source RISC-V ecosystem by providing a highly specialized and efficient core for extreme edge AI computation, closing the gap with proprietary architectures.
• Enabling Edge Intelligence: By providing simultaneous support for high-efficiency quantized inference and robust FP16 learning, the processor overcomes a major limitation in current edge hardware, allowing developers to deploy models that can continuously learn and adapt locally.
• Standard for Flexible Quantization: The precision-scalable architecture sets a new standard for handling the increasingly diverse quantization levels found in modern, highly optimized DNNs, ensuring maximum efficiency regardless of the chosen model compression strategy.
• Performance Benchmark: The measured performance gains (up to 14.6x vs. XpulpNN) position this design as a leading candidate for next-generation energy-constrained AI accelerators.