A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference

Abstract

This work introduces the MPIC (Mixed Precision Inference Core), a novel RISC-V processor designed to efficiently execute fine-grained mixed-precision Quantized Neural Networks (QNNs) on extreme-edge microcontrollers. It solves the challenge of supporting sub-byte and asymmetric quantization without excessive ISA complexity by utilizing dynamic, status-based SIMD instructions. MPIC achieves significant performance gains (up to 11.7x) and dramatically higher energy efficiency (up to 155x) compared to commercial Cortex-M microcontrollers.

Report

Key Highlights

• Target: Extreme-edge devices and Microcontrollers (MCUs) for low bit-width Quantized Neural Networks (QNNs).
• Innovation: Introduction of MPIC (Mixed Precision Inference Core), a novel RISC-V core extension based on the open-source RI5CY core.
• Key Feature: Implementation of status-based SIMD instructions to dynamically set operand precision (16-, 8-, 4-, and 2-bit) without requiring extra opcodes or increasing decode stage complexity.
• Performance Gain: MPIC achieves 1.1-4.9x better performance and energy efficiency than software-based mixed-precision on RI5CY.
• Competitive Edge: Delivers 3.6-11.7x better performance and 41-155x higher efficiency compared to commercially available Cortex-M4 and M7 microcontrollers.

Technical Details

• Core Architecture: Based on the existing open-source RI5CY core (a RISC-V implementation).
• Quantization Method: Supports fine-grained asymmetric quantization, where weights and activations can be assigned different, low bit-widths (sub-byte) on a tensor-by-tensor basis.
• ISA Mechanism: The core uses a dynamic approach where operand precision is configured via a core status register, rather than being explicitly encoded in the instruction set architecture (ISA).
• Precision Formats: Full hardware support for mixed-precision QNN inference with operand combinations at 16-bit, 8-bit, 4-bit, and 2-bit precision.

Implications

• RISC-V Specialization: This work showcases the inherent flexibility of the RISC-V ISA to be extended and specialized for high-efficiency domain-specific acceleration (like DNN inference) without significant architectural overhead.
• Democratization of Edge AI: By achieving massive gains in energy efficiency (up to 155x), MPIC enables the deployment of increasingly complex machine learning models on highly constrained, battery-powered IoT devices previously limited by memory footprint and computational cost.
• Challenge to Incumbents: The documented performance and efficiency lead over established commercial embedded architectures (Cortex-M4/M7) validates RISC-V's potential to dominate the extreme-edge microcontroller market for AI applications.