SIMD-CP: SIMD with Redundant Bits Compression and Mixed-Precision Packing for Quantized DNNs

Abstract

SIMD-CP is a novel architectural approach designed to significantly accelerate quantized Deep Neural Networks (DNNs) by enhancing Single Instruction, Multiple Data (SIMD) processing. It integrates redundant bits compression to reduce data transfer overhead, thereby maximizing effective memory bandwidth utilization. Furthermore, the method employs an optimized mixed-precision packing strategy to efficiently handle varying low bit-width data within SIMD registers, boosting computational density.

Report

Key Highlights

• Quantization Acceleration: The primary focus is optimizing the execution speed and efficiency of quantized DNNs, a common requirement for edge and embedded AI.
• SIMD Integration: The technique leverages standard SIMD parallelism while introducing specific optimizations tailored for low-precision data.
• Redundant Bits Compression: A core innovation involves compressing data by eliminating non-critical or redundant bits, reducing the effective data footprint and improving data throughput.
• Mixed-Precision Packing: SIMD-CP supports the heterogeneous data types (e.g., 4-bit weights alongside 8-bit activations) typical of modern quantized models by packing them efficiently into SIMD registers.

Technical Details

• Architecture Modification: SIMD-CP likely requires modifications to the SIMD pipeline, particularly the load/store unit and the input alignment logic, to handle compressed and mixed-precision data streams.
• Compression Mechanism: Redundant Bits Compression implies a scheme where data is stored in a highly compact format (e.g., packing 8-bit values into 7 bits if the most significant bit is known to be zero based on quantization bounds).
• Packing Strategy: The mixed-precision packing mechanism must efficiently align and unpack data elements of different sizes (e.g., 4-bit, 8-bit) from the packed SIMD lane before execution by the Arithmetic Logic Unit (ALU).
• Performance Metric: The goal is to maximize the arithmetic intensity—the number of useful operations performed per byte of data loaded—which is critical for bandwidth-limited inference tasks.

Implications

• RISC-V AI Acceleration: SIMD-CP provides a strong blueprint for developing highly efficient custom extensions or vector unit designs (e.g., RVV extensions) specifically optimized for AI/ML workloads on RISC-V cores.
• Energy Efficiency: By significantly reducing memory access and data movement (due to compression), SIMD-CP directly contributes to lower power consumption, making RISC-V processors highly competitive in energy-constrained edge and IoT applications.
• Domain-Specific Architecture (DSA) Enhancement: This research validates the utility of micro-architectural tailoring for post-quantization benefits, potentially leading to new, dedicated RISC-V instruction set additions for bit-manipulation, packing, and decompression, increasing performance density for quantized DNN operations.