xTern: Energy-Efficient Ternary Neural Network Inference on RISC-V-Based Edge Systems

Abstract

xTern is a lightweight RISC-V instruction set architecture extension designed to accelerate Ternary Neural Network (TNN) inference on general-purpose edge cores. This ISA extension, complemented by optimized kernels, achieves 67% higher throughput and a 57.1% improvement in energy efficiency with only a 0.9% area overhead. By enabling efficient TNN deployment, xTern allows RISC-V platforms to achieve up to 1.6 percentage points higher classification accuracy than comparable 2-bit networks at equal inference latency.

Report

Key Highlights

• Novel Acceleration Method: xTern accelerates Ternary Neural Network (TNN) inference by using a lightweight RISC-V ISA extension, avoiding the need for specialized hardware accelerators.
• High Throughput Gain: The system demonstrates 67% higher throughput compared to 2-bit network equivalents.
• Energy Efficiency: xTern achieves a 57.1% improvement in energy efficiency.
• Minimal Overhead: Integration into an octa-core compute cluster incurs a minimal silicon area overhead of just 0.9% with no measurable impact on timing.
• Accuracy Improvement: End-to-end benchmarks show TNNs deployed via xTern achieve up to 1.6 percentage points higher CIFAR-10 classification accuracy than 2-bit networks.

Technical Details

• Architecture Extension: xTern is a lightweight ISA extension specifically targeted at optimizing TNN calculations on general-purpose RISC-V cores.
• Power/Energy Metrics: The power consumption is only marginally increased by 5.2%, leading directly to the high overall energy efficiency gain.
• Implementation Environment: The extension was integrated and tested within an octa-core compute cluster.
• Software Complement: Optimized kernels were developed to leverage the new xTern instructions.
• Quantization Scheme: The acceleration targets Ternary Neural Networks, which offer a superior accuracy-energy trade-off compared to Binary Neural Networks (BNNs).

Implications

• Democratization of TNNs: xTern removes the traditional barrier of needing complex, specialized accelerators for TNN deployment, making high-efficiency neural networks accessible on standard RISC-V cores.
• Enhancing Edge AI Competitiveness: This technology significantly boosts the performance and efficiency of RISC-V-based ultra-low-power edge AI platforms, allowing them to handle more complex models (TNNs) efficiently.
• RISC-V Ecosystem Growth: The introduction of specialized, yet lightweight, ISA extensions like xTern showcases the flexibility and extensibility of the RISC-V architecture, encouraging further innovation for domain-specific acceleration on general-purpose hardware.
• Improved Accuracy on Edge Devices: By enabling high-efficiency TNNs, edge devices can run models that maintain high classification accuracy (demonstrated up to 1.6 pp higher than 2-bit networks) without sacrificing battery life or increasing area cost substantially.