Abstract

This study conducts a comprehensive energy consumption benchmark of deep learning inference models running on the 64-core SOPHON SG2042 RISC-V server CPU. The research compared leading AI frameworks, including PyTorch, ONNX Runtime, and TensorFlow, to assess their efficiency on the RV64 architecture. Key findings indicate that frameworks utilizing the XNNPACK back-end (ONNX Runtime and TensorFlow) achieve significantly lower energy consumption compared to PyTorch, which was compiled using the native OpenBLAS back-end.

Report

Structured Report: Exploring energy consumption of AI frameworks on a 64-core RV64 Server CPU

Key Highlights

• Benchmarking Focus: The research performs a comprehensive benchmark analysis specifically targeting the energy consumption of popular Machine Learning (ML) applications and deep learning inference models.
• Hardware Platform: The study utilizes a specific, high-core-count RISC-V platform: the 64-core SOPHON SG2042 RV64 Server CPU.
• Framework Comparison: Three leading AI frameworks were tested: PyTorch, ONNX Runtime, and TensorFlow.
• Principal Finding: Frameworks compiled using the optimized XNNPACK back-end (namely ONNX Runtime and TensorFlow) demonstrated superior energy efficiency, consuming less energy than PyTorch, which relied on the native OpenBLAS back-end.

Technical Details

• Architecture: RISC-V (RV64 Instruction Set Architecture).
• Testbed CPU: SOPHON SG2042 (a 64-core RISC-V server-class processor).
• Test Application: Deep learning inference models.
• Frameworks Tested: PyTorch, ONNX Runtime, and TensorFlow.
• Observed Back-ends: The difference in energy efficiency was linked directly to the underlying computational libraries used:
  • More Efficient Back-end: XNNPACK (used by ONNX Runtime and TensorFlow).
  • Less Efficient Back-end: OpenBLAS (used in the PyTorch compilation).

Implications

• Validating RISC-V for AI: This study provides crucial, specific data affirming the potential of high-core-count RISC-V CPUs for energy-efficient AI and ML computation in server and cluster environments.
• Software Optimization Priority: The findings clearly emphasize that specialized, optimized software libraries (like XNNPACK) are critically important for harnessing the energy benefits of the RISC-V ISA; raw architectural efficiency is not enough without proper software implementation.
• Ecosystem Development Roadmap: The results offer a clear roadmap for software developers and vendors within the RISC-V ecosystem, indicating where optimization efforts must be focused—specifically on core computational back-ends—to make RISC-V competitive for demanding, power-sensitive deep learning workloads.