TinyVers: A Tiny Versatile System-on-chip with State-Retentive eMRAM for ML Inference at the Extreme Edge

Abstract

TinyVers is an ultra-low power, versatile System-on-Chip designed for Machine Learning inference at the Extreme Edge, integrating a RISC-V host processor and a dataflow reconfigurable ML accelerator. The SoC leverages state-retentive embedded MRAM (eMRAM) for efficient boot code and parameter retention, enabling aggressive duty-cycling for always-on smart sensing applications. This design achieves a peak efficiency of 17 TOPS/W, managing to operate continuously at under 230 µW, with deep sleep modes consuming only 1.7 µW.

Report

Key Highlights

• Target Audience: Designed specifically for extreme edge devices and Internet-of-Things (IoT) nodes requiring ultra-low power, always-on processing and on-demand sampling/processing.
• Versatile Design: Achieves multi-modal support for diverse ML workloads (e.g., voice recognition, machine monitoring) by exploiting dataflow reconfiguration.
• State Retention: Features state-retentive eMRAM for retaining boot code and critical ML parameters, greatly aiding in efficient duty-cycling and fast wake-up.
• Extreme Low Power: Achieves a power consumption range from an aggressive 1.7 µW (deep sleep) up to 20 mW (peak operation).
• High Efficiency: The ML accelerator demonstrates a maximum energy efficiency of 17 TOPS/W.

Technical Details

• Core Components: The SoC comprises four main elements: a RISC-V host processor, a dataflow reconfigurable ML accelerator, a 1.7 µW deep sleep wake-up controller, and eMRAM memory.
• ML Performance: The SoC is capable of performing up to 17.6 GOPS.
• ML Workload Efficiency: When executing continuous ML models, the chip maintains energy efficiency between 1-2 TOPS/W while keeping power consumption below 230 µW.
• Duty-Cycling Performance: In a practical duty-cycling use case (like machine monitoring), the average power consumption is reduced to below 10 µW.
• Innovation Method: The architecture utilizes dataflow reconfiguration to provide flexibility and aggressive on-chip power management for optimal duty-cycling.

Implications

• Advancing Extreme Edge AI: TinyVers sets a new benchmark for combining versatility and energy efficiency in extreme edge hardware, proving that complex ML tasks can be managed with sub-milliwatt power budgets.
• Validation of RISC-V in Specialized SoCs: The inclusion of the RISC-V processor as the host confirms its role as a flexible, low-power control core within highly specialized accelerators designed for application-specific tasks.
• Leveraging Non-Volatile Memory (eMRAM): The use of state-retentive eMRAM is a critical enabler for true 'always-on' sensing. It drastically reduces the energy and time costs typically associated with refreshing or reloading large ML models during deep sleep wake cycles, making aggressive duty-cycling practical.
• Ecosystem Growth: This design demonstrates a pathway for developing standardized, yet reconfigurable, AI accelerators that can serve multiple verticals (industrial IoT, smart homes, wearable tech), driving broader adoption of efficient edge computing solutions.