ColibriES: A Milliwatts RISC-V Based Embedded System Leveraging Neuromorphic and Neural Networks Hardware Accelerators for Low-Latency Closed-loop Control Applications

Abstract

ColibriES is a novel milliwatt RISC-V based embedded system featuring the first-ever dedicated event-sensor interfaces and full neuromorphic processing pipelines. Built upon the Kraken System-on-Chip (SoC), it integrates a heterogeneous PULP processor and dual hardware accelerators for event-based Spiking Neural Networks (SNNs) and frame-based Ternary Convolutional Neural Networks (TCNNs). This architecture demonstrates robust efficiency for closed-loop control, achieving 7.7 mJ energy and 164.5 ms latency during gesture recognition tasks, making it ideal for battery-powered edge AI.

Report

Key Highlights

• First Integrated Neuromorphic Platform: ColibriES is presented as the first neuromorphic hardware embedded system platform that features dedicated event-sensor interfaces and complete processing pipelines, specifically designed to bridge the gap between event-based sensors and spike-based processors.
• RISC-V/PULP Core: The system is based on the Kraken SoC, leveraging a heterogeneous Parallel Ultra-Low Power (PULP) processor built on the RISC-V instruction set architecture.
• Dual Hardware Acceleration: It incorporates two specialized hardware accelerators: one for event-based SNN computation and one for frame-based TCNN computation.
• Ultra-Low Energy Efficiency: Performance testing using the DVS Gesture event dataset demonstrated an extremely low chip energy consumption of 7.7 mJ per inference with a latency of 164.5 ms, suitable for long-life embedded applications.

Technical Details

• Architecture Base: The system utilizes the Kraken System-on-Chip, which hosts a heterogeneous Parallel Ultra-Low Power (PULP) processor cluster.
• Sensor Interfaces: ColibriES supports dual data input modalities, including interfaces for both standard frame-based data and high-efficiency event-based data (e.g., from DVS cameras).
• Computation Strategy: The design prioritizes "end-to-end event-based computation" to maximize streamlining opportunities previously missed by decoupled designs, thereby pushing the envelope in latency and energy efficiency.
• Target Applications: The platform is optimized for low-latency closed-loop control scenarios, specifically targeting applications like wearable devices and Unmanned Aerial Vehicles (UAVs).

Implications

• Validation of RISC-V in Edge AI: ColibriES proves that RISC-V based platforms, specifically the PULP architecture, are mature enough to host complex, heterogeneous systems required for demanding, milliwatt-level neuromorphic computing.
• Accelerating Neuromorphic Adoption: By providing a fully integrated hardware solution for both event-sensor input and SNN processing, ColibriES addresses a critical hardware infrastructure hurdle, potentially accelerating the deployment and commercialization of biologically-inspired AI.
• Enabling Ultra-Efficient Control: The documented ultra-low energy and latency performance (7.7 mJ) sets a new standard for closed-loop control in energy-constrained edge devices, expanding the possibility for sophisticated, real-time AI processing in battery-dependent applications.