ColibriES: A Milliwatts RISC-V Based Embedded System Leveraging Neuromorphic and Neural Networks Hardware Accelerators for Low-Latency Closed-loop Control Applications

by   Georg Rutishauser, et al.

End-to-end event-based computation has the potential to push the envelope in latency and energy efficiency for edge AI applications. Unfortunately, event-based sensors (e.g., DVS cameras) and neuromorphic spike-based processors (e.g., Loihi) have been designed in a decoupled fashion, thereby missing major streamlining opportunities. This paper presents ColibriES, the first-ever neuromorphic hardware embedded system platform with dedicated event-sensor interfaces and full processing pipelines. ColibriES includes event and frame interfaces and data processing, aiming at efficient and long-life embedded systems in edge scenarios. ColibriES is based on the Kraken system-on-chip and contains a heterogeneous parallel ultra-low power (PULP) processor, frame-based and event-based camera interfaces, and two hardware accelerators for the computation of both event-based spiking neural networks and frame-based ternary convolutional neural networks. This paper explores and accurately evaluates the performance of event data processing on the example of gesture recognition on ColibriES, as the first step of full-system evaluation. In our experiments, we demonstrate a chip energy consumption of 7.7 and latency of 164.5 of each inference with the DVS Gesture event data set as an example for closed-loop data processing, showcasing the potential of ColibriES for battery-powered applications such as wearable devices and UAVs that require low-latency closed-loop control.


page 1

page 2

page 3

page 4


ColibriUAV: An Ultra-Fast, Energy-Efficient Neuromorphic Edge Processing UAV-Platform with Event-Based and Frame-Based Cameras

The interest in dynamic vision sensor (DVS)-powered unmanned aerial vehi...

Evaluating Spiking Neural Network On Neuromorphic Platform For Human Activity Recognition

Energy efficiency and low latency are crucial requirements for designing...

Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline

Edge computing solutions that enable the extraction of high level inform...

A Self-Driving Robot Using Deep Convolutional Neural Networks on Neuromorphic Hardware

Neuromorphic computing is a promising solution for reducing the size, we...

Neuromorphic Neuromodulation: Towards the next generation of on-device AI-revolution in electroceuticals

Neuromodulation techniques have emerged as promising approaches for trea...

Closed-Loop Neural Prostheses with On-Chip Intelligence: A Review and A Low-Latency Machine Learning Model for Brain State Detection

The application of closed-loop approaches in systems neuroscience and th...

CarSNN: An Efficient Spiking Neural Network for Event-Based Autonomous Cars on the Loihi Neuromorphic Research Processor

Autonomous Driving (AD) related features provide new forms of mobility t...

Please sign up or login with your details

Forgot password? Click here to reset