Adaptive Subsampling for ROI-based Visual Tracking: Algorithms and FPGA Implementation

by   Odrika Iqbal, et al.

There is tremendous scope for improving the energy efficiency of embedded vision systems by incorporating programmable region-of-interest (ROI) readout in the image sensor design. In this work, we study how ROI programmability can be leveraged for tracking applications by anticipating where the ROI will be located in future frames and switching pixels off outside of this region. We refer to this process of ROI prediction and corresponding sensor configuration as adaptive subsampling. Our adaptive subsampling algorithms comprise an object detector and an ROI predictor (Kalman filter) which operate in conjunction to optimize the energy efficiency of the vision pipeline with the end task being object tracking. To further facilitate the implementation of our adaptive algorithms in real life, we select a candidate algorithm and map it onto an FPGA. Leveraging Xilinx Vitis AI tools, we designed and accelerated a YOLO object detector-based adaptive subsampling algorithm. In order to further improve the algorithm post-deployment, we evaluated several competing baselines on the OTB100 and LaSOT datasets. We found that coupling the ECO tracker with the Kalman filter has a competitive AUC score of 0.4568 and 0.3471 on the OTB100 and LaSOT datasets respectively. Further, the power efficiency of this algorithm is on par with, and in a couple of instances superior to, the other baselines. The ECO-based algorithm incurs a power consumption of approximately 4 W averaged across both datasets while the YOLO-based approach requires power consumption of approximately 6 W (as per our power consumption model). In terms of accuracy-latency tradeoff, the ECO-based algorithm provides near-real-time performance (19.23 FPS) while managing to attain competitive tracking precision.


page 1

page 2

page 3

page 6

page 10

page 11

page 12

page 14


A Novel Low-cost FPGA-based Real-time Object Tracking System

In current visual object tracking system, the CPU or GPU-based visual ob...

Expanding a robot's life: Low power object recognition via FPGA-based DCNN deployment

FPGAs are commonly used to accelerate domain-specific algorithmic implem...

FPGA-based Acceleration System for Visual Tracking

Visual tracking is one of the most important application areas of comput...

Towards real-time and energy efficient Siamese tracking – a hardware-software approach

Siamese trackers have been among the state-of-the-art solutions in each ...

Multi-Object Tracking as Attention Mechanism

We propose a conceptually simple and thus fast multi-object tracking (MO...

A transprecision floating-point cluster for efficient near-sensor data analytics

Recent applications in the domain of near-sensor computing require the a...

EBBINNOT: A Hardware Efficient Hybrid Event-Frame Tracker for Stationary Neuromorphic Vision Sensors

In this paper, we present a hybrid event-frame approach for detecting an...

Please sign up or login with your details

Forgot password? Click here to reset