Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking

by   Pengyu Zhang, et al.

In this study, we propose a novel RGB-T tracking framework by jointly modeling both appearance and motion cues. First, to obtain a robust appearance model, we develop a novel late fusion method to infer the fusion weight maps of both RGB and thermal (T) modalities. The fusion weights are determined by using offline-trained global and local multimodal fusion networks, and then adopted to linearly combine the response maps of RGB and T modalities. Second, when the appearance cue is unreliable, we comprehensively take motion cues, i.e., target and camera motions, into account to make the tracker robust. We further propose a tracker switcher to switch the appearance and motion trackers flexibly. Numerous results on three recent RGB-T tracking datasets show that the proposed tracker performs significantly better than other state-of-the-art algorithms.


page 1

page 3

page 4

page 7

page 9


Appearance Fusion of Multiple Cues for Video Co-localization

This work addresses a problem named video co-localization that aims at l...

Temporal Aggregation for Adaptive RGBT Tracking

Visual object tracking with RGB and thermal infrared (TIR) spectra avail...

Simple Cues Lead to a Strong Multi-Object Tracker

For a long time, the most common paradigm in Multi-Object Tracking was t...

Multi-Modal Fusion for End-to-End RGB-T Tracking

We propose an end-to-end tracking framework for fusing the RGB and TIR m...

Saliency Guided Hierarchical Robust Visual Tracking

A saliency guided hierarchical visual tracking (SHT) algorithm containin...

Dense Feature Aggregation and Pruning for RGBT Tracking

How to perform effective information fusion of different modalities is a...

Part-Based Tracking by Sampling

We propose a novel part-based method for tracking an arbitrary object in...

Code Repositories


The official repository of 'Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking'. The code and results are comming soon.

view repo

Please sign up or login with your details

Forgot password? Click here to reset