Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking

07/04/2020
by   Pengyu Zhang, et al.
1

In this study, we propose a novel RGB-T tracking framework by jointly modeling both appearance and motion cues. First, to obtain a robust appearance model, we develop a novel late fusion method to infer the fusion weight maps of both RGB and thermal (T) modalities. The fusion weights are determined by using offline-trained global and local multimodal fusion networks, and then adopted to linearly combine the response maps of RGB and T modalities. Second, when the appearance cue is unreliable, we comprehensively take motion cues, i.e., target and camera motions, into account to make the tracker robust. We further propose a tracker switcher to switch the appearance and motion trackers flexibly. Numerous results on three recent RGB-T tracking datasets show that the proposed tracker performs significantly better than other state-of-the-art algorithms.

READ FULL TEXT

page 1

page 3

page 4

page 7

page 9

03/21/2020

Appearance Fusion of Multiple Cues for Video Co-localization

This work addresses a problem named video co-localization that aims at l...
01/22/2022

Temporal Aggregation for Adaptive RGBT Tracking

Visual object tracking with RGB and thermal infrared (TIR) spectra avail...
06/09/2022

Simple Cues Lead to a Strong Multi-Object Tracker

For a long time, the most common paradigm in Multi-Object Tracking was t...
08/30/2019

Multi-Modal Fusion for End-to-End RGB-T Tracking

We propose an end-to-end tracking framework for fusing the RGB and TIR m...
12/21/2018

Saliency Guided Hierarchical Robust Visual Tracking

A saliency guided hierarchical visual tracking (SHT) algorithm containin...
07/24/2019

Dense Feature Aggregation and Pruning for RGBT Tracking

How to perform effective information fusion of different modalities is a...
05/22/2018

Part-Based Tracking by Sampling

We propose a novel part-based method for tracking an arbitrary object in...

Code Repositories

JMMAC

The official repository of 'Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking'. The code and results are comming soon.


view repo

Please sign up or login with your details

Forgot password? Click here to reset