TransTrack: Multiple-Object Tracking with Transformer

by   Peize Sun, et al.

Multiple-object tracking(MOT) is mostly dominated by complex and multi-step tracking-by-detection algorithm, which performs object detection, feature extraction and temporal association, separately. Query-key mechanism in single-object tracking(SOT), which tracks the object of the current frame by object feature of the previous frame, has great potential to set up a simple joint-detection-and-tracking MOT paradigm. Nonetheless, the query-key method is seldom studied due to its inability to detect new-coming objects. In this work, we propose TransTrack, a baseline for MOT with Transformer. It takes advantage of query-key mechanism and introduces a set of learned object queries into the pipeline to enable detecting new-coming objects. TransTrack has three main advantages: (1) It is an online joint-detection-and-tracking pipeline based on query-key mechanism. Complex and multi-step components in the previous methods are simplified. (2) It is a brand new architecture based on Transformer. The learned object query detects objects in the current frame. The object feature query from the previous frame associates those current objects with the previous ones. (3) For the first time, we demonstrate a much simple and effective method based on query-key mechanism and Transformer architecture could achieve competitive 65.8% MOTA on the MOT17 challenge dataset. We hope TransTrack can provide a new perspective for multiple-object tracking. The code is available at: <>.


page 1

page 2

page 7


MOTR: End-to-End Multiple-Object Tracking with TRansformer

The key challenge in multiple-object tracking (MOT) task is temporal mod...

PatchTrack: Multiple Object Tracking Using Frame Patches

Object motion and object appearance are commonly used information in mul...

MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors

In this paper, we propose MOTRv2, a simple yet effective pipeline to boo...

MeMOT: Multi-Object Tracking with Memory

We propose an online tracking algorithm that performs the object detecti...

EnsembleMOT: A Step towards Ensemble Learning of Multiple Object Tracking

Multiple Object Tracking (MOT) has rapidly progressed in recent years. E...

Efficient Joint Detection and Multiple Object Tracking with Spatially Aware Transformer

We propose a light-weight and highly efficient Joint Detection and Track...

FocalFormer3D : Focusing on Hard Instance for 3D Object Detection

False negatives (FN) in 3D object detection, e.g., missing predictions o...

Please sign up or login with your details

Forgot password? Click here to reset