InstanceMotSeg: Real-time Instance Motion Segmentation for Autonomous Driving

by   Eslam Mohamed, et al.

Moving object segmentation is a crucial task for autonomous vehicles as it can be used to segment objects in a class agnostic manner based on its motion cues. It will enable the detection of objects unseen during training (e.g., moose or a construction truck) generically based on their motion. Although pixel-wise motion segmentation has been studied in the literature, it is not dealt with at instance level, which would help separate connected segments of moving objects leading to better trajectory planning. In this paper, we proposed a motion-based instance segmentation task and created a new annotated dataset based on KITTI, which will be released publicly. We make use of the YOLACT model to solve the instance motion segmentation network by feeding inflow and image as input and instance motion masks as output. We extend it to a multi-task model that learns semantic and motion instance segmentation in a computationally efficient manner. Our model is based on sharing a prototype generation network between the two tasks and learning separate prototype coefficients per task. To obtain real-time performance, we study different efficient encoders and obtain 39 fps on a Titan Xp GPU using MobileNetV2 with an improvement of 10 our work is available in


page 1

page 3

page 7

page 8


Video Class Agnostic Segmentation Benchmark for Autonomous Driving

Semantic segmentation approaches are typically trained on large-scale da...

VM-MODNet: Vehicle Motion aware Moving Object Detection for Autonomous Driving

Moving object Detection (MOD) is a critical task in autonomous driving a...

InstMove: Instance Motion for Object-centric Video Segmentation

Despite significant efforts, cutting-edge video segmentation methods sti...

MODNet: Moving Object Detection Network with Motion and Appearance for Autonomous Driving

We propose a novel multi-task learning system that combines appearance a...

Towards Segmenting Everything That Moves

Video analysis is the task of perceiving the world as it changes. Often,...

RNGDet++: Road Network Graph Detection by Transformer with Instance Segmentation and Multi-scale Features Enhancement

The graph structure of road networks is critical for downstream tasks of...

Recurrent Instance Segmentation using Sequences of Referring Expressions

The goal of this work is to segment the objects in an image that are ref...

Please sign up or login with your details

Forgot password? Click here to reset