MVFlow: Deep Optical Flow Estimation of Compressed Videos with Motion Vector Prior

by   Shili Zhou, et al.

In recent years, many deep learning-based methods have been proposed to tackle the problem of optical flow estimation and achieved promising results. However, they hardly consider that most videos are compressed and thus ignore the pre-computed information in compressed video streams. Motion vectors, one of the compression information, record the motion of the video frames. They can be directly extracted from the compression code stream without computational cost and serve as a solid prior for optical flow estimation. Therefore, we propose an optical flow model, MVFlow, which uses motion vectors to improve the speed and accuracy of optical flow estimation for compressed videos. In detail, MVFlow includes a key Motion-Vector Converting Module, which ensures that the motion vectors can be transformed into the same domain of optical flow and then be utilized fully by the flow estimation module. Meanwhile, we construct four optical flow datasets for compressed videos containing frames and motion vectors in pairs. The experimental results demonstrate the superiority of our proposed MVFlow, which can reduce the AEPE by 1.09 compared to existing models or save 52


page 2

page 3

page 4

page 5

page 7

page 8

page 10


Offline and Online Optical Flow Enhancement for Deep Video Compression

Video compression relies heavily on exploiting the temporal redundancy b...

DTVNet: Dynamic Time-lapse Video Generation via Single Still Image

This paper presents a novel end-to-end dynamic time-lapse video generati...

Crowd Flow Segmentation in Compressed Domain using CRF

Crowd flow segmentation is an important step in many video surveillance ...

Real-Time Viewport-Aware Optical Flow Estimation in 360-degree Videos for Visually-Induced Motion Sickness Mitigation

Visually-induced motion sickness (VIMS), a side effect of illusionary mo...

Neural Cell Video Synthesis via Optical-Flow Diffusion

The biomedical imaging world is notorious for working with small amounts...

Estimation of Clinical Workload and Patient Activity using Deep Learning and Optical Flow

Contactless monitoring using thermal imaging has become increasingly pro...

Position and Vector Detection of Blind Spot motion with the Horn-Schunck Optical Flow

The proposed method uses live image footage which, based on calculations...

Please sign up or login with your details

Forgot password? Click here to reset