Working memory inspired hierarchical video decomposition with transformative representations

by   Binjie Qin, et al.

Video decomposition is very important to extract moving foreground objects from complex backgrounds in computer vision, machine learning, and medical imaging, e.g., extracting moving contrast-filled vessels from the complex and noisy backgrounds of X-ray coronary angiography (XCA). However, the challenges caused by dynamic backgrounds, overlapping heterogeneous environments and complex noises still exist in video decomposition. To solve these problems, this study is the first to introduce a flexible visual working memory model in video decomposition tasks to provide interpretable and high-performance hierarchical deep architecture, integrating the transformative representations between sensory and control layers from the perspective of visual and cognitive neuroscience. Specifically, robust PCA unrolling networks acting as a structure-regularized sensor layer decompose XCA into sparse/low-rank structured representations to separate moving contrast-filled vessels from noisy and complex backgrounds. Then, patch recurrent convolutional LSTM networks with a backprojection module embody unstructured random representations of the control layer in working memory, recurrently projecting spatiotemporally decomposed nonlocal patches into orthogonal subspaces for heterogeneous vessel retrieval and interference suppression. This video decomposition deep architecture effectively restores the heterogeneous profiles of intensity and the geometries of moving objects against the complex background interferences. Experiments show that the proposed method significantly outperforms state-of-the-art methods in accurate moving contrast-filled vessel extraction with excellent flexibility and computational efficiency.


page 10

page 11


Robust PCA Unrolling Network for Super-resolution Vessel Extraction in X-ray Coronary Angiography

Although robust PCA has been increasingly adopted to extract vessels fro...

Augmented Robust PCA For Foreground-Background Separation on Noisy, Moving Camera Video

This work presents a novel approach for robust PCA with total variation ...

Panoramic Robust PCA for Foreground-Background Separation on Noisy, Free-Motion Camera Video

This work presents a novel approach for robust PCA with total variation ...

Shadow-Background-Noise 3D Spatial Decomposition Using Sparse Low-Rank Gaussian Properties for Video-SAR Moving Target Shadow Enhancement

Moving target shadows among video synthetic aperture radar (Video-SAR) i...

Robust CUR Decomposition: Theory and Imaging Applications

This paper considers the use of Robust PCA in a CUR decomposition framew...

Robust Dual-Graph Regularized Moving Object Detection

Moving object detection and its associated background-foreground separat...

Implementation And Performance Evaluation Of Background Subtraction Algorithms

The study evaluates three background subtraction techniques. The techniq...

Please sign up or login with your details

Forgot password? Click here to reset