VQFR: Blind Face Restoration with Vector-Quantized Dictionary and Parallel Decoder

by   Yuchao Gu, et al.

Although generative facial prior and geometric prior have recently demonstrated high-quality results for blind face restoration, producing fine-grained facial details faithful to inputs remains a challenging problem. Motivated by the classical dictionary-based methods and the recent vector quantization (VQ) technique, we propose a VQ-based face restoration method – VQFR. VQFR takes advantage of high-quality low-level feature banks extracted from high-quality faces and can thus help recover realistic facial details. However, the simple application of the VQ codebook cannot achieve good results with faithful details and identity preservation. Therefore, we further introduce two special network designs. 1). We first investigate the compression patch size in the VQ codebook and find that the VQ codebook designed with a proper compression patch size is crucial to balance the quality and fidelity. 2). To further fuse low-level features from inputs while not "contaminating" the realistic details generated from the VQ codebook, we proposed a parallel decoder consisting of a texture decoder and a main decoder. Those two decoders then interact with a texture warping module with deformable convolution. Equipped with the VQ codebook as a facial detail dictionary and the parallel decoder design, the proposed VQFR can largely enhance the restored quality of facial details while keeping the fidelity to previous methods. Codes will be available at https://github.com/TencentARC/VQFR.


page 2

page 4

page 5

page 8

page 10

page 11


Towards Real-World Blind Face Restoration with Generative Facial Prior

Blind face restoration usually relies on facial priors, such as facial g...

FaceFormer: Scale-aware Blind Face Restoration with Transformers

Blind face restoration usually encounters with diverse scale face inputs...

Learning Dual Memory Dictionaries for Blind Face Restoration

To improve the performance of blind face restoration, recent works mainl...

Blind Face Restoration via Deep Multi-scale Component Dictionaries

Recent reference-based face restoration methods have received considerab...

Towards Robust Blind Face Restoration with Codebook Lookup Transformer

Blind face restoration is a highly ill-posed problem that often requires...

DiffBFR: Bootstrapping Diffusion Model Towards Blind Face Restoration

Blind face restoration (BFR) is important while challenging. Prior works...

DifFace: Blind Face Restoration with Diffused Error Contraction

While deep learning-based methods for blind face restoration have achiev...

Please sign up or login with your details

Forgot password? Click here to reset