RepSR: Training Efficient VGG-style Super-Resolution Networks with Structural Re-Parameterization and Batch Normalization

by   Xintao Wang, et al.

This paper explores training efficient VGG-style super-resolution (SR) networks with the structural re-parameterization technique. The general pipeline of re-parameterization is to train networks with multi-branch topology first, and then merge them into standard 3x3 convolutions for efficient inference. In this work, we revisit those primary designs and investigate essential components for re-parameterizing SR networks. First of all, we find that batch normalization (BN) is important to bring training non-linearity and improve the final performance. However, BN is typically ignored in SR, as it usually degrades the performance and introduces unpleasant artifacts. We carefully analyze the cause of BN issue and then propose a straightforward yet effective solution. In particular, we first train SR networks with mini-batch statistics as usual, and then switch to using population statistics at the later training period. While we have successfully re-introduced BN into SR, we further design a new re-parameterizable block tailored for SR, namely RepSR. It consists of a clean residual path and two expand-and-squeeze convolution paths with the modified BN. Extensive experiments demonstrate that our simple RepSR is capable of achieving superior performance to previous SR re-parameterization methods among different model sizes. In addition, our RepSR can achieve a better trade-off between performance and actual running time (throughput) than previous SR methods. Codes will be available at


page 4

page 5

page 8


Wide Activation for Efficient and Accurate Image Super-Resolution

In this report we demonstrate that with same parameters and computationa...

Efficient Image Super-Resolution using Vast-Receptive-Field Attention

The attention mechanism plays a pivotal role in designing advanced super...

Transitive Learning: Exploring the Transitivity of Degradations for Blind Super-Resolution

Being extremely dependent on the iterative estimation and correction of ...

DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models

Image super-resolution (SR) with generative adversarial networks (GAN) h...

AdaDM: Enabling Normalization for Image Super-Resolution

Normalization like Batch Normalization (BN) is a milestone technique to ...

DyRep: Bootstrapping Training with Dynamic Re-parameterization

Structural re-parameterization (Rep) methods achieve noticeable improvem...

A heterogeneous group CNN for image super-resolution

Convolutional neural networks (CNNs) have obtained remarkable performanc...

Please sign up or login with your details

Forgot password? Click here to reset