TUCaN: Progressively Teaching Colourisation to Capsules

by   Rita Pucci, et al.

Automatic image colourisation is the computer vision research path that studies how to colourise greyscale images (for restoration). Deep learning techniques improved image colourisation yielding astonishing results. These differ by various factors, such as structural differences, input types, user assistance, etc. Most of them, base the architectural structure on convolutional layers with no emphasis on layers specialised in object features extraction. We introduce a novel downsampling upsampling architecture named TUCaN (Tiny UCapsNet) that exploits the collaboration of convolutional layers and capsule layers to obtain a neat colourisation of entities present in every single image. This is obtained by enforcing collaboration among such layers by skip and residual connections. We pose the problem as a per pixel colour classification task that identifies colours as a bin in a quantized space. To train the network, in contrast with the standard end to end learning method, we propose the progressive learning scheme to extract the context of objects by only manipulating the learning process without changing the model. In this scheme, the upsampling starts from the reconstruction of low resolution images and progressively grows to high resolution images throughout the training phase. Experimental results on three benchmark datasets show that our approach with ImageNet10k dataset outperforms existing methods on standard quality metrics and achieves state of the art performances on image colourisation. We performed a user study to quantify the perceptual realism of the colourisation results demonstrating: that progressive learning let the TUCaN achieve better colours than the end to end scheme; and pointing out the limitations of the existing evaluation metrics.


page 2

page 3

page 6

page 7

page 9

page 12

page 13


Collaboration among Image and Object Level Features for Image Colourisation

Image colourisation is an ill-posed problem, with multiple correct solut...

Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections

In this paper, we propose a very deep fully convolutional encoding-decod...

Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections

Image restoration, including image denoising, super resolution, inpainti...

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

We introduce Pixel-aligned Implicit Function (PIFu), a highly effective ...

A Fully Progressive Approach to Single-Image Super-Resolution

Recent deep learning approaches to single image super-resolution have ac...

Deep Recurrent Level Set for Segmenting Brain Tumors

Variational Level Set (VLS) has been a widely used method in medical seg...

SRN: Side-output Residual Network for Object Reflection Symmetry Detection and Beyond

In this paper, we establish a baseline for object reflection symmetry de...

Please sign up or login with your details

Forgot password? Click here to reset