Unsupervised Semantic Attribute Discovery and Control in Generative Models

by   William Paul, et al.

This work focuses on the ability to control via latent space factors semantic image attributes in generative models, and the faculty to discover mappings from factors to attributes in an unsupervised fashion. The discovery of controllable semantic attributes is of special importance, as it would facilitate higher level tasks such as unsupervised representation learning to improve anomaly detection, or the controlled generation of novel data for domain shift and imbalanced datasets. The ability to control semantic attributes is related to the disentanglement of latent factors, which dictates that latent factors be "uncorrelated" in their effects. Unfortunately, despite past progress, the connection between control and disentanglement remains, at best, confused and entangled, requiring clarifications we hope to provide in this work. To this end, we study the design of algorithms for image generation that allow unsupervised discovery and control of semantic attributes.We make several contributions: a) We bring order to the concepts of control and disentanglement, by providing an analytical derivation that connects mutual information maximization, which promotes attribute control, to total correlation minimization, which relates to disentanglement. b) We propose hybrid generative model architectures that use mutual information maximization with multi-scale style transfer. c) We introduce a novel metric to characterize the performance of semantic attributes control. We report experiments that appear to demonstrate, quantitatively and qualitatively, the ability of the proposed model to perform satisfactory control while still preserving competitive visual quality. We compare to other state of the art methods (e.g., Frechet inception distance (FID)= 9.90 on CelebA and 4.52 on EyePACS).


page 2

page 9

page 10

page 11

page 12


Evaluation of Latent Space Disentanglement in the Presence of Interdependent Attributes

Controllable music generation with deep generative models has become inc...

Cluster-guided Image Synthesis with Unconditional Models

Generative Adversarial Networks (GANs) are the driving force behind the ...

Example-Based Framework for Perceptually Guided Audio Texture Generation

Generative models for synthesizing audio textures explicitly encode cont...

Learning to Manipulate Individual Objects in an Image

We describe a method to train a generative model with latent factors tha...

Multi-Attribute Guided Painting Generation

Controllable painting generation plays a pivotal role in image stylizati...

PluGeN: Multi-Label Conditional Generation From Pre-Trained Models

Modern generative models achieve excellent quality in a variety of tasks...

ProtoVAE: Prototypical Networks for Unsupervised Disentanglement

Generative modeling and self-supervised learning have in recent years ma...

Please sign up or login with your details

Forgot password? Click here to reset