Talk-to-Edit: Fine-Grained Facial Editing via Dialog

by   Yuming Jiang, et al.

Facial editing is an important task in vision and graphics with numerous applications. However, existing works are incapable to deliver a continuous and fine-grained editing mode (e.g., editing a slightly smiling face to a big laughing one) with natural interactions with users. In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system. Our key insight is to model a continual "semantic field" in the GAN latent space. 1) Unlike previous works that regard the editing as traversing straight lines in the latent space, here the fine-grained editing is formulated as finding a curving trajectory that respects fine-grained attribute landscape on the semantic field. 2) The curvature at each step is location-specific and determined by the input image as well as the users' language requests. 3) To engage the users in a meaningful dialog, our system generates language feedback by considering both the user request and the current state of the semantic field. We also contribute CelebA-Dialog, a visual-language facial editing dataset to facilitate large-scale study. Specifically, each image has manually annotated fine-grained attribute annotations as well as template-based textual descriptions in natural language. Extensive quantitative and qualitative experiments demonstrate the superiority of our framework in terms of 1) the smoothness of fine-grained editing, 2) the identity/attribute preservation, and 3) the visual photorealism and dialog fluency. Notably, user study validates that our overall system is consistently favored by around 80 participants. Our project page is


page 6

page 8

page 17

page 18

page 19

page 20

page 21

page 22


ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulation

Editing real facial images is a crucial task in computer vision with sig...

Chat-to-Design: AI Assisted Personalized Fashion Design

In this demo, we present Chat-to-Design, a new multimodal interaction sy...

MaskGAN: Towards Diverse and Interactive Facial Image Manipulation

Facial image manipulation has achieved great progresses in recent years....

I2Edit: Towards Multi-turn Interactive Image Editing via Dialogue

Although there have been considerable research efforts on controllable f...

Edit As You Wish: Video Description Editing with Multi-grained Commands

Automatically narrating a video with natural language can assist people ...

ImageEye: Batch Image Processing Using Program Synthesis

This paper presents a new synthesis-based approach for batch image proce...

ChangeBeadsThreader: An Interactive Environment for Tailoring Automatically Untangled Changes

To improve the usability of a revision history, change untangling, which...

Please sign up or login with your details

Forgot password? Click here to reset