Controllable Animation of Fluid Elements in Still Images

by   Aniruddha Mahapatra, et al.

We propose a method to interactively control the animation of fluid elements in still images to generate cinemagraphs. Specifically, we focus on the animation of fluid elements like water, smoke, fire, which have the properties of repeating textures and continuous fluid motion. Taking inspiration from prior works, we represent the motion of such fluid elements in the image in the form of a constant 2D optical flow map. To this end, we allow the user to provide any number of arrow directions and their associated speeds along with a mask of the regions the user wants to animate. The user-provided input arrow directions, their corresponding speed values, and the mask are then converted into a dense flow map representing a constant optical flow map (FD). We observe that FD, obtained using simple exponential operations can closely approximate the plausible motion of elements in the image. We further refine computed dense optical flow map FD using a generative-adversarial network (GAN) to obtain a more realistic flow map. We devise a novel UNet based architecture to autoregressively generate future frames using the refined optical flow map by forward-warping the input image features at different resolutions. We conduct extensive experiments on a publicly available dataset and show that our method is superior to the baselines in terms of qualitative and quantitative metrics. In addition, we show the qualitative animations of the objects in directions that did not exist in the training set and provide a way to synthesize videos that otherwise would not exist in the real world.


page 1

page 4

page 6

page 7

page 8


Learning to Estimate and Refine Fluid Motion with Physical Dynamics

Extracting information on fluid motion directly from images is challengi...

Learning Human Optical Flow

The optical flow of humans is well known to be useful for the analysis o...

Learning optical flow from still images

This paper deals with the scarcity of data for training optical flow net...

Animating Pictures with Eulerian Motion Fields

In this paper, we demonstrate a fully automatic method for converting a ...

Hierarchical Video Generation from Orthogonal Information: Optical Flow and Texture

Learning to represent and generate videos from unlabeled data is a very ...

Where computer vision can aid physics: dynamic cloud motion forecasting from satellite images

This paper describes a new algorithm for solar energy forecasting from a...

Temporal Flow Mask Attention for Open-Set Long-Tailed Recognition of Wild Animals in Camera-Trap Images

Camera traps, unmanned observation devices, and deep learning-based imag...

Please sign up or login with your details

Forgot password? Click here to reset