Evaluating Continual Learning Algorithms by Generating 3D Virtual Environments

by   Enrico Meloni, et al.

Continual learning refers to the ability of humans and animals to incrementally learn over time in a given environment. Trying to simulate this learning process in machines is a challenging task, also due to the inherent difficulty in creating conditions for designing continuously evolving dynamics that are typical of the real-world. Many existing research works usually involve training and testing of virtual agents on datasets of static images or short videos, considering sequences of distinct learning tasks. However, in order to devise continual learning algorithms that operate in more realistic conditions, it is fundamental to gain access to rich, fully customizable and controlled experimental playgrounds. Focussing on the specific case of vision, we thus propose to leverage recent advances in 3D virtual environments in order to approach the automatic generation of potentially life-long dynamic scenes with photo-realistic appearance. Scenes are composed of objects that move along variable routes with different and fully customizable timings, and randomness can also be included in their evolution. A novel element of this paper is that scenes are described in a parametric way, thus allowing the user to fully control the visual complexity of the input stream the agent perceives. These general principles are concretely implemented exploiting a recently published 3D virtual environment. The user can generate scenes without the need of having strong skills in computer graphics, since all the generation facilities are exposed through a simple high-level Python interface. We publicly share the proposed generator.


page 4

page 5

page 6


Continual Learning for Robotics

Continual learning (CL) is a particular machine learning paradigm where ...

Visually Grounded Continual Learning of Compositional Semantics

Children's language acquisition from the visual world is a real-world ex...

SAILenv: Learning in Virtual Visual Environments Made Simple

Recently, researchers in Machine Learning algorithms, Computer Vision sc...

L2Explorer: A Lifelong Reinforcement Learning Assessment Environment

Despite groundbreaking progress in reinforcement learning for robotics, ...

AI Autonomy: Self-Initiation, Adaptation and Continual Learning

As more and more AI agents are used in practice, it is time to think abo...

A Procedural World Generation Framework for Systematic Evaluation of Continual Learning

Several families of continual learning techniques have been proposed to ...

Please sign up or login with your details

Forgot password? Click here to reset