Learning Predictive Models From Observation and Interaction

by   Karl Schmeckpeper, et al.
University of Pennsylvania

Learning predictive models from interaction with the world allows an agent, such as a robot, to learn about how the world works, and then use this learned model to plan coordinated sequences of actions to bring about desired outcomes. However, learning a model that captures the dynamics of complex skills represents a major challenge: if the agent needs a good model to perform these skills, it might never be able to collect the experience on its own that is required to learn these delicate and complex behaviors. Instead, we can imagine augmenting the training set with observational data of other agents, such as humans. Such data is likely more plentiful, but represents a different embodiment. For example, videos of humans might show a robot how to use a tool, but (i) are not annotated with suitable robot actions, and (ii) contain a systematic distributional shift due to the embodiment differences between humans and robots. We address the first challenge by formulating the corresponding graphical model and treating the action as an observed variable for the interaction data and an unobserved variable for the observation data, and the second challenge by using a domain-dependent prior. In addition to interaction data, our method is able to leverage videos of passive observations in a driving dataset and a dataset of robotic manipulation videos. A robotic planning agent equipped with our method can learn to use tools in a tabletop robotic manipulation setting by observing humans without ever seeing a robotic video of tool use.


page 5

page 6

page 7

page 8

page 14

page 15

page 16

page 17


Structured World Models from Human Videos

We tackle the problem of learning complex, general behaviors directly in...

Self-Supervised Visual Planning with Temporal Skip Connections

In order to autonomously learn wide repertoires of complex skills, robot...

Learning Reward Functions for Robotic Manipulation by Observing Humans

Observing a human demonstrator manipulate objects provides a rich, scala...

Robotic Visuomotor Control with Unsupervised Forward Model Learned from Videos

Learning an accurate model of the environment is essential for model-bas...

Reinforcement Learning with Videos: Combining Offline Observations with Interaction

Reinforcement learning is a powerful framework for robots to acquire ski...

Deep Visual Foresight for Planning Robot Motion

A key challenge in scaling up robot learning to many skills and environm...

Shaping embodied agent behavior with activity-context priors from egocentric video

Complex physical tasks entail a sequence of object interactions, each wi...

Please sign up or login with your details

Forgot password? Click here to reset