Learning visual policies for building 3D shape categories

by   Alexander Pashevich, et al.

Manipulation and assembly tasks require non-trivial planning of actions depending on the environment and the final goal. Previous work in this domain often assembles particular instances of objects from known sets of primitives. In contrast, we here aim to handle varying sets of primitives and to construct different objects of the same shape category. Given a single object instance of a category, e.g. an arch, and a binary shape classifier, we learn a visual policy to assemble other instances of the same category. In particular, we propose a disassembly procedure and learn a state policy that discovers new object instances and their assembly plans in state space. We then render simulated states in the observation space and learn a heatmap representation to predict alternative actions from a given input image. To validate our approach, we first demonstrate its efficiency for building object categories in state space. We then show the success of our visual policies for building arches from different primitives. Moreover, we demonstrate (i) the reactive ability of our method to re-assemble objects using additional primitives and (ii) the robust performance of our policy for unseen primitives resembling building blocks used during training. Our visual assembly policies are trained with no real images and reach up to 95


page 1

page 2

page 7

page 9

page 10

page 11

page 12


Reinforcement Learning with Parameterized Manipulation Primitives for Robotic Assembly

A common theme in robot assembly is the adoption of Manipulation Primiti...

Learning Sequences of Manipulation Primitives for Robotic Assembly

This paper explores the idea that skillful assembly is best represented ...

Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly

Is it possible to learn policies for robotic assembly that can generaliz...

Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks

This paper introduces a discrete-continuous action space to learn insert...

CARTO: Category and Joint Agnostic Reconstruction of ARTiculated Objects

We present CARTO, a novel approach for reconstructing multiple articulat...

Pose Induction for Novel Object Categories

We address the task of predicting pose for objects of unannotated object...

Combinatorial 3D Shape Generation via Sequential Assembly

3D shape generation has drawn attention in computer vision and machine l...

Please sign up or login with your details

Forgot password? Click here to reset