Can robots mold soft plastic materials by shaping depth images?

by   Ege Gursoy, et al.

Can robots mold soft plastic materials by shaping depth images? The short answer is no: current day robots can't. In this article, we address the problem of shaping plastic material with an anthropomorphic arm/hand robot, which observes the material with a fixed depth camera. Robots capable of molding could assist humans in many tasks, such as cooking, scooping or gardening. Yet, the problem is complex, due to its high-dimensionality at both perception and control levels. To address it, we design three alternative data-based methods for predicting the effect of robot actions on the material. Then, the robot can plan the sequence of actions and their positions, to mold the material into a desired shape. To make the prediction problem tractable, we rely on two original ideas. First, we prove that under reasonable assumptions, the shaping problem can be mapped from point cloud to depth image space, with many benefits (simpler processing, no need for registration, lower computation time and memory requirements). Second, we design a novel, simple metric for quickly measuring the distance between two depth images. The metric is based on the inherent point cloud representation of depth images, which enables direct and consistent comparison of image pairs through a non-uniform scaling approach, and therefore opens promising perspectives for designing depth image – based robot controllers. We assess our approach in a series of unprecedented experiments, where a robotic arm/hand molds flour from initial to final shapes, either with its own dataset, or by transfer learning from a human dataset. We conclude the article by discussing the limitations of our framework and those of current day hardware, which make human-like robot molding a challenging open research problem.


page 1

page 4

page 8

page 10

page 11

page 12

page 13

page 15


Model-free vision-based shaping of deformable plastic materials

We address the problem of shaping deformable plastic materials using non...

A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU

In this paper, we present a multimodal mobile teleoperation system that ...

Co-manipulation of soft-materials estimating deformation from depth images

Human-robot co-manipulation of soft materials, such as fabrics, composit...

Controlling Assistive Robots with Learned Latent Actions

Assistive robots enable users with disabilities to perform everyday task...

A Laser-based Dual-arm System for Precise Control of Collaborative Robots

Collaborative robots offer increased interaction capabilities at relativ...

Mobile Robot-Assisted Mapping of Materials in Unknown Environments

Active perception by robots of surrounding objects and environmental ele...

Robotic Dough Shaping

We address the problem of shaping a piece of dough-like deformable mater...

Please sign up or login with your details

Forgot password? Click here to reset