Learning Cloth Folding Tasks with Refined Flow Based Spatio-Temporal Graphs

by   Peng Zhou, et al.

Cloth folding is a widespread domestic task that is seemingly performed by humans but which is highly challenging for autonomous robots to execute due to the highly deformable nature of textiles; It is hard to engineer and learn manipulation pipelines to efficiently execute it. In this paper, we propose a new solution for robotic cloth folding (using a standard folding board) via learning from demonstrations. Our demonstration video encoding is based on a high-level abstraction, namely, a refined optical flow-based spatiotemporal graph, as opposed to a low-level encoding such as image pixels. By constructing a new spatiotemporal graph with an advanced visual corresponding descriptor, the policy learning can focus on key points and relations with a 3D spatial configuration, which allows to quickly generalize across different environments. To further boost the policy searching, we combine optical flow and static motion saliency maps to discriminate the dominant motions for better handling the system dynamics in real-time, which aligns with the attentional motion mechanism that dominates the human imitation process. To validate the proposed approach, we analyze the manual folding procedure and developed a custom-made end-effector to efficiently interact with the folding board. Multiple experiments on a real robotic platform were conducted to validate the effectiveness and robustness of the proposed method.


page 1

page 3

page 5

page 7


FlowControl: Optical Flow Based Visual Servoing

One-shot imitation is the vision of robot programming from a single demo...

Unsupervised motion saliency map estimation based on optical flow inpainting

The paper addresses the problem of motion saliency in videos, that is, i...

Dynamical optical flow of saliency maps for predicting visual attention

Saliency maps are used to understand human attention and visual fixation...

An Efficient Optical Flow Based Motion Detection Method for Non-stationary Scenes

Real-time motion detection in non-stationary scenes is a difficult task ...

Joint direct estimation of 3D geometry and 3D motion using spatio temporal gradients

Conventional image motion based structure from motion methods first comp...

Investigation of Factorized Optical Flows as Mid-Level Representations

In this paper, we introduce a new concept of incorporating factorized fl...

Solving Robot Assembly Tasks by Combining Interactive Teaching and Self-Exploration

Many high precision (dis)assembly tasks are still being performed by hum...

Please sign up or login with your details

Forgot password? Click here to reset