What went wrong?: Identification of Everyday Object Manipulation Anomalies

by   Dogan Altan, et al.

Extending the abilities of service robots is important for expanding what they can achieve in everyday manipulation tasks. On the other hand, it is also essential to ensure them to determine what they can not achieve in certain cases due to either anomalies or permanent failures during task execution. Robots need to identify these situations, and reveal the reasons behind these cases to overcome and recover from them. In this paper, we propose and analyze a Long Short-Term Memories-based (LSTM-based) awareness approach to reveal the reasons behind an anomaly case that occurs during a manipulation episode in an unstructured environment. The proposed method takes into account the real-time observations of the robot by fusing visual, auditory and proprioceptive sensory modalities to achieve this task. We also provide a comparative analysis of our method with Hidden Markov Models (HMMs) and Conditional Random Fields (CRFs). The symptoms of anomalies are first learned from a given training set, then they can be classified in real-time based on the learned models. The approaches are evaluated on a Baxter robot executing object manipulation scenarios. The results indicate that the LSTM-based method outperforms the other methods with a 0.94 classification rate in revealing causes of anomalies in case of an unexpected deviation.


page 9

page 17

page 22

page 23


Multimodal Detection and Identification of Robot Manipulation Failures

An autonomous service robot should be able to interact with its environm...

CLUE-AI: A Convolutional Three-stream Anomaly Identification Framework for Robot Manipulation

Robot safety has been a prominent research topic in recent years since r...

Probabilistic 3D Multilabel Real-time Mapping for Multi-object Manipulation

Probabilistic 3D map has been applied to object segmentation with multip...

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-based Variational Autoencoder

The detection of anomalous executions is valuable for reducing potential...

Neuromorphic Event-Based Slip Detection and suppression in Robotic Grasping and Manipulation

Slip detection is essential for robots to make robust grasping and fine ...

A Causal-based Approach to Explain, Predict and Prevent Failures in Robotic Tasks

Robots working in real environments need to adapt to unexpected changes ...

Please sign up or login with your details

Forgot password? Click here to reset