Visuo-Tactile-Based Slip Detection Using A Multi-Scale Temporal Convolution Network

by   Junli Gao, et al.

Humans can accurately determine whether the object in hand has slipped or not by visual and tactile perception. However, it is still a challenge for robots to detect in-hand object slip through visuo-tactile fusion. To address this issue, a novel visuo-tactile fusion deep neural network is proposed to detect slip, which is a time-dependent continuous action. By using the multi-scale temporal convolution network (MS-TCN) to extract the temporal features of visual and tactile data, the slip can be detected effectively. In this paper, a 7-dregree-of-freedom (7-DoF) robot manipulator equipped with a camera and a tactile sensor is used for data collection on 50 daily objects with different shapes, materials, sizes, and weights. Therefore, a dataset is built, where the grasping data of 40 objects and 10 objects are used for network training and testing, respectively. The detection accuracy is 96.96 model. Also, the proposed model is compared with a visuo-tactile fusion deep neural network (DNN) based on long short-term memory network (LSTM) on the collected dataset and a public dataset using the GelSight tactile sensor. The results demonstrate that the proposed model performs better on both dataset. The proposed model can help robots grasp daily objects reliably. In addition, it can be used in grasping force control, grasping policy generation and dexterous manipulation.


page 1

page 4

page 5

page 7

page 8

page 9


Grasp State Assessment of Deformable Objects Using Visual-Tactile Fusion Perception

Humans can quickly determine the force required to grasp a deformable ob...

Slip Detection with Combined Tactile and Visual Information

Slip detection plays a vital role in robotic manipulation and it has lon...

FingerVision Tactile Sensor Design and Slip Detection Using Convolutional LSTM Network

Tactile sensing is essential to the human perception system, so as to ro...

Object recognition for robotics from tactile time series data utilising different neural network architectures

Robots need to exploit high-quality information on grasped objects to in...

Leveraging distributed contact force measurements for slip detection: a physics-based approach enabled by a data-driven tactile sensor

Grasping objects whose physical properties are unknown is still a great ...

Grasp Stability Assessment Through Attention-Guided Cross-Modality Fusion and Transfer Learning

Extensive research has been conducted on assessing grasp stability, a cr...

Multi-Scale DenseNet-Based Electricity Theft Detection

Electricity theft detection issue has drawn lots of attention during las...

Please sign up or login with your details

Forgot password? Click here to reset