Cross-Skeleton Interaction Graph Aggregation Network for Representation Learning of Mouse Social Behaviour

by   Feixiang Zhou, et al.

Automated social behaviour analysis of mice has become an increasingly popular research area in behavioural neuroscience. Recently, pose information (i.e., locations of keypoints or skeleton) has been used to interpret social behaviours of mice. Nevertheless, effective encoding and decoding of social interaction information underlying the keypoints of mice has been rarely investigated in the existing methods. In particular, it is challenging to model complex social interactions between mice due to highly deformable body shapes and ambiguous movement patterns. To deal with the interaction modelling problem, we here propose a Cross-Skeleton Interaction Graph Aggregation Network (CS-IGANet) to learn abundant dynamics of freely interacting mice, where a Cross-Skeleton Node-level Interaction module (CS-NLI) is used to model multi-level interactions (i.e., intra-, inter- and cross-skeleton interactions). Furthermore, we design a novel Interaction-Aware Transformer (IAT) to dynamically learn the graph-level representation of social behaviours and update the node-level representation, guided by our proposed interaction-aware self-attention mechanism. Finally, to enhance the representation ability of our model, an auxiliary self-supervised learning task is proposed for measuring the similarity between cross-skeleton nodes. Experimental results on the standard CRMI13-Skeleton and our PDMB-Skeleton datasets show that our proposed model outperforms several other state-of-the-art approaches.


page 1

page 3

page 9

page 10

page 15

page 16

page 18


IGFormer: Interaction Graph Transformer for Skeleton-based Human Interaction Recognition

Human interaction recognition is very important in many applications. On...

GTEA: Representation Learning for Temporal Interaction Graphs via Edge Aggregation

We consider the problem of representation learning for temporal interact...

SM-SGE: A Self-Supervised Multi-Scale Skeleton Graph Encoding Framework for Person Re-Identification

Person re-identification via 3D skeletons is an emerging topic with grea...

Multi-Level Graph Encoding with Structural-Collaborative Relation Learning for Skeleton-Based Person Re-Identification

Skeleton-based person re-identification (Re-ID) is an emerging open topi...

A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification

Person re-identification (Re-ID) via gait features within 3D skeleton se...

Temporally Guided Music-to-Body-Movement Generation

This paper presents a neural network model to generate virtual violinist...

Multi-Relation Aware Temporal Interaction Network Embedding

Temporal interaction networks are formed in many fields, e.g., e-commerc...

Please sign up or login with your details

Forgot password? Click here to reset