Social Processes: Self-Supervised Forecasting of Nonverbal Cues in Social Conversations

by   Chirag Raman, et al.

The default paradigm for the forecasting of human behavior in social conversations is characterized by top-down approaches. These involve identifying predictive relationships between low level nonverbal cues and future semantic events of interest (e.g. turn changes, group leaving). A common hurdle however, is the limited availability of labeled data for supervised learning. In this work, we take the first step in the direction of a bottom-up self-supervised approach in the domain. We formulate the task of Social Cue Forecasting to leverage the larger amount of unlabeled low-level behavior cues, and characterize the modeling challenges involved. To address these, we take a meta-learning approach and propose the Social Process (SP) models–socially aware sequence-to-sequence (Seq2Seq) models within the Neural Process (NP) family. SP models learn extractable representations of non-semantic future cues for each participant, while capturing global uncertainty by jointly reasoning about the future for all members of the group. Evaluation on synthesized and real-world behavior data shows that our SP models achieve higher log-likelihood than the NP baselines, and also highlights important considerations for applying such techniques within the domain of social human interactions.


page 2

page 16

page 17

page 18


Unleash Model Potential: Bootstrapped Meta Self-supervised Learning

The long-term goal of machine learning is to learn general visual repres...

Self-supervised Social Relation Representation for Human Group Detection

Human group detection, which splits crowd of people into groups, is an i...

Semi-supervised Learning: Fusion of Self-supervised, Supervised Learning, and Multimodal Cues for Tactical Driver Behavior Detection

In this paper, we presented a preliminary study for tactical driver beha...

Improving Limited Labeled Dialogue State Tracking with Self-Supervision

Existing dialogue state tracking (DST) models require plenty of labeled ...

Dataset Inference for Self-Supervised Models

Self-supervised models are increasingly prevalent in machine learning (M...

Connecting Web Event Forecasting with Anomaly Detection: A Case Study on Enterprise Web Applications Using Self-Supervised Neural Networks

Recently web applications have been widely used in enterprises to assist...

Multipar-T: Multiparty-Transformer for Capturing Contingent Behaviors in Group Conversations

As we move closer to real-world AI systems, AI agents must be able to de...

Please sign up or login with your details

Forgot password? Click here to reset