Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Prompts

by   Xiaocui Yang, et al.

Multimodal sentiment analysis is a trending topic with the explosion of multimodal content on the web. Present studies in multimodal sentiment analysis rely on large-scale supervised data. Collating supervised data is time-consuming and labor-intensive. As such, it is essential to investigate the problem of few-shot multimodal sentiment analysis. Previous works in few-shot models generally use language model prompts, which can improve performance in low-resource settings. However, the textual prompt ignores the information from other modalities. We propose Multimodal Probabilistic Fusion Prompts, which can provide diverse cues for multimodal sentiment detection. We first design a unified multimodal prompt to reduce the discrepancy in different modal prompts. To improve the robustness of our model, we then leverage multiple diverse prompts for each input and propose a probabilistic method to fuse the output predictions. Extensive experiments conducted on three datasets confirm the effectiveness of our approach.


page 1

page 2

page 3

page 4


Tensor Fusion Network for Multimodal Sentiment Analysis

Multimodal sentiment analysis is an increasingly popular research area, ...

CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection

Compared with unimodal data, multimodal data can provide more features t...

Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt

We have witnessed the rapid proliferation of multimodal data on numerous...

Syntax-aware Hybrid prompt model for Few-shot multi-modal sentiment analysis

Multimodal Sentiment Analysis (MSA) has been a popular topic in natural ...

TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models

Pre-trained large language models have recently achieved ground-breaking...

CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

Multimodal sentiment analysis and depression estimation are two importan...

Interpretable multimodal sentiment analysis based on textual modality descriptions by using large-scale language models

Multimodal sentiment analysis is an important area for understanding the...

Please sign up or login with your details

Forgot password? Click here to reset