Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis

10/26/2021
by   Juhua Liu, et al.
0

Aspect-based Sentiment Analysis (ABSA) aims to determine the sentiment polarity towards an aspect. Because of the expensive and limited labelled data, the pretraining strategy has become the de-facto standard for ABSA. However, there always exists severe domain shift between the pretraining and downstream ABSA datasets, hindering the effective knowledge transfer when directly finetuning and making the downstream task performs sub-optimal. To mitigate such domain shift, we introduce a unified alignment pretraining framework into the vanilla pretrain-finetune pipeline with both instance- and knowledge-level alignments. Specifically, we first devise a novel coarse-to-fine retrieval sampling approach to select target domain-related instances from the large-scale pretraining dataset, thus aligning the instances between pretraining and target domains (First Stage). Then, we introduce a knowledge guidance-based strategy to further bridge the domain gap at the knowledge level. In practice, we formulate the model pretrained on the sampled instances into a knowledge guidance model and a learner model, respectively. On the target dataset, we design an on-the-fly teacher-student joint fine-tuning approach to progressively transfer the knowledge from the knowledge guidance model to the learner model (Second Stage). Thereby, the learner model can maintain more domain-invariant knowledge when learning new knowledge from the target dataset. In the Third Stage, the learner model is finetuned to better adapt its learned knowledge to the target dataset. Extensive experiments and analyses on several ABSA benchmarks demonstrate the effectiveness and universality of our proposed pretraining framework. Notably, our pretraining framework pushes several strong baseline models up to the new state-of-the-art records. We release our code and models.

READ FULL TEXT
research
11/16/2018

Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification

Aspect-level sentiment classification (ASC) aims at identifying sentimen...
research
05/16/2023

Bidirectional Generative Framework for Cross-domain Aspect-based Sentiment Analysis

Cross-domain aspect-based sentiment analysis (ABSA) aims to perform vari...
research
01/29/2022

A Simple Information-Based Approach to Unsupervised Domain-Adaptive Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analy...
research
06/05/2023

LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

Lifelong learning offers a promising paradigm of building a generalist a...
research
07/26/2021

Improve Unsupervised Pretraining for Few-label Transfer

Unsupervised pretraining has achieved great success and many recent work...
research
06/11/2020

What makes instance discrimination good for transfer learning?

Unsupervised visual pretraining based on the instance discrimination pre...
research
07/16/2021

Rectifying the Shortcut Learning of Background: Shared Object Concentration for Few-Shot Image Recognition

Few-Shot image classification aims to utilize pretrained knowledge learn...

Please sign up or login with your details

Forgot password? Click here to reset