Learning Label Modular Prompts for Text Classification in the Wild

by   Hailin Chen, et al.
Nanyang Technological University

Machine learning models usually assume i.i.d data during training and testing, but data and tasks in real world often change over time. To emulate the transient nature of real world, we propose a challenging but practical task: text classification in-the-wild, which introduces different non-stationary training/testing stages. Decomposing a complex task into modular components can enable robust generalisation under such non-stationary environment. However, current modular approaches in NLP do not take advantage of recent advances in parameter efficient tuning of pretrained language models. To close this gap, we propose MODULARPROMPT, a label-modular prompt tuning framework for text classification tasks. In MODULARPROMPT, the input prompt consists of a sequence of soft label prompts, each encoding modular knowledge related to the corresponding class label. In two of most formidable settings, MODULARPROMPT outperforms relevant baselines by a large margin demonstrating strong generalisation ability. We also conduct comprehensive analysis to validate whether the learned prompts satisfy properties of a modular representation.


page 1

page 2

page 3

page 4


PTR: Prompt Tuning with Rules for Text Classification

Fine-tuned pre-trained language models (PLMs) have achieved awesome perf...

HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification

Hierarchical text classification (HTC) is a challenging subtask of multi...

Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

Tuning pre-trained language models (PLMs) with task-specific prompts has...

Prompt-Learning for Short Text Classification

In the short text, the extreme short length, feature sparsity and high a...

Description Based Text Classification with Reinforcement Learning

The task of text classification is usually divided into two stages: tex...

Privacy Leakage in Text Classification: A Data Extraction Approach

Recent work has demonstrated the successful extraction of training data ...

Is a Modular Architecture Enough?

Inspired from human cognition, machine learning systems are gradually re...

Please sign up or login with your details

Forgot password? Click here to reset