PromptSum: Parameter-Efficient Controllable Abstractive Summarization

by   Mathieu Ravaut, et al.

Prompt tuning (PT), a parameter-efficient technique that only tunes the additional prompt embeddings while keeping the backbone pre-trained language model (PLM) frozen, has shown promising results in language understanding tasks, especially in low-resource scenarios. However, effective prompt design methods suitable for generation tasks such as summarization are still lacking. At the same time, summarization guided through instructions (discrete prompts) can achieve a desirable double objective of high quality and controllability in summary generation. Towards a goal of strong summarization performance under the triple conditions of parameter-efficiency, data-efficiency, and controllability, we introduce PromptSum, a method combining PT with a multi-task objective and discrete entity prompts for abstractive summarization. Our model achieves competitive ROUGE results on popular abstractive summarization benchmarks coupled with a strong level of controllability through entities, all while only tuning several orders of magnitude less parameters.


page 1

page 2

page 3

page 4


To Adapt or to Fine-tune: A Case Study on Abstractive Summarization

Recent advances in the field of abstractive summarization leverage pre-t...

Pre-trained Language Model Representations for Language Generation

Pre-trained language model representations have been successful in a wid...

LightNER: A Lightweight Generative Framework with Prompt-guided Attention for Low-resource NER

Most existing NER methods rely on extensive labeled data for model train...

SuryaKiran at MEDIQA-Sum 2023: Leveraging LoRA for Clinical Dialogue Summarization

Finetuning Large Language Models helps improve the results for domain-sp...

TempoSum: Evaluating the Temporal Generalization of Abstractive Summarization

Recent pre-trained language models (PLMs) achieve promising results in e...

Exploiting Summarization Data to Help Text Simplification

One of the major problems with text simplification is the lack of high-q...

SAGA: Summarization-Guided Assert Statement Generation

Generating meaningful assert statements is one of the key challenges in ...

Please sign up or login with your details

Forgot password? Click here to reset