Evaluating the Effectiveness of Pre-trained Language Models in Predicting the Helpfulness of Online Product Reviews

by   Ali Boluki, et al.

Businesses and customers can gain valuable information from product reviews. The sheer number of reviews often necessitates ranking them based on their potential helpfulness. However, only a few reviews ever receive any helpfulness votes on online marketplaces. Sorting all reviews based on the few existing votes can cause helpful reviews to go unnoticed because of the limited attention span of readers. The problem of review helpfulness prediction is even more important for higher review volumes, and newly written reviews or launched products. In this work we compare the use of RoBERTa and XLM-R language models to predict the helpfulness of online product reviews. The contributions of our work in relation to literature include extensively investigating the efficacy of state-of-the-art language models – both monolingual and multilingual – against a robust baseline, taking ranking metrics into account when assessing these approaches, and assessing multilingual models for the first time. We employ the Amazon review dataset for our experiments. According to our study on several product categories, multilingual and monolingual pre-trained language models outperform the baseline that utilizes random forest with handcrafted features as much as 23 for complex text feature engineering. However, our results suggest that pre-trained multilingual models may not be used for fine-tuning only one language. We assess the performance of language models with and without additional features. Our results show that including additional features like product rating by the reviewer can further help the predictive methods.


page 1

page 2

page 3

page 4


Investigating Monolingual and Multilingual BERTModels for Vietnamese Aspect Category Detection

Aspect category detection (ACD) is one of the challenging tasks in the A...

Assessing Guest Nationality Composition from Hotel Reviews

Many hotels target guest acquisition efforts to specific markets in orde...

The Multilingual Amazon Reviews Corpus

We present the Multilingual Amazon Reviews Corpus (MARC), a large-scale ...

Understanding the Impact of Culture in Assessing Helpfulness of Online Reviews

Online reviews have become essential for users to make informed decision...

Does the Objective Matter? Comparing Training Objectives for Pronoun Resolution

Hard cases of pronoun resolution have been used as a long-standing bench...

MedMine: Examining Pre-trained Language Models on Medication Mining

Automatic medication mining from clinical and biomedical text has become...

Using Argument-based Features to Predict and Analyse Review Helpfulness

We study the helpful product reviews identification problem in this pape...

Please sign up or login with your details

Forgot password? Click here to reset