Automatic Assessment of Divergent Thinking in Chinese Language with TransDis: A Transformer-Based Language Model Approach

by   Tianchen Yang, et al.

Language models have been increasingly popular for automatic creativity assessment, generating semantic distances to objectively measure the quality of creative ideas. However, there is currently a lack of an automatic assessment system for evaluating creative ideas in the Chinese language. To address this gap, we developed TransDis, a scoring system using transformer-based language models, capable of providing valid originality (quality) and flexibility (variety) scores for Alternative Uses Task (AUT) responses in Chinese. Study 1 demonstrated that the latent model-rated originality factor, comprised of three transformer-based models, strongly predicted human originality ratings, and the model-rated flexibility strongly correlated with human flexibility ratings as well. Criterion validity analyses indicated that model-rated originality and flexibility positively correlated to other creativity measures, demonstrating similar validity to human ratings. Study 2 3 showed that TransDis effectively distinguished participants instructed to provide creative vs. common uses (Study 2) and participants instructed to generate ideas in a flexible vs. persistent way (Study 3). Our findings suggest that TransDis can be a reliable and low-cost tool for measuring idea originality and flexibility in Chinese language, potentially paving the way for automatic creativity assessment in other languages. We offer an open platform to compute originality and flexibility for AUT responses in Chinese and over 50 other languages (


page 1

page 2

page 3

page 4


Putting GPT-3's Creativity to the (Alternative Uses) Test

AI large language models have (co-)produced amazing written works from n...

Automatic cry analysis and classification for infant pain assessment

The effectiveness of pain management relies on the choice and the correc...

SuperCLUE: A Comprehensive Chinese Large Language Model Benchmark

Large language models (LLMs) have shown the potential to be integrated i...

The language of sounds unheard: Exploring musical timbre semantics of large language models

Semantic dimensions of sound have been playing a central role in underst...

Evaluating Transformer Models and Human Behaviors on Chinese Character Naming

Neural network models have been proposed to explain the grapheme-phoneme...

A Pyramid Recurrent Network for Predicting Crowdsourced Speech-Quality Ratings of Real-World Signals

The real-world capabilities of objective speech quality measures are lim...

Can Transformer Language Models Predict Psychometric Properties?

Transformer-based language models (LMs) continue to advance state-of-the...

Please sign up or login with your details

Forgot password? Click here to reset