Bot or Human? Detecting ChatGPT Imposters with A Single Question

by   Hong Wang, et al.

Large language models like ChatGPT have recently demonstrated impressive capabilities in natural language understanding and generation, enabling various applications including translation, essay writing, and chit-chatting. However, there is a concern that they can be misused for malicious purposes, such as fraud or denial-of-service attacks. Therefore, it is crucial to develop methods for detecting whether the party involved in a conversation is a bot or a human. In this paper, we propose a framework named FLAIR, Finding Large language model Authenticity via a single Inquiry and Response, to detect conversational bots in an online manner. Specifically, we target a single question scenario that can effectively differentiate human users from bots. The questions are divided into two categories: those that are easy for humans but difficult for bots (e.g., counting, substitution, positioning, noise filtering, and ASCII art), and those that are easy for bots but difficult for humans (e.g., memorization and computation). Our approach shows different strengths of these questions in their effectiveness, providing a new way for online service providers to protect themselves against nefarious activities and ensure that they are serving real users. We open-sourced our dataset on and welcome contributions from the community to enrich such detection datasets.


page 1

page 2

page 3

page 4


Fauno: The Italian Large Language Model that will leave you senza parole!

This paper presents Fauno, the first and largest open-source Italian con...

CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities

Large language models (LMs) offer unprecedented language generation capa...

Large Language Models Encode Clinical Knowledge

Large language models (LLMs) have demonstrated impressive capabilities i...

Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models

Conversation agents fueled by Large Language Models (LLMs) are providing...

A Conversational Paradigm for Program Synthesis

Program synthesis strives to generate a computer program as a solution t...

DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents

Large language models (LLMs) have emerged as valuable tools for many nat...

Semantic Parsing for Conversational Question Answering over Knowledge Graphs

In this paper, we are interested in developing semantic parsers which un...

Please sign up or login with your details

Forgot password? Click here to reset