Decoding the Enigma: Benchmarking Humans and AIs on the Many Facets of Working Memory

by   Ankur Sikarwar, et al.

Working memory (WM), a fundamental cognitive process facilitating the temporary storage, integration, manipulation, and retrieval of information, plays a vital role in reasoning and decision-making tasks. Robust benchmark datasets that capture the multifaceted nature of WM are crucial for the effective development and evaluation of AI WM models. Here, we introduce a comprehensive Working Memory (WorM) benchmark dataset for this purpose. WorM comprises 10 tasks and a total of 1 million trials, assessing 4 functionalities, 3 domains, and 11 behavioral and neural characteristics of WM. We jointly trained and tested state-of-the-art recurrent neural networks and transformers on all these tasks. We also include human behavioral benchmarks as an upper bound for comparison. Our results suggest that AI models replicate some characteristics of WM in the brain, most notably primacy and recency effects, and neural clusters and correlates specialized for different domains and functionalities of WM. In the experiments, we also reveal some limitations in existing models to approximate human behavior. This dataset serves as a valuable resource for communities in cognitive psychology, neuroscience, and AI, offering a standardized framework to compare and enhance WM models, investigate WM's neural underpinnings, and develop WM models with human-like capabilities. Our source code and data are available at


page 3

page 8

page 14

page 26

page 28


AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models

Evaluating the general abilities of foundation models to tackle human-le...

Toward reliable signals decoding for electroencephalogram: A benchmark study to EEGNeX

The development of brain-computer interfaces (BCI) has facilitated our s...

Human-Like Intuitive Behavior and Reasoning Biases Emerged in Language Models – and Disappeared in GPT-4

Large language models (LLMs) are currently at the forefront of intertwin...

Artificial Neuropsychology: Are Large Language Models Developing Executive Functions?

Artificial Intelligence (AI) has been rapidly advancing and has demonstr...

Cognitive Models as Simulators: The Case of Moral Decision-Making

To achieve desirable performance, current AI systems often require huge ...

Vision-based Behavioral Recognition of Novelty Preference in Pigs

Behavioral scoring of research data is crucial for extracting domain-spe...

Please sign up or login with your details

Forgot password? Click here to reset