SEAM: An Integrated Activation-Coupled Model of Sentence Processing and Eye Movements in Reading

by   Maximilian M. Rabe, et al.

Models of eye-movement control during reading, developed largely within psychology, usually focus on visual, attentional, and motor processes but neglect post-lexical language processing; by contrast, models of sentence comprehension processes, developed largely within psycholinguistics, generally focus only on post-lexical language processes. We present a model that combines these two research threads, by integrating eye-movement control and sentence processing. Developing such an integrated model is extremely challenging and computationally demanding, but such an integration is an important step toward complete mathematical models of natural language comprehension in reading. We combine the SWIFT model of eye-movement control (Engbert et al., Psychological Review, 112, 2005, pp. 777-813) with key components of the Lewis and Vasishth sentence processing model (Lewis and Vasishth, Cognitive Science, 29, 2005, pp. 375-419). This integration becomes possible, for the first time, due in part to recent advances in successful parameter identification in dynamical models, which allows us to investigate profile log-likelihoods for individual model parameters. We present a fully implemented proof-of-concept model demonstrating how such an integrated model can be achieved; our approach includes Bayesian model inference with Markov Chain Monte Carlo (MCMC) sampling as a key computational tool. The integrated model, SEAM, can successfully reproduce eye movement patterns that arise due to similarity-based interference in reading. To our knowledge, this is the first-ever integration of a complete process model of eye-movement control with linguistic dependency completion processes in sentence comprehension. In future work, this proof of concept model will need to be evaluated using a comprehensive set of benchmark data.


page 1

page 2

page 3

page 4


Integrating large language models and active inference to understand eye movements in reading and dyslexia

We present a novel computational model employing hierarchical active inf...

The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts

Eye movement recordings from reading are one of the richest signals of h...

A computational investigation of sources of variability in sentence comprehension difficulty in aphasia

We present a computational evaluation of three hypotheses about sources ...

A Discriminative Model for Identifying Readers and Assessing Text Comprehension from Eye Movements

We study the problem of inferring readers' identities and estimating the...

Feature overwriting as a finite mixture process: Evidence from comprehension data

The ungrammatical sentence "The key to the cabinets are on the table" is...

Bayesian Dynamical Modeling of Fixational Eye Movements

Humans constantly move their eyes, even during visual fixations, where m...

Please sign up or login with your details

Forgot password? Click here to reset