Contextuality from missing and versioned data
share
Traditionally categorical data analysis (e.g. generalized linear models) works with simple, flat datasets akin to a single table in a database with no notion of missing data or conflicting versions. In contrast, modern data analysis must deal with distributed databases with many partial local tables that need not always agree. The computational agents tabulating these tables are spatially separated, with binding speed-of-light constraints and data arriving too rapidly for these distributed views ever to be fully informed and globally consistent. Contextuality is a mathematical property which describes a kind of inconsistency arising in quantum mechanics (e.g. in Bell's theorem). In this paper we show how contextuality can arise in common data collection scenarios, including missing data and versioning (as in low-latency distributed databases employing snapshot isolation). In the companion paper, we develop statistical models adapted to this regime.
READ FULL TEXT