A Weyl Criterion for Finite-State Dimension
share
Finite-state dimension, introduced early in this century as a finite-state version of Hausdorff dimension, is a quantitative measure of the lower asymptotic density of information in an infinite sequence over a finite alphabet, as perceived by finite automata. It is a robust concept with equivalent formulations in terms of finite-state gambling, finite-state data compression, finite-state prediction, entropy rates, and Kolmogorov complexity. The 1972 Schnorr-Stimm dichotomy theorem gave the first automata theoretic characterization of normal sequences, which had been studied in number theory since Borel defined them in 1909. This theorem implies, in present-day terminology, that a sequence is normal if and only if it has finite-state dimension 1. One of the most powerful classical tools for investigating normal numbers is the 1916 Weyl criterion, which characterizes normality in terms of exponential sums. These are well studied objects with connections to other aspects of analytic number theory, and this has made the Weyl criterion especially fruitful. This raises the question whether the Weyl criterion can be generalized from finite-state dimension 1 to arbitrary finite-state dimensions, thereby making it a quantitative tool for studying data compression, prediction, etc. This paper does exactly this. We extend the Weyl criterion to characterize every finite-state dimension. This turns out not to be routine - exponential sums may diverge for non-normal numbers. But we characterize finite state dimension in terms of the dimensions of subsequence limits of the exponential sums. If the exponential sums converge, they converge to the Fourier coefficients of a probability measure whose dimension is precisely the finite state dimension of the sequence. We illustrate some applications, including a new, Fourier analytic, proof of Schnorr and Stimm's landmark result.
READ FULL TEXT