Coding-theorem Like Behaviour and Emergence of the Universal Distribution from Resource-bounded Algorithmic Probability
share
Previously referred to as 'miraculous' because of its surprisingly powerful properties and its application as the optimal theoretical solution to induction/inference, (approximations to) Algorithmic Probability (AP) and the Universal Distribution are of the greatest importance in computer science and science in general. Here we investigate the emergence, the rates of emergence and convergence, and the Coding-theorem like behaviour of AP in subuniversal models of computation. We investigate empirical distributions of computer programs of weaker computational power according to the Chomsky hierarchy. We introduce measures of algorithmic probability and algorithmic complexity based upon resource-bounded computation, in contrast to previously thoroughly investigated distributions produced from the output distribution of Turing machines. This approach allows for numerical approximations to algorithmic (Kolmogorov-Chaitin) complexity-based estimations at each of the levels of a computational hierarchy. We demonstrate that all these estimations are correlated in rank and that they converge both in rank and values as a function of computational power, despite the fundamental differences of each computational model. In the context of natural processes that may operate below the Turing universal level due to the constraint of resources and physical degradation, the investigation of natural biases coming from algorithmic laws is highly relevant. We show that the simplicity/complexity bias in distributions produced even by the weakest of the computational models can be accounted up to 60 Universal Distribution.
READ FULL TEXT