Structural Information Learning Machinery: Learning from Observing, Associating, Optimizing, Decoding, and Abstracting
share
In the present paper, we propose the model of structural information learning machines (SiLeM for short), leading to a mathematical definition of learning by merging the theories of computation and information. Our model shows that the essence of learning is to gain information, that to gain information is to eliminate uncertainty embedded in a data space, and that to eliminate uncertainty of a data space can be reduced to an optimization problem, that is, an information optimization problem, which can be realized by a general encoding tree method. The principle and criterion of the structural information learning machines are maximization of decoding information from the data points observed together with the relationships among the data points, and semantical interpretation of syntactical essential structure, respectively. A SiLeM machine learns the laws or rules of nature. It observes the data points of real world, builds the connections among the observed data and constructs a data space, for which the principle is to choose the way of connections of data points so that the decoding information of the data space is maximized, finds the encoding tree of the data space that minimizes the dynamical uncertainty of the data space, in which the encoding tree is hence referred to as a decoder, due to the fact that it has already eliminated the maximum amount of uncertainty embedded in the data space, interprets the semantics of the decoder, an encoding tree, to form a knowledge tree, extracts the remarkable common features for both semantical and syntactical features of the modules decoded by a decoder to construct trees of abstractions, providing the foundations for intuitive reasoning in the learning when new data are observed.
READ FULL TEXT