Zeta Distribution and Transfer Learning Problem

06/23/2018
by   Eray Özkural, et al.
0

We explore the relations between the zeta distribution and algorithmic information theory via a new model of the transfer learning problem. The program distribution is approximated by a zeta distribution with parameter near 1. We model the training sequence as a stochastic process. We analyze the upper temporal bound for learning a training sequence and its entropy rates, assuming an oracle for the transfer learning problem. We argue from empirical evidence that power-law models are suitable for natural processes. Four sequence models are proposed. Random typing model is like no-free lunch where transfer learning does not work. Zeta process independently samples programs from the zeta distribution. A model of common sub-programs inspired by genetics uses a database of sub-programs. An evolutionary zeta process samples mutations from Zeta distribution. The analysis of stochastic processes inspired by evolution suggest that AI may be feasible in nature, countering no-free lunch sort of arguments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2016

Learning Bound for Parameter Transfer Learning

We consider a transfer-learning problem by using the parameter transfer ...
research
10/14/2018

Theoretical Guarantees of Transfer Learning

Transfer learning has been proven effective when within-target labeled d...
research
06/23/2020

Limits of Transfer Learning

Transfer learning involves taking information and insight from one probl...
research
09/03/2021

A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms

Transfer learning is a machine learning paradigm where knowledge from on...
research
05/04/2021

Online Transfer Learning: Negative Transfer and Effect of Prior Knowledge

Transfer learning is a machine learning paradigm where the knowledge fro...
research
12/24/2018

Motion Blur removal via Coupled Autoencoder

In this paper a joint optimization technique has been proposed for coupl...
research
05/08/2022

Data-Driven Approximations of Chance Constrained Programs in Nonstationary Environments

We study sample average approximations (SAA) of chance constrained progr...

Please sign up or login with your details

Forgot password? Click here to reset