Learning curves for multi-task Gaussian process regression

by   Simon R. F. Ashton, et al.

We study the average case performance of multi-task Gaussian process (GP) regression as captured in the learning curve, i.e. the average Bayes error for a chosen task versus the total number of examples n for all tasks. For GP covariances that are the product of an input-dependent covariance function and a free-form inter-task covariance matrix, we show that accurate approximations for the learning curve can be obtained for an arbitrary number of tasks T. We use these to study the asymptotic learning behaviour for large n. Surprisingly, multi-task learning can be asymptotically essentially useless, in the sense that examples from other tasks help only when the degree of inter-task correlation, ρ, is near its maximal value ρ=1. This effect is most extreme for learning of smooth target functions as described by e.g. squared exponential kernels. We also demonstrate that when learning many tasks, the learning curves separate into an initial phase, where the Bayes error on each task is reduced down to a plateau value by "collective learning" even though most tasks have not seen examples, and a final decay that occurs once the number of examples is proportional to the number of tasks.


page 1

page 2

page 3

page 4


Ensemble Multi-task Gaussian Process Regression with Multiple Latent Processes

Multi-task/Multi-output learning seeks to exploit correlation among task...

Asymptotic performance of regularized multi-task learning

This paper analyzes asymptotic performance of a regularized multi-task l...

Replica theory for learning curves for Gaussian processes on random graphs

Statistical physics approaches can be used to derive accurate prediction...

Time-Varying Transition Matrices with Multi-task Gaussian Processes

In this paper, we present a kernel-based, multi-task Gaussian Process (G...

Enrichment Score: a better quantitative metric for evaluating the enrichment capacity of molecular docking models

The standard quantitative metric for evaluating enrichment capacity know...

Estimating Causal Effects using a Multi-task Deep Ensemble

Over the past few decades, a number of methods have been proposed for ca...

Learning Multi-Task Gaussian Process Over Heterogeneous Input Domains

Multi-task Gaussian process (MTGP) is a well-known non-parametric Bayesi...

Please sign up or login with your details

Forgot password? Click here to reset