Deep Bayesian Supervised Learning given Hypercuboidally-shaped, Discontinuous Data, using Compound Tensor-Variate & Scalar-Variate Gaussian Processes
share
We undertake learning of the high-dimensional functional relationship between a system vector and a tensor-valued observable, where this observable affects the system parameter. The final aim is to undertake Bayesian inverse prediction of system parameter values, at which test data on the observable is reported. We attempt such learning given hypercuboidally-shaped data comprising multiple measurements of the observable, where the data displays strong discontinuities. We undertake modelling of the sought functional relationship, with a tensor-variate Gaussian Process (GP) and use three independent ways for learning covariance matrices of the resulting Tensor-Normal likelihood. We demonstrate that when the covariance matrix is kernel parametrised, the discontinuous nature of the data demands the implemented kernel to be non-stationary, which we achieve by modelling each kernel hyperparameter, as a function of the sample function of the invoked tensor-variate GP. This translates to a dynamically varying function, within our Bayesian inference scheme, with this function treated as a realisation from a scalar-variate GP, the covariance structure of which is described adaptively by collating information from a historical set of samples. Parameters of the scalar-variate GPs, are updated first in our Metropolis-within-Gibbs scheme, thus allowing kernel hyperparameters to be updated; remaining parameters of the tensor-variate GP are then updated. Thus, the functional relation between the system parameter and observable is learnt, and subsequently, we perform inverse Bayesian prediction. We apply our method to a cuboidally-shaped, discontinuous, real dataset, while undertaking forward prediction to generate data from our model, given our results--to perform model-checking.
READ FULL TEXT