Modernizing Latent Gaussian Process Inference for Non-Gaussian Responses

Abstract

Gaussian processes (GPs) are powerful tools for modeling non-linear data. In many situations, however, direct GP inference is not possible due to the nature of the response. Categorical and directional data, for instance, are examples of responses for which a Gaussian likelihood assumption is not appropriate. Latent GPs, typically in tandem with appropriate link functions, can be introduced to model responses with non-Gaussian likelihoods. But latent GPs do not scale well to large training data, especially when Monte Carlo integration is required. Consequently, fully Bayesian, sampling-based approaches have been largely abandoned in favor of maximization-based alternatives, such as Laplace/variational inference (VI) combined with low rank approximations. Though feasible for large training data sets, such schemes sacrifice uncertainty quantification and modeling fidelity, two aspects that are important to mu work on surrogate modeling of computer simulation experiments. In this work I propose a GP inference framework that takes advantage of a remarkably powerful rejection sampling approach known as elliptical slice sampling (ESS). My approach allows for computationally thriftier posterior integration while preserving fully Bayesian inference. I leverage this framework in the contexts of classification and circular modeling, both of which introduce unique latent inferential challenges. I demonstrate superiority over VI-based alternatives for both real and simulated examples, including a Binary Black Hole simulator (binary response) and data from a radio frequency identification experiment (angular response).