Multi-Stage Modeling with Gaussian Processes

Abstract

Gaussian processes (GPs) furnish accurate nonlinear predictions with well-calibrated uncertainty. However, GPs alone are not always flexible enough to model complex real-world phenomena. To remedy this issue, it is beneficial to chain together multiple models. These so-called multi-stage models fit models in sequence, using output from one model as training data in the next model. I utilize multi-stage modeling with GPs to achieve two tasks. First, I improve GP prediction accuracy for data from processes with sudden changes, or "jumps,"' in the output variable by creating a new cluster-based (latent) feature and adding it to the input matrix. Then, I fit a GP to understand the circumstances under which a machine learning (ML) model performs best, and use that GP to propose the best circumstances in which to make future data acquisitions. I do this by treating the composition of metadata and the performance of the ML model, respectively, as the inputs and output of a GP. I vet both methods on a selection of real and synthetic benchmark examples from the recent literature.