A Julia implementation of Posterior and Computational Uncertainty in Gaussian Processes.
IterGP uses the AbstractGPs interface.
using Random
using Plots
using KernelFunctions
using AbstractGPs
using IterGP
# Generate some toy data
n = 50
σ = 0.1
rng = Xoshiro(1234)
noise = randn(rng, n).*σ
x = rand(rng, n) .*6 .+ 0.1.*randn(rng)
y = sin.(x) .+ noise
# Set up a vanilla AbstractGPs.GP
xx = collect(1.4 .* range(extrema(x)..., 200))
k = Matern32Kernel()
f = GP(k)
fx = f(x, σ^2)
# Cholesky actions
maxiters = 15
chol_pf = posterior(fx, y, CholeskyPolicy(length(x), maxiters))
# Conjugate gradient actions
maxiters = 15
x0 = zeros(length(x))
cg_pf = posterior(fx, y, ConjugateGradientPolicy(x0, maxiters))
plt = plot(xx, chol_pf, label="Cholesky", title="Posterior after $maxiters iterations", color=1)
plot!(plt, xx, rand(rng, chol_pf(xx), 5), color=1, label=nothing, alpha=0.5)
plot!(plt, xx, cg_pf, label="CG", title="Posterior after $maxiters iterations", color=3)
plot!(plt, xx, rand(rng, cg_pf(xx), 5), color=3, label=nothing, alpha=0.5)
scatter!(plt, x, y, label="Data", color=2)
The IterGP API builds around AbstractGPs.AbstractGP, and exposes policies that can be used to call IterGP.posterior, which is where the magic happens.
As a way to cache computations (such as preconditioners), the policy object itself is not called in the innermost loop. Instead, the policy is used to create an /actor/ which is then called in the inner loop until a convergence criteria is met.
Apart from the actual paper, these lecture notes (Scaling GPs), (Computation aware GPs) explain a lot of the implementation.