Playing around with Gaussian Processes for Bayesian Optimization. This project was mostly to test to runtime performance of different approaches, and isn't a complete implementation (although with some effort it could be turned into one).
Pre-reqs are:
- Sbt
- Scala
- Java
- BLAS
Note: On a Mac Sbt and Scala can be brew installed, and Blas comes built in.
To run a very simple example app (see code here /core/src/scala/tomsopt/math/example/) that fits a simple function and then predicts new points and plots them.
sbt core/run
To run the unit tests:
sbt test
To run the benchmark suite (which tests 10,000,000 points and the calculates the point with highest expected improvement using both JNI and pure Java versions).
sbt bench
Last run of bench on my MacBook Pro (Retina, 15-inch, Mid 2015) had produced the following results (in seconds):
[info] Benchmark Mode Cnt Score Error Units
[info] EndToEnd.predictJava avgt 4 9.254 ± 2.149 s/op
[info] EndToEnd.predictNative avgt 4 6.306 ± 3.577 s/op
Bits still missing:
-
Kernel parameters are currently hardcoded in C versions (obviously these need to be passed in).
-
Kernel parameters aren't fit, would need to explore Breeze's optimization stuff (from quick look it seems straight forward)
-
There's no definition of what the feature space is. Features are assumed to be -inf to +inf range when sampling.
-
Doesn't support producing a batch of points to explore, would need to introduce Monte Carlo estimate of best
-
Haven't really checked anything for correctness, probably lots of bugs ;)