Patch criterion + support for JMH
fosskers opened this issue · comments
Which is funny, since I'm on a Linux system. I got this error when trying to install criterion as a dependency for a test project:
Configuring unix-2.7.2.2...
Preprocessing library for unix-2.7.2.2..
etlas: can't find source for System/Posix/Resource in ., dist/build/autogen,
dist/build/global-autogen
etlas: Leaving directory '/tmp/etlas-tmp-13260/unix-2.7.2.2'
etlas: Error: some packages failed to install:
unix-2.7.2.2-EjsyPnKNuFmAYXDUkSFrP5 failed during the building phase. The
exception was:
ExitFailure 1
Hi, i am afraid that being jvm os-agnostic, eta cant build packages packages that depends heavily on native c code, like unix or win32, without patching them almost completely: see typelead/eta#376 (comment)
The workaround is replacing unix package calls in client packages like criterion with equivalent java ffi calls. We can take directory an example: https://github.com/typelead/eta-hackage/blob/master/patches/directory-1.3.1.0.patch#L224-L233
So, if you agree, we can add an issue about patching criterion itself to make it work with eta.
I usually reach for criterion for benchmarking Haskell code, but maybe there's an alternative that doesn't require FFI calls? What does Eta use for its own benchmarking?
The benchmarks of eta are in this project: https://github.com/typelead/eta-benchmarks
Afaik, as eta generates jvm bytecode the benchmarking, written in haskell, uses the java library jmh.
Maybe @rahulmutt could give you more precise info about them.
I'm going to try benchmarking some Eta functions with gauge first, we'll see how that goes. Gauge has a much smaller dependency graph.
Hm, when trying to compile gauge:
Configuring library for gauge-0.2.1..
Preprocessing library for gauge-0.2.1..
etlas: can't find source for Gauge/Source/RUsage in ., statistics, mwc-random,
math-functions, dist/build/autogen, dist/build/global-autogen
Hi, gauge have .hsc files (Gauge.Source.RUsage, Gauge.Source.Time) that includes c libraries so, although it has fewer haskell dependencies, it depends of c sources in exchange (including all from cbits folder!). I am afraid they should be replaced with java ffi calls too 😟
So maybe the right one could be create an etlas project with a external api compatible with the more used haskell library (criterion?) but using jmh uderneath, what do you think about @rahulmutt?
@jneira I agree with your suggestion about creating an API-compatible version of criterion that uses JMH.
Patched in 0eb84ca although without JMH bindings for now. Currently inflicted with a runtime bug:
typelead/eta#845
The runtime bug is fixed - criterion is working properly now. The next step is to work on integrating with JMH.
@rahulmutt does that mean criterion is usable as-is right now?
Yes it should be with 0.8.6b1. I've only tested the fib example so there may be other bugs you encounter as you try more advanced examples, so if anything seems wrong/unexpected, feel free to file an issue on the main repo and we will fix it asap.