Eratic partree benchmark runtime
toddaaro opened this issue · comments
I had some irregular results while running the benchmark script. The machine has a quad core i7 Nehalem. The benchmark itself is running in a Vmware virtual machine with 8 cores ( 1 per CPU thread ). Host OS is Win7 x64.
The partree benchmark usually takes under one second to run, but in this particular instance took over 40 seconds.
# TestName Variant NumThreads MinTime MedianTime MaxTime
#
# Tue Oct 4 12:25:17 PDT 2011
# Linux ubuntu 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
# Determined machine to have 8 hardware threads.
# The Glorious Glasgow Haskell Compilation System, version 7.0.3
#
# Running each test for 1 trials.
# ... with default compiler options: -O2 -rtsopts
# ... with default runtime options: -qa
# Using the following settings from the benchmarking environment:
# BENCHLIST= THREADSETTINGS=8 TRIALS= SHORTRUN=1 KEEPGOING= GHC= GHC_FLAGS= GHC_RTS=
# *** Config [0 ..], testing with command/args: parfib.exe
parfib.exe 8 0.00 0.00 0.00
# *** Config [1 ..], testing with command/args: parfib.exe
parfib.exe 8 0.00 0.00 0.00
# *** Config [2 ..], testing with command/args: blackscholes.exe
blackscholes.exe 8 0.21 0.21 0.21
# *** Config [3 ..], testing with command/args: cholesky.exe
cholesky.exe 8
# *** Config [4 ..], testing with command/args: nbody.exe
nbody.exe 8 0.03 0.03 0.03
# *** Config [5 ..], testing with command/args: mandel.exe
mandel.exe 8 0.10 0.10 0.10
# *** Config [6 ..], testing with command/args: coins.exe
coins.exe 8 0.94 0.94 0.94
# *** Config [7 ..], testing with command/args: queens.exe
queens.exe 8 0.22 0.22 0.22
# *** Config [8 ..], testing with command/args: partree/partree.exe
partree/partree.exe 8 40.33 40.33 40.33
# *** Config [9 ..], testing with command/args: matmult/matmult.exe
matmult/matmult.exe 8 0.28 0.28 0.28
# *** Config [10 ..], testing with command/args: sumeuler/sumeuler.exe
sumeuler/sumeuler.exe 8 0.93 0.93 0.93
The Strategies version of partree also gives erratic results, FWIW, but I've never seen this degree of variability.